This guide covers how to validate Salesforce objects, uploads, and updates using DataOps Suite. It explains how Metadata Validation keeps Dev and Production Salesforce schemas in sync, catching unintended changes to fields, page layouts, or workflow rules before deployment. It also covers common upload issues (formatting, integrity, permissions), and how the Data Compare Node determines which records need “upsert” (insert/update) versus “update” operations by identifying differences between Salesforce objects and source files.
Key Takeaways
- Dev and Prod metadata often drift apart — testing-related changes to fields, page layouts, workflow rules, or custom objects in a sandbox environment can unintentionally differ from production if not validated before deployment.
- Metadata Validation Node keeps schemas in sync — DataOps Suite can directly compare Dev and Prod Salesforce schemas to catch structural mismatches before they cause issues.
- Data Compare Node validates uploads against source files — it pulls data directly from Salesforce objects and compares it to an on-premise file or dataset to catch formatting, integrity, and permission issues.
- Choosing upsert vs. update requires precision — using upsert on duplicate external IDs updates existing records instead of inserting new ones, so DataOps Suite uses Data Compare results to correctly split records into “new” (upsert) and “existing” (update) groups.
Intro to Salesforce
Salesforce is a cloud-based CRM platform businesses use to manage and analyze customer interactions and data throughout the customer lifecycle — storing contact details, communication history, and preferences to build a complete view of each customer. This helps businesses understand customer needs and personalize their interactions. Validating that this data is accurate and consistent, across sandboxes, uploads, and updates, is what this post covers.
This makes data quality inside Salesforce especially high-stakes. According to Salesforce’s own research, a little more than one-fifth of sales reps’ time is spent researching bad data — time that validated objects, uploads, and updates are meant to give back.
In addition to its CRM capabilities, Salesforce also provides a range of tools and features for data management and integration, including data import and export, data modeling, and data governance. This makes it possible for businesses to manage and analyze their customer data in a centralized location and to integrate it with other systems and applications.
Production vs Development – Metadata Validation
In a Salesforce deployment, it is common for there to be differences in metadata between the development environment (also known as a sandbox or dev environment) and the production environment (also known as prod). This is because the development environment is often used for testing and experimentation, which can result in changes to the metadata that are not intended for the production environment.
Some examples of metadata changes that may occur in the development environment but not be intended for the production environment include:
Modifying the structure of objects or fields: This could involve adding or deleting fields, or changing field data types. For example, a developer may be testing a new feature that requires adding a new field to the Account object to store additional data. If this field is not needed in the production environment, it would be important to remove it before deploying the changes to prod.
Changing page layouts or field-level security settings: This could involve modifying the layout of a page to display new fields or rearranging existing ones, or changing the visibility of fields based on user roles or profiles. For example, a business user may be testing a new page layout for the Account object in the dev environment, but this layout may not be ready for production.
Modifying workflow rules or approval processes: This could involve adding or modifying rules that trigger actions based on certain conditions, or changing the steps or participants in an approval process. For example, a developer may be testing a new workflow rule in the dev environment that sends an email notification when an Account is created, but this rule may not be ready for production.
Adding or modifying custom objects or custom fields: This could involve creating new objects to store custom data or adding new fields to existing objects. For example, a business user may be testing a new custom object in the dev environment to track project tasks, but this object may not be ready for production.
These changes may occur in the development environment for a variety of reasons. For example, a developer may be testing a new feature or functionality and need to make changes to the metadata to support it. Or, a business user may be exploring different options for customizing the Salesforce instance and may make a series of changes as they iterate on their design.
Here, the Metadata Validation Node of the DataOps Suite. can be used to ensure the metadata of the Dev and Prod Objects are in sync
A Metadata Validation Node against Dev and Prod Salesforce Schemas
Upload Validation
Several distinct types of issues can arise while uploading data to Salesforce, and which ones you hit often depends on the type of data, its source, and the Salesforce object you’re uploading to. The four most common categories are:
If the data you are uploading is not properly formatted, it may not be accepted by Salesforce. For example, if you are uploading a CSV file and the data in the file is not properly structured, Salesforce may not be able to parse the data correctly.
If the data you are uploading contains errors or inconsistencies, it may cause issues with the integrity of your Salesforce data. For example, if you are uploading a list of leads and some of the leads are missing required fields, the upload may fail.
Each Salesforce object has its own set of fields and requirements, and if the data you are uploading does not meet these requirements, the upload may fail. For example, if you are uploading data to the Account object and the data does not contain a value for the required “Name” field, the upload may fail.
If you do not have the correct permissions in Salesforce, you may not be able to upload data to certain objects or fields. For example, if you are trying to upload data to a custom object that you do not have permission to access, the upload may fail.
The Data Compare Node can be used to pull data directly from the Salesforce Object and compare it against the file or dataset on premise as seen below.
A Basic Data Compare checking salesforce against a dataset
Upsert vs Update
In this case, you may want to use the update function instead of the upsert function. The update function allows you to specify a query to select the records you want to modify, rather than relying on the external ID field to identify matching records. This can be useful if you want to update records based on criteria other than the external ID, or if you want to insert records as new records rather than updating existing ones.
There are several issues that can arise while uploading data to Salesforce, and the specific issues you may encounter can depend on the type of data you are uploading, the source of the data, and the type of Salesforce object you are uploading to. Here are a few examples of issues that can arise while uploading data to Salesforce:
Upsert
The “upsert” function is used to either update existing records or create new records in an object, depending on whether a matching record already exists. When using the upsert function, you specify a field in the object that will be used as the unique identifier, called the “external ID”. If a record with a matching external ID already exists, the upsert function will update that record with the new data. If no matching record is found, the upsert function will create a new record with the provided data.
Update
The “update” function is used to modify existing records in an object. When using the update function, you specify the records that you want to update using a query, and then specify the new field values that you want to set for those records. The update function will only modify records that already exist in the object, and will not create new records.
Both the upsert and update functions can be used to modify a single record or multiple records at once. They can be useful for updating or creating records in bulk, or for keeping data in Salesforce synchronized with data from other sources.
To solve the complex job of figuring out which records to use with Upsert and which to use Upload, the Data Compare Node comes in handy once again. Every node in the DataOps Suite has its results, and comparisons saved as views that can be called upon internally. This allows for reference and loops of the system to ensure complex solutions can be easily defined and solved in the Suite.
In this case, after defining the data comparison between the Salesforce object and the CSV file, the node generates a set of related views, including records that exist only in the Salesforce object, records that exist only in the CSV file, records identified as different, and others. Our focus is on the dataset containing the differences and the dataset containing records that exist only in the CSV file. The dataset with records found only in the CSV file contains records that are not present in the Salesforce object and therefore requires the upsert function to create these new records in the Salesforce object.
The other dataset contains records that already exist in the Salesforce object but require updates to specific value sets. In this case, the update function is used along with prefixed Python code to identify the exact values that need to be updated.
The different views created by the Data Compare Node and the corresponding upsert and update nodes.
Note that distinguishing between the addition of a new external ID field and updating an existing record is a highly quintessential task especially when the objects are called by CRM and reporting tools themselves where the difference between a new record and an updated record is really critical.
Conclusion
In conclusion, it is important to carefully validate Salesforce objects, uploads, and updates to ensure that your data is accurate and consistent. By following best practices and using the appropriate tools and techniques, such as the DataOps Suite, you can avoid common issues such as data formatting errors, data integrity problems, and object-specific issues. Whether you are working with Veeva objects, pre-sales CRM objects, or any other type of object in Salesforce, taking the time to validate your data will help you maintain the quality and reliability of your Salesforce data.
FAQs: Salesforce Metadata Validation and Data Deployment
1) Why do Salesforce Dev and Production environments sometimes fall out of sync?
Differences can arise when changes made in a Salesforce sandbox environment—such as updates to fields, page layouts, workflow rules, or custom objects—are not fully validated before deployment. These inconsistencies can lead to unexpected behavior in the Production environment.
2) How does Metadata Validation help with Salesforce deployments?
The Metadata Validation Node in DataOps Suite compares metadata between Salesforce Development and Production environments to identify schema differences before deployment, reducing the risk of failed releases and configuration issues.
3) What issues commonly occur during Salesforce data uploads?
Salesforce data uploads can fail because of formatting inconsistencies, data integrity violations, validation rule failures, or insufficient user permissions. Identifying these issues before loading data helps ensure successful updates.
4) How does DataOps Suite decide whether to upsert or update a Salesforce record?
Using the Data Compare Node, DataOps Suite compares Salesforce object data with a source dataset to determine whether a record already exists. Records with matching external IDs are updated, while new records are upserted, helping prevent duplicate data and ensuring accurate synchronization.
Rajesh Kumar A
Digital Marketing Manager, Datagaps
Digital Marketing Manager at Datagaps. Drives data-driven growth through content, performance campaigns, and marketing technology.
S P S Murthy Akella
Director, Technology Strategy, Datagaps
Director of Technology Strategy at Datagaps. Business solutions architect and Certified Scrum Master in data engineering, responsible AI, and ML across BFSI, telecom, aviation, and energy.
