DataOps Suite
Data Quality and Agentic AI platform that validates, reconciles, scores, monitors and remediates data from ingestion to BI and AI.
Why one platform
DataOps Data Quality
Continuously profiles, validates and scores enterprise data, with Agentic AI that recommends rules and flags anomalies, so analytics and AI run on trusted data.
Capabilities
ETL Validator
Automates ETL/ELT testing, with Agentic AI that builds tests from mapping docs, confirming every record arrives complete, accurate and correctly transformed.
Capabilities
BI Validator
Automates testing and governance for Power BI, Tableau and Oracle Analytics, with Agentic AI that builds tests from plain English and keeps reports accurate and reliable.
Capabilities
Test Data Manager
Uses Agentic AI to detect and mask sensitive data and generate synthetic data that mirrors production, so teams can test without exposing real records.
Capabilities
Data Quality Use Case
Start from the outcome your team needs, with Agentic AI doing the groundwork.
Cloud Platforms Use Case
Validate and monitor data and its quality on the platforms you already run, backed by Agentic AI.
BI and Analytics Use Case
Keep every report accurate across your BI tools, as Agentic AI builds the tests for you.
Banking, Financial Services, and Insurance (BFSI)
Stop reporting errors before examiners find them, with every number traced from core system to board dashboard.
How Datagaps helps
Healthcare
Keep claims and clinical data accurate through every migration, without exposing patient records in testing.
How Datagaps helps
Life Sciences
Catch issues in clinical and real-world data before submission, and keep trial subject data out of test environments.
How Datagaps helps
Manufacturing
Avoid broken ERP cutovers and supplier overpayments, and catch sensor drift before it hits production.
How Datagaps helps
Retail and Utilities
Catch settlement, inventory and pricing errors before your customers or regulators do.
How Datagaps helps
Higher Education & Research
Get enrollment, financial-aid and transcript data right before SIS cutover, before students feel it.
How Datagaps helps
Trusted Data Summit
A practitioner summit focused on the validation, observability, governance, and reconciliation practices that determine whether enterprise data is ready for AI.
Learn
Customers and tools
Support
Automating Financial and Production Data Reconciliation for a U.S. Property Insurer
A major U.S. property insurer managing 600 to 800 data interfaces and pipelines across Oracle, Oracle PeopleSoft, Guidewire, and AWS
Datagaps
Get in touch
Why teams trust Datagaps
100+ Enterprise Customers
Datagaps partners with Vega IT
Bringing trusted data foundations to digital and AI transformation.
Technology Partners
Datagaps works natively with the cloud, data and BI platforms your teams already run.
Our technology partners
SI Partners
Consulting and integration firms that deliver Datagaps validation for enterprise clients worldwide.
Our SI partners
Platform
Solutions
Use cases
Cloud platforms
BI and analytics
Industries
Trusted Data Summit
A practitioner summit focused on the validation, observability, governance, and reconciliation practices that determine whether enterprise data is ready for AI.
Resources
Learn
Customers and tools
Support
Company
Datagaps
Get in touch
Getting Started โ Sandbox Documentation
Welcome to the DataOps Suite Sandbox. Before starting with the specific use cases you want, we highly recommend watching this below video and documentation. This introduction will start with an overview of how test cases are created from scratch, followed by a few exercises. The solutions of the exercises are already present in the Sandbox for reference.
DataOps Suite has a full-fledged ETL tool as its backbone to support its ETL Validation capabilities. Thus, it helps to look at the product as kin to typical ETL tools.
Firstly, a source and target connection need to be defined to pull and push datasets with the Suite.
These connections are defined as โSourcesโ in the Sources Tab and can range from JDBC Connections to Flat Files to APIs and much more.
Once the connections are defined, data can be pulled into Suite and then transformed, tested, compared, profiled, analyzed, and/or pushed back into a different target. This is achieved in the Dataflows Tab. After test cases (dataflows) are created, they can be bundled together, sequenced, and ran as Pipelines in Pipeline Tab. The Scheduler helps you schedule the pipeline run. All dataflow and pipeline runs have a History section to look back at previous runs.
For these, we highly recommend tinkering with the application, going through the documentation, and connecting with our support desk (accessible via the application). We also provide a full fledge training certification for the Suite.
Before testing specific use cases, kindly attempt the following exercises in the order provided to understand the Suite better. We will be working with the following business use cases as the basis of our exercises.
Our architecture involves a SQL Server, an Oracle DB, and an AWS s3 as our main 3 data sources. We want to perform the following validation โ
Note: The first step of the process, the creation of Sources, is already done in the Sandbox.
Context โ Validation of migrated assets is a primary use case in many migrations and DQ processes.
The Dimension Table โDimProductโ was transferred from an Oracle DB to a SQL Server. We need first to validate the metadata of both tables; then, we need to validate all the records of both tables against each other.
Create a new dataflow within the Dataflow Tab using the โCreate Dataflowโ button (top-right).
Nodes used โ JDBC Connector, Metadata Compare, Data Compare
Steps โ
Create a new dataflow within the Dataflow Tab using the โCreate Dataflowโ button (top-right).
Nodes used โ File Connector, Data Rule
Steps โ
Context โ Based on business requirements, the DimEmployee table has to be updated. The mappings are provided on the right. A developer already implements the changes, and the updated dataset is saved as a file named โDim_Employee_Updatedโ in AWS S3. You are a QA tester and will perform white box testing to validate this transformation. Using the mapping spreadsheet, create an updated version based on your implementation of the raw data from the SQL Server and compare it to the updated CSV file to validate the transformation.
Create a new dataflow within the Dataflow Tab using the โCreate Dataflowโ button (top-right).
Nodes used โ File Connector, JDBC Connector, SQL
The sandbox also comes with a Data Model in the Data Quality Tab with rules already defined test cases to help new users see how business rules and data quality test cases can be defined. New users are encouraged to explore DQM by โ
These activities are left for the user as an exercise. The video can be used as a reference.
Thank you for completing the exercises. With the basics covered, you and your team should understand the plethora of nodes and toolsets the application provides better. Feel free to tinker with the environment to fit your specific use cases and explore all the nodes and applications. The application can also be connected via CLI Commands, REST APIs, and integrated plugins. Our documentation, which can be easily accessed in the bottom left corner of the Suite, will provide your team with ample starting resources. The application comes with wizards, pipelines, and monitoring capabilities that new users can easily configure and deploy. In case of any specific examples, feel free to contact us directly.
In case of any questions or errors that your team cannot resolve, feel free to contact our support team either by contacting us on our email or using the support portal present in the bottom left corner of the Suite.
By subscribing youโre allowing data graphs and/or its associates to reach you with periodic informative updates.