Datagaps’ DataOps Suite Version 2022.5.0 focuses on integration and simplified analysis. Key updates include an API Component for pulling and validating REST API data, a Plugin Component supporting Python/Scala for extensibility, a Metrics Compare Component for parallel cross-system comparisons, and Data Profile enhancements for automatic anomaly detection. The release also adds inline anomaly prediction, a no-code SQL Query Builder, Azure AD SSO support, and improvements to Data Rules, Test Cases, and Test Data Manager (TDM).
Key Takeaways
- New API Component pulls data directly from REST APIs, automatically breaking down complex/hierarchical JSON responses into structured, queryable datasets for validation and comparison.
- New Plugin Component adds extensibility through custom Python or Scala scripts, letting teams create reusable, parameterized methods across DataFlows instead of duplicating logic.
- Data Profile enhancements introduce automatic anomaly detection, flagging unexpected shifts in record counts, null percentages, or mean values compared to historical profiles.
- No-code SQL Query Builder and a new Metrics Compare Component make query creation and cross-system comparisons easier, alongside Azure AD SSO support and improvements to Data Rules, Test Cases, and Test Data Manager.
Datagaps Inc. announced its latest product feature updates for the DataOps Suite Version 2022.5.0 – The theme of this update is Integration and Easy to apply Analysis. If you are interested in getting a demo of DataOps Suite, you can fill your details here.
DataOps Suite Top Feature Updates – Version 2022.5.0
REST APIs are being used extensively for Application development as well as a means of integration between systems. DataOps suite now supports API as a data source and the corresponding API Component in DataFlow can be used to call the REST API and automatically convert the response into datasets for further validation and processing.
Since the REST API responses can be deeply hierarchical JSON Nodes, DataOps suite automatically splits the data into separate datasets with relations so that anyone with SQL query knowledge can easily query the REST API output and validate the data by comparing it with data from another data source.
Plugin
Component
Plugin component adds extensibility to DataOps suite. The plugin system lets users define methods with user-defined parameters as inputs which once defined, can be called on with a few clicks by the end user. Plugins can be written in Python or Scala. Once configured, they can be used within the DataFlow without the need for copy-pasting the code thus promoting code re-use.
One of the example plugin we recently created was to read multiple sheets of excel (usually reports received by mail) as separate datasets to be used for comparisons.
Metrics Compare Component
Metrics component makes it easy to compare metrics such as count, sum etc. across different systems. When it comes to production data monitoring, it is a common requirement to compare the counts of source and target records to validate that the ETL job processes all the records as expected.
Another feature of Metrics component is that it can be used to compare multiple metrics in a single component. These metrics can be executed in parallel for better performance.
Data Profile Component
Data Profile component has been available in DataOps suite for a very long time. It takes a dataset as an input and automatically computes aggregates such as maximum value, minimum value, minimum length, maximum length, Null (%), distinct (%), distinct count, Null count, mean, sum, row count, Constancy, Kurtosis, Skewness, and standard deviation.
In this release, Data Profile component can be used to identify anomalies in data automatically based on the historical data profile values. For example, if an ETL job is ingesting 1000 records on daily basis and if one day it only processed 100 records, Data Profile component will automatically flag that as an anomaly.
This observability feature of Data Profile component can be used to automatically detect anomalies in other data profile values such as % of Nulls in a column or Mean value.
Data Observability Improvements
Data Analysis component automatically learns from historical data and identifies anomalies in the latest data. Within the realms of data validation, anomaly detection plays one of the most crucial roles as it is not defined with strict logic or rules but with patterns and aggregates. Some of the prediction methods supported include time series based machine learning models as well statistical methods such as IQR and Standard Deviation.
Version 2022.5.0 simplifies anomaly detection by providing the option to do inline prediction. The results screen was also improved to show anomalies that were identified during the run.
SQL Query Builder
The SQL Query Builder option simplifies the generation of SQL queries from the data model without writing the query manually. It has three sections: Entities, Columns, and Conditions.
- The Entities section displays the tables and their columns corresponding to the schema of the selected data model.
- The Columns section displays the columns that will be used to generate the SQL query with or without the usage of the aggregate function.
- The Conditions section allows the user to generate the WHERE clause condition as per the end user’s requirement.
In addition to above-mentioned updates, a lot of additions and enhancements were made to Data Rules, Test Cases, Test Data Manger (TDM) and CLI. Version 2022.5.1 also adds support for Azure AD SSO integration.
Conclusion
Version 2022.5.0 pushes DataOps Suite further as an integration hub, not just a testing tool — connecting to REST APIs, extending logic through custom Python/Scala plugins, and automatically flagging anomalies against historical data profiles. Combined with the no-code SQL Query Builder and faster cross-system metric comparisons, this release reduces the manual setup work that typically slows down validation projects. For teams juggling multiple data sources and formats, these updates mean fewer custom scripts and less repetitive configuration — letting testers focus on defining what “correct” looks like rather than wiring up the connections to check it.
DataOps Suite – Free Trial
Try DataOps Suite Free for 14 days…
Frequently Asked Questions: DataOps Suite Version 2022.5.0
1) What is the theme of DataOps Suite Version 2022.5.0?
The release centers on integration and easy-to-apply analysis, adding new components for connecting external systems and simplifying data validation workflows.
2) What does the new API Component do?
It lets DataOps Suite use REST APIs as a data source, automatically converting API responses into datasets, including splitting hierarchical JSON into related datasets that can be queried with SQL and compared against other sources.
3) How does the Plugin Component add extensibility?
It lets users write custom methods in Python or Scala with user-defined parameters, which can then be reused across DataFlows without copy-pasting code — for example, reading multiple Excel sheets as separate datasets.
4) What’s new in the Data Profile Component?
It can now automatically flag anomalies based on historical data profile values — for instance, catching a sudden drop in record counts or unusual changes in null percentages or mean values.
5) Does this version include a no-code way to build SQL queries?
Yes — the new SQL Query Builder generates queries through three sections (Entities, Columns, Conditions) without requiring users to write SQL manually.
Rajesh Kumar A
Digital Marketing Manager, Datagaps
Digital Marketing Manager at Datagaps. Drives data-driven growth through content, performance campaigns, and marketing technology.
Subrahmanya Narayana Chirravuri
Senior Director, Technology, Datagaps
Senior Director of Technology at Datagaps. Leads engineering for the ETL, BI, and data-quality validation platforms.
