Data Quality Dimensions provide a framework to categorize validation rules and measure data quality across seven areas: Completeness, Conformity, Validity, Accuracy, Uniqueness, Consistency, and Timeliness. DataOps Suite computes Gen AI Data Quality Scores for each rule, rolling results up to the Table, Data Model, and System level. Scores are calculated as [1 – (bad records / total records)] x 100, supported by rule types including SQL Query, SQL Expression, Duplicate Check, Foreign Key, Domain, Attribute, Dataset, and Delta Rules.
Key Takeaways
- Seven dimensions define data quality — Completeness, Conformity, Validity, Accuracy, Uniqueness, Consistency, and Timeliness together provide a structured way to categorize and measure data validation rules.
- Data Quality Score has a simple formula — calculated as [1 – (number of bad records / total records)] x 100, giving a clear percentage score that’s easy to track over time.
- Scores roll up across three levels — individual rule scores aggregate to Table, Data Model, and System-level views, giving visibility from granular checks to enterprise-wide data health.
- Eight rule types support dimension-based validation — including SQL Query, SQL Expression, Duplicate Check, Foreign Key, Domain, Attribute, Dataset Rules, and Delta Rules, plus Data Reconciliation tests.
AI Data Quality Dimensions
“A recent survey by TDWI found that 66% of organizations are looking for ways to improve data quality and trust. Data Validation using Data Quality testing tools such as Datagaps DataOps suite is essential to ensure trust in your data and analytics.”
The Data Quality dimensions provide a way to categorize data validation rules and measure data quality. There are seven data quality dimensions that are commonly used to measure data quality.
Top 7 Data Quality Dimensions AI Can Track
Completeness refers to the existence of all required attributes in the population of data records. Data element is:
- Always required (or)
- Required based on the condition of another data element.
Example:
- Person record with a null First Name
- The person record is missing a value for marital status. The married (Y/N) field should have a non-null value of ‘Y’ or ‘N’ but is populated with a “null” value instead.
Conformity means the data is following the set of standard data definitions like data type, size, and format. All data values conform to the requirements of their respective field.
Example:
- Date of Birth is listed as “26/05/1990” but should be in the format “mm/dd/yyyy“
- Zip Code contains letters but it should be numeric
Validity is determined by how closely data values correspond to reference tables, lists of golden values documented in metadata, and value ranges, etc. All data values are valid in relation to reference tables.
Example:
- Country Code should be a valid value from the reference data for countries
- Age for a Person should be less than 100 years old
Accuracy refers to the degree to which information accurately reflects what’s being described. It can be measured against either original documents or authoritative sources and validated against defined business rules.
Example:
- US Zip Codes should match a list of legal US postal codes
- Person name is spelled incorrectly
Uniqueness refers to the singularity of records and or attributes. The objective is a single (unique) recording of data. Data element is unique — there are no duplicate values.
Example:
- Each Person should only have one record, but there are two instances of the same Person with different identifiers or spellings.
- SSN should be unique, but there are two Person records that have the same social security number.
Consistency means data across all systems reflects the same information and are in synch with each other across the enterprise. The absence of difference, when comparing two or more representations of a thing against a definition.
Example:
- Employee status is terminated but pay status is active
- Employee start date cannot be later than the Employee end date
- N number for a Person record must be the same across systems
Timeliness references whether the information is available when it is expected and needed.
Example:
- For quarterly reporting, data must be up to date by the time the data is extracted
- Last Review Date for the policy must be within the last three years
Get a Free Trial of DataOps DQM - Data Quality Monitor for 14 Days
How To Measure Data Quality Using Data Quality Dimensions?
Datagaps DataOps suite automatically computes Gen AI Data Quality Scores for each rule based on the number of good vs bad data. This score is rolled up to Data Quality Dimensions at Table, Data Model, and System level.
A sample dashboard showing the Data Quality trend is shown below:
Figure: Shows the Data Quality Dimensions and their scores at a system-level DataOps Data Quality comes with the following rule types to make it easy to define Data Validation rules:
Customized SQL Query to identify good or bad records.
Use Query Builder to provide a SQL Expression for identifying bad records.
Automatically Identifies duplicate records. Useful for measuring the Uniqueness of the data elements.
Check for Referential Integrity of data. Useful for measuring Data Integrity.
Check if data in a column conforms to the list of acceptable values. Useful for measuring the Validity of the data attributes.
How To Compute Data Quality Score?
Data Quality Score allows us to quickly understand the current state of our data and more easily compare quality over time. Data Quality Scores will be a percent calculated by the following:
Figure: Shows the trend of Data Quality scores at the system level
Automated Data Quality testing can be done using Data Validation rules. Data Quality Score provides a means to measure and track the Data Quality of your enterprise data at rest and in motion. Data Quality dimensions help categorize the data validation rules into meaningful buckets. DataOps Data Quality is a simple Data Validation testing tool that can be used to automate the Data Quality testing process.
FAQs: Data Quality Dimensions and Scorecards
1) What are the seven data quality dimensions?
The seven core data quality dimensions are Completeness, Conformity, Validity, Accuracy, Uniqueness, Consistency, and Timeliness. Together, they provide a structured framework for defining validation rules and measuring the overall quality of enterprise data.
2) How is a Data Quality Score calculated?
A Data Quality Score is calculated using the formula [1 − (Number of Bad Records ÷ Total Records)] × 100. The resulting percentage indicates how much of a dataset complies with the defined data quality rules, making it easy to track quality over time.
3) At what levels can Data Quality Scores be measured?
Data Quality Scores can be viewed at multiple levels, including individual rules, tables, data models, and entire systems. This hierarchical approach enables teams to investigate specific issues while maintaining a comprehensive view of enterprise-wide data quality.
4) What types of rules can be used to check data quality dimensions?
DataOps Suite supports a wide range of validation rules, including SQL Query Rules, SQL Expression Rules, Duplicate Check Rules, Foreign Key Rules, Domain Rules, Attribute Rules, Dataset Rules, Delta Rules, and Data Reconciliation tests. These rule types collectively validate different aspects of data quality across enterprise datasets.
