Datagaps - Automated Testing Tools for ETL, BI & BigData Testing

ETL testing applies to Data Warehouses or Data integration projects while Database Testing applies to any database holding data (typically transaction systems). Here are the high level tests done in each:

ETL Testing : Primary goal is to check if the data moved properly as expected.

Database Testing : Primary goal is check if the data is following the rules/standards defined in the Data Model.

ETL Testing : 

  • Verify that the counts in the source and target are matching.
  • Verify that the data is matching between source and target.
  • Verify that the transformed data is as per expectation.
  • Verify that the data is incrementally getting updated.
  • Verify that the foreign – primary key relations are preserved during the ETL.
  • Verify if there are any duplicates in the loaded data.

Database Testing : 

  • Verify that the foreign – primary key relations are maintained and there are no orphan records.
  • Verify that the data in the columns had valid values. eg: If there is a domain (encoded list) defined for a column, check if the values in the column meet that requirement.
  • Verify that the data in columns is accurate. eg: If you have an age column, does it have values more than 100.
  • Verify if data is missing in columns where required. eg. If is a column is expected to have data always, check if there are any null values.