Files are great data sources and each one of us use many text and excel files to keep our data and calculation. Real time servers also keep data in files and skip the option to save transactions in a relational database.
Files are everywhere and they provide many challenges for data governance when integrated.
Quality Gates helps overcoming easily many files data challenges. Here are some examples of challenges and solutions:
Challenge: Users hold their own private excel, have difficulties to validate whether their own numbers and the organization numbers are really matching.
Solution: Users can validate consistency between their own private files (or extractions from system) and the company numbers in the DWH.
Challenge: How to validate that all files were loaded correctly to our central database (Staging, Datawarehouse, etc.)
Quality Gates can union several files into a single source and make all the classic data validation as if it was a single source.
Quality Gates provides file statistics (i.e. number of rows in files) and bypasses a full file loading which could take long.
Challenge: Acceptance test at the end or at the middle of a project requires many hours of tedious manual validation that needs to be redone over an over again after every small changes.
Solution: The process of acceptance can be simply automated, by loading the file (including the query and acceptance result) and Quality Gates will run the query in the source and compare it to the expected value.
Challenge: How to validate non regression between reports and files over different version.
Solution: IT can load the 2 extractions to Quality Gates and all gaps will be revealed in seconds, assisting to make a rapid impact analysis.
Combining all those great features with the easy and collaborative environment Quality Gates offers, makes it an essential tool to every company handling files and database.