Sumati Jain

Increased Ataccama adoption by 40% by empowering data and application owners across five business units.

The client used Ataccama and Collibra as strategic tools for DQ and metadata management, respectively. As part of the data foundations program, I ensured that each business unit complied with centrally laid data management principles. One of the requirements was to publish data quality rules and results metadata for critical data elements to Collibra.

I created a proposal and implemented a DQ reporting layer through data engineering pipelines built using the Ataccama IDE. The IDE fetched data from a highly normalized Ataccama operational database into a denormalized dimensional schema using data warehousing methodologies. This enabled end users to create bespoke data visualization using tools of their choice and acted as an intermediate stage before data gets pushed to Collibra.

Additionally, I used Collibra exposed API endpoints to push this denormalized data from the reporting layer while maintaining proper business metadata lineage. This demonstrated data quality at various levels, from the physical table to the data domain level, helping data owners, application owners, and data stewards keep track of data quality.

A Power BI-based data quality dashboard whose purpose is to give a summary of data quality metrics for hosts of applications used by insurance business units in a top British financial institution.

This dashboard aimed to give a holistic understanding of data quality by summarizing results measured across various DQ dimensions like completeness, validity, accuracy, uniqueness, etc. while providing a view of DQ over time. This was built on top of a reporting layer that captured historical results of DQ evaluations performed by data asset stewards using Ataccama.

Further, multiple reports were built to provide drill-down capability by allowing users to look into the details of DQ rules that were used, various glossary terms that are tagged to it indicating critical data elements (CDE), total records that were processed, and number of passed and failed records.