Priyanshu Bahuguna

I worked on curating telemetry data from customer devices and enriched it with identity card (IC) signatures to classify syslogs as critical or non-critical. I created the medallion architecture for the data warehouse to ingest and transform the data and deploy machine learning models using Vertex AI to classify the syslog.

I architected the E2E data pipeline using serverless offerings to curate the data and chose a medallion architecture for designing a data warehouse after looking at the use cases. The goal was to provide transformed data to ML models and allow other consumers to access the data for their use cases.
In the medallion architecture, the data is segregated into bronze/silver and gold layers, with each subsequent layer providing more enriched data.

Stakeholders can then choose which degree of enrichment they want to consume. Data ingestion was done through files pushed to landing zones from where it was consumed by Dataflow jobs. I converted Snowflake queries into data build tool (dbt) workflows for better handling and quality mismatch.
Created a custom audit logging framework to capture all writing operations in the database, allowing efficient audit logging alerts for any unwanted write operations.