Ganesh Jujjuru, Developer in Hyderabad, Telangana, India
Ganesh is available for hire
Hire Ganesh

Ganesh Jujjuru

Bio

Ganesh is a seasoned Enterprise Architect with 15+ years in data architecture, governance, analytics, and cloud solutions. He has led multi-terabyte Data Migration and ERP modernization projects for top firms including PennyMac, Nokia, Dell, TCS, Shell. He unifies metadata, automates pipelines, and delivers compliant, high-performance enterprise solutions. He also has experience working on prescriptive and exploratory analysis, including ChatGPT, and is enthusiastic about his next venture.

Portfolio

Data Pride Solutions Private Limited
Collibra, Azure, Snowflake, Python, Databricks, Data Build Tool (dbt)...
Inbox Insight Limited
Terraform, Azure, Database Architecture, Fact Tables, Database Optimization...
PennyMac
Azure Databricks, ADF, Dynamics CRM 365, Data Governance, Data Quality...

Experience

  • Data Engineering - 10 years
  • Informatica - 10 years
  • MDM - 8 years
  • Data Governance - 7 years
  • Azure - 6 years
  • Spark - 3 years
  • Amazon Web Services (AWS) - 3 years
  • Data Science - 3 years

Preferred Environment

Azure, Informatica, Snowflake, Data Science, Amazon Web Services (AWS), Google Cloud Platform (GCP), PySpark, Machine Learning, Data Governance

The most amazing...

...thing I've done is build and lead a team of 19 associates in data governance and Unity integration for a supply chain organization with an approved SOW.

Work Experience

Enterprise Architect

2021 - PRESENT
Data Pride Solutions Private Limited
  • Demonstrated expertise in strategy roadmaps and became a trusted advisor to customers with knowledge in enterprise architecture, data architecture, data management, data governance, data science, and AI-driven functions.
  • Drove end-to-end execution of the solution, including identification, interaction with the client, requirement analysis, workflow solution development, statement of work (SOW) preparation, solution customization, configuration, and implementation.
  • Worked on the SOW and the active mobilization of the project map for 12 associates at MobileOrg. Aligned the financial organization's roadmap for data governance and data lake migration modules. Approved the SOW for six quarters with six associates.
  • Led a team of 19 associates in data quality and Unity integration for a supply chain organization with an approved SOW for 7 quarters. Led a team of 7 associates for an FCMG client in the UK on data governance with an approved SOW for 6 quarters.
  • Acted as a data science and governance AI engineer for a gas and power organization.
  • Contributed to working on the networking organization, handling data governance, data lake, and enterprise data warehouse (EDW) projects.
  • Handled master data management and policy management at a networking organization.
  • Collaborated with a supply chain organization on data quality and EDW projects and with a manufacturing organization on data governance and data lake projects.
  • Engaged as a data architect for a financial organization, working on a data lake migration project.
  • Carried our data governance and data engineering, reordering the unclustered domains for a fast-moving consumer goods (FMCG) organization.
Technologies: Collibra, Azure, Snowflake, Python, Databricks, Data Build Tool (dbt), Data Governance, Data Engineering, Data Science, Enterprise Architecture, Profisee MDM, Reltio, Spark, Master Data Management (MDM), Large Language Models (LLMs), Retrieval-augmented Generation (RAG), Azure AI Studio, Machine Learning, Artificial Intelligence (AI), Azure Data Lake, Azure Data Factory (ADF), Microsoft Power BI, Data Analysis, Agile Project Management, Technical Project Management, Data Architecture, Data Management, Analytical Thinking, Requirements Analysis, Business Requirements, Data Pipelines, Business Intelligence (BI), Data Warehousing, ETL, Data Modeling, Data Analytics, Engineering Management, IT Management, Leadership, AWS Glue, PySpark, Amazon SageMaker, Machine Learning Operations (MLOps), Terraform, Amazon EMR Studio, ELT, Microsoft Fabric, Qlik, Data Security, Microsoft SQL Server, Data Migration, Fivetran, Data Vault 2.0, Data Vaults, Git, Google BigQuery, Star Schema, Jinja, Redshift, Enterprise Resource Planning (ERP), Data, ServiceNow, SAP, Stakeholder Management, SharePoint, Amazon RDS, Amazon S3 (AWS S3), Amazon Glacier, Data Lakes, Medallion Architecture, Tableau, Odoo, Apache Airflow, Databases, APIs, Database Security, BigQuery, Google Cloud Platform (GCP), Architecture, Amazon Machine Learning, Cloud Infrastructure, Bedrock, Amazon Bedrock, Algorithms, Cloud, Financial Engineering, API Development, REST APIs, RESTFul APIs, Azure Service Fabric, Microsoft Dynamics 365, Microsoft Graph, Microsoft AI, Microsoft Dynamics, Agile Software Development, Fractional CTO, Minimum Viable Product (MVP), Technical Leadership, Time Series, Supply Chain, Supply Chain Management (SCM), Azure Logic Apps, Salesforce, Data Fabric, Data Lake Design, Microsoft Visual Studio, Cloud Architecture, TOGAF, Azure DevOps, DevOps, Azure Cloud Services, CISSP, Kubernetes, Solution Architecture, CI/CD Pipelines, Performance Optimization, Data Orchestration, ETL Pipelines, Database Schema Design, Azure SQL Data Warehouse, Azure SQL Databases, Enterprise, Azure SQL, Database Design, Team Leadership, Front-end, Project Management, Java, SQL Stored Procedures, Performance Tuning, Azure Cosmos DB, Database Management, Relational Databases, ClickHouse, Metadata, Document Management, Document Management Systems (DMS), Content Management, Data Labeling, Reporting, Fabric Lakehouse, Database Architecture, Fact Tables, Database Optimization, Database Table Optimization, MySQL, Claude, Data Warehouse Implementation, Row-level Security (RLS), Role-based Access Control (RBAC), Model Context Protocol (MCP), Data Loss Prevention (DLP), Microsoft 365, ETL Tools, SaaS, Microsoft Entra ID, Microsoft Power Pages, Zoho, Scalability, Concurrency, Azure Blob Storage

Azure Databricks Expert

2025 - 2025
Inbox Insight Limited
  • Led end-to-end design and deployment of the Inbox Insight Terraform module, ensuring scalable and compliant infrastructure.
  • Delivered a production-ready Terraform module that reduced manual provisioning effort by 60%.
  • Automated deployment of Azure resources (Event Hub, Function Apps, Storage, Log Analytics, and App Insights) for inbox data ingestion and monitoring.
  • Integrated observability and alerting into the module for proactive error detection. Maintained compliance and governance standards by embedding retention policies, audit tags, and managed identities.
  • Improved developer productivity by providing a plug-and-play module with clear documentation and reusable patterns.
  • Recognized for establishing a governance-first approach, ensuring auditability and lineage tracking within the Terraform-managed resources.
Technologies: Terraform, Azure, Database Architecture, Fact Tables, Database Optimization, Database Table Optimization, MySQL, Data Warehouse Implementation, Row-level Security (RLS), Role-based Access Control (RBAC), Model Context Protocol (MCP), ETL Tools, Scalability, Concurrency, Azure Blob Storage

Enterprise Architect | Senior Data Engineer

2023 - 2025
PennyMac
  • Achieved 8% reduction in preventable readmissions, improved model accuracy to 92%, and delivered 99.9% uptime for real-time AI inference across regulated financial and healthcare workloads.
  • Scaled ingestion to 50+ TB daily and delivered 10–50TB ERP migration cycles with full GDPR, HIPAA, and SOX compliance, enabling a unified and audit-ready data foundation.
  • Reduced metadata stewardship effort by 40% through reusable templates, automated lineage, standardized glossary alignment, and integrated quality workflows.
  • Built unified metadata ingestion and automated lineage using Erwin DI across ADF, Synapse, ADLS, SQL Server, Postgres, and Databricks, improving traceability and governance.
  • Designed comprehensive data profiling, cleansing, validation, and ETL pipelines using ADF, Databricks, and SQL to ensure accurate, compliant, low-disruption ERP migrations.
  • Led Oracle-to-Dynamics 365 F&O migration across GL, AP, AR, Assets, and Supply Chain, creating mapping logic, reconciliation reports, and audit-ready documentation.
  • Automated impact analysis for ADF, Databricks, Synapse, SQL, Postgres, Logic Apps, and Function Apps using Erwin Mapping Manager, reducing downstream deployment risk.
  • Delivered governance KPIs via Power BI and integrated ServiceNow workflows for data quality alerts, triage, and SLA-driven issue management across business domains.
  • Built Azure ML pipelines with training, validation, bias detection, SHAP explainability, GPU clusters, and HyperDrive tuning, reducing training time by 35% and boosting F1 by 11%.
  • Deployed ONNX-optimized models to AKS with blue-green rollout, App Gateway integration, RBAC security, <150ms inference latency, and full monitoring with drift-based retraining.
Technologies: Azure Databricks, ADF, Dynamics CRM 365, Data Governance, Data Quality, Erwin DI, Synapse, Azure DevOps, Terraform, Agentic AI, Data Fabric, Data Lake Design, Microsoft Visual Studio, Machine Learning, ServiceNow, TensorFlow, Azure ML Studio, Cloud Architecture, TOGAF, DevOps, Azure Cloud Services, CISSP, Solution Architecture, Kubernetes, Informatica Data Quality, CI/CD Pipelines, Performance Optimization, Data Orchestration, ETL Pipelines, Database Schema Design, Azure SQL Data Warehouse, Azure SQL Databases, Enterprise, Azure SQL, Database Design, Team Leadership, Project Management, SQL Stored Procedures, Performance Tuning, Database Management, Relational Databases, Metadata, Data Labeling, Reporting, Database Architecture, Fact Tables, Database Optimization, Database Table Optimization, MySQL, Data Warehouse Implementation, Row-level Security (RLS), Role-based Access Control (RBAC), ETL Tools, Scalability, Concurrency, Azure Blob Storage

Enterprise Architect | Senior Tech Lead

2023 - 2024
Nokia
  • Handled solution architecture for rules, business glossary within data management, which includes data governance, data quality, metadata management, reference management, and master data management.
  • Implemented Data Vault approach of data modeling on the data aspect within Snowflake, integrating source-scoped dimensions with conformed dimensions, with integration of master data management (customer, product, and SKU) and reference management.
  • Created point-to-point data flows within Informatica Data Integration Hub on the process aspect, to orchestrate, govern, and share data across the hub of Snowflake, Oracle DBs, SQL DBs, and Dell Boomi end-to-end Integrations.
  • Used Fivetran as an accelerator for one-time till the streaming and batch processing is established. Used DBT and ADF for transformation within Azure ADLS, Synapse, and Snowflake to move the data across layers of raw, EH, EUH, and curated.
  • Used custom transformation templates, macros, LookML, and lift-and-shift in DBT modeling for materialized incremental views, snapshot dimensions, and facts. Used Logic Apps and Function Apps (API Gateway creation) and Swagger (external integration).
  • Developed rich graphical representations and visualizations in dashboards to create live charts, bar charts, multi-cards, tree maps/hierarchical maps, and trends within PowerBI -DAX queries and canonical views.
  • Established event-driven architecture for MDM layers, enabling real-time triggers and semantic consistency across domains.
  • Enabled CXO-level governance insights via Power BI dashboards, increasing metadata coverage visibility and SLA adherence.
  • Optimized reporting efficiency by migrating EDW to Snowflake Data Lake, reducing query latency by 35%.Accelerated migration timelines by 30% using Fivetran and DBT macros for incremental views and transformations.
  • Reduced manual stewardship effort by 40% through reusable onboarding templates and automated MDM workflows.
Technologies: Microsoft Power BI, REST APIs, Master Data Management (MDM), Data Engineering, Data Governance, Snowflake, dbt Cloud, Data Build Tool (dbt), DNB, Salesforce, CI/CD Pipelines, Performance Optimization, Data Orchestration, ETL Pipelines, Database Schema Design, Azure SQL Data Warehouse, Azure SQL Databases, Enterprise, Azure SQL, Database Design, Team Leadership, Front-end, Project Management, SQL Stored Procedures, Performance Tuning, Database Management, Relational Databases, Metadata, Data Labeling, Reporting, Database Architecture, Fact Tables, Database Optimization, Database Table Optimization, MySQL, ETL Tools, Scalability, Concurrency, Azure Blob Storage

Data Engineering and Governance Architect

2020 - 2021
Advance Auto Parts India
  • Aligned the roadmap of data governance and gained the approval of the Architecture Review Board (ARB) on the model and process design, identifying the data fabric and ontology of the existing integration.
  • Developed conceptual and logical information models within the context of the enterprise and line of business information architecture.
  • Unified the governance and centralized data engineering framework development.
  • Facilitated the data governance council for CDEs sourced from the enterprise data lake. Implemented Collibra Data Governance Center (DGC) workflows to enable data management capabilities to leverage Collibra API and Connect to adjacent platforms.
  • Utilized Reltio MDM for match/merge, data stewardship, hierarchy patterns, rules finalization, enrichment, and consolidation. Integrated with Axiom for household and address validation.
  • Leveraged Dun and Bradstreet (DNB) for customer enrichment to eliminate duplicate super Data Universal Numbering Systems (DUNS).
Technologies: Collibra, Snowflake, Reltio, Azure Data Lake, Azure Data Factory (ADF), Amazon Athena, Agile Project Management, Technical Project Management, Data Architecture, Data Management, Analytical Thinking, Data Pipelines, Business Intelligence (BI), Data Warehousing, ETL, Data Modeling, Data Analytics, Engineering Management, IT Management, Leadership, AWS Glue, PySpark, Azure Databricks, Amazon SageMaker, Machine Learning Operations (MLOps), Microsoft Power BI, ELT, Microsoft Fabric, Data Security, Microsoft SQL Server, Data Migration, Data Vault 2.0, Data Vaults, Git, Star Schema, Jinja, Redshift, Enterprise Resource Planning (ERP), Data, ServiceNow, SAP, Stakeholder Management, SharePoint, Amazon RDS, Amazon S3 (AWS S3), Data Lakes, Medallion Architecture, Odoo, Apache Airflow, Databases, APIs, SQL Server Integration Services (SSIS), Database Security, Architecture, Amazon Machine Learning, Cloud Infrastructure, Algorithms, Cloud, API Development, REST APIs, RESTFul APIs, Agile Software Development, Minimum Viable Product (MVP), Technical Leadership, Azure Logic Apps, Salesforce, Data Lake Design, Microsoft Visual Studio, DevOps, Azure Cloud Services, CISSP, Solution Architecture, Informatica Data Quality, Performance Optimization, Data Orchestration, ETL Pipelines, Database Schema Design, Azure SQL Data Warehouse, Azure SQL Databases, Enterprise, Azure SQL, Database Design, Team Leadership, Project Management, SQL Stored Procedures, Performance Tuning, Database Management, Relational Databases, Data Labeling, Reporting, Database Architecture, Fact Tables, Database Optimization, Database Table Optimization, MySQL, Scalability, Concurrency, Azure Blob Storage

Principal Architect for Data Governance

2018 - 2020
F5
  • Worked on the solution architecture for rules and the business glossary in data management, which includes data governance, data quality, metadata management, reference management, and master data management (MDM).
  • Carried out the solution for "one source of truth" on data and process aspects. Utilized the data vault approach of data modeling on data aspects with Snowflake for source scope and conformed dimensions with the integration of MDM.
  • Created point-to-point data flows on the process aspect, including the hub of Snowflake, Oracle DBs, SQL DBs, Dell Boomi end-end Integrations, Apigee messaging queues, Salesforce, Marketo, Workday, Cornerstone, OBIEE, and MSBI reporting platforms.
  • Built the enterprise data warehouse for order management within SQL Server and later migrated to a data lake and Snowflake, using Azure Data Factory and WhereScape as data ingestion platforms.
  • Used Fivetran as an accelerator until streaming and batch processing were established. We then utilized dbt and ADF for transformation within Azure ADLS, Synapse, and Snowflake to move the data across the Raw, EH, and EUH layers and curate it.
  • Incorporated custom transformation templates, including macros, LookML, Lift, and shift into dbt modeling for creating materialized incremental views, snapshots-dimensions, and facts.
  • Contributed to building a customer 360-degree view and building customer MDM, including product MDM, invoices, order management, SKU lifecycle, customer success, and campaigns.
  • Created API for outside integration and integrated the customer with DNB and Axiom to identify the industry Standard Industrial Classification (SIC) codes and build the hierarchy using the Data Universal Numbering System (DUNS).
  • Developed rich graphical representations and visualizations in dashboards to create live charts, bar charts, multi-cards, treemaps/hierarchical maps, and trends within PowerBI and DAX queries and canonical views.
  • Constructed 360 views of the customer with Tableau references to lead generation, product enablement, lifecycle changes, and order history.
Technologies: Informatica, MDM, Azure, Python, Dell Boomi, Snowflake, Agile Project Management, Technical Project Management, Data Engineering, Data Architecture, Data Management, Data Analysis, Analytical Thinking, Requirements Analysis, Business Requirements, Data Pipelines, Business Intelligence (BI), Data Warehousing, ETL, Data Modeling, Data Analytics, IT Management, Leadership, PySpark, ELT, Qlik, Data Security, Microsoft SQL Server, Data Migration, Fivetran, Data Vault 2.0, Data Vaults, Star Schema, Jinja, Redshift, Enterprise Resource Planning (ERP), Data, ServiceNow, Stakeholder Management, SharePoint, Data Lakes, Medallion Architecture, Apache Airflow, Databases, APIs, PL/SQL, SQL Server Integration Services (SSIS), Database Security, Architecture, Cloud Infrastructure, Cloud, API Development, REST APIs, RESTFul APIs, Agile Software Development, Minimum Viable Product (MVP), Technical Leadership, Microsoft Visual Studio, Azure Cloud Services, Solution Architecture, Performance Optimization, ETL Pipelines, Database Schema Design, Azure SQL Data Warehouse, Azure SQL Databases, Enterprise, Azure SQL, Database Design, Team Leadership, Project Management, SQL Stored Procedures, Performance Tuning, Database Management, Relational Databases, Metadata, Data Labeling, Reporting, Database Architecture, Fact Tables, Database Optimization, Database Table Optimization, MySQL, Data Warehouse Implementation, SaaS, Scalability, Concurrency, Azure Blob Storage

Senior Data Engineer

2017 - 2018
Dell
  • Worked on building enterprise rules that can be utilized across different modules to track data quality dimensions, including completeness, consistency, accuracy, data decay, uniqueness, referential integrity, and logical total passing metrics.
  • Built the gateway to store the scorecard and profiling information in the reporting layer, which reports KPIs' data lineage to Collibra and Tableau.
  • Handled the ETL design for the integration framework for data lineage of Collibra and Tableau reports.
Technologies: Informatica, Collibra, Data Quality, Agile Project Management, Data Engineering, Data Architecture, Data Management, Data Analysis, Analytical Thinking, Requirements Analysis, Business Requirements, Data Pipelines, Business Intelligence (BI), Data Warehousing, ETL, Data Modeling, Data Analytics, IT Management, Leadership, Data Migration, Data Vaults, Star Schema, Enterprise Resource Planning (ERP), Data, Stakeholder Management, SharePoint, Data Lakes, Medallion Architecture, Tableau, Databases, PL/SQL, SQL Server Integration Services (SSIS), Database Security, Technical Leadership, Supply Chain, Microsoft Visual Studio, Informatica Data Quality, Performance Optimization, ETL Pipelines, Database Schema Design, Azure SQL Data Warehouse, Azure SQL Databases, Database Design, Performance Tuning, Database Management, Relational Databases, Data Labeling, Reporting, Database Architecture, Fact Tables, Database Optimization, Database Table Optimization, MySQL, Data Warehouse Implementation, ETL Tools, Concurrency

Data Engineer

2015 - 2017
Cognizant
  • Acted as a senior ETL developer and prepared the initial business requirement documents, including estimation and technical design documents with data modeling and getting sign-off from the client for MDM, IDQ, and data integration projects.
  • Worked with Informatica to utilize various transformations, including XML, HTTP, Salesforce lookup, Java, Match, and Parser within Informatica Data Quality (IDQ). Contributed to synchronization and replication mappings within the cloud environment.
  • Managed different modules within the project, including Network Access Protection (NAP), Ads, Serenity, and Hub Console.
Technologies: Informatica, SQL, Data Quality, Data Engineering, Data Management, Data Analysis, Business Requirements, Data Pipelines, Business Intelligence (BI), Data Warehousing, ETL, Data Analytics, Data Migration, Star Schema, Enterprise Resource Planning (ERP), Data, Databases, PL/SQL, SQL Server Integration Services (SSIS), Database Security, Minimum Viable Product (MVP), Supply Chain, Supply Chain Management (SCM), Informatica Data Quality, Database Design, Front-end, Performance Tuning, Database Management, Relational Databases, Data Labeling, Fact Tables, Database Optimization, Database Table Optimization, MySQL, Data Warehouse Implementation, ETL Tools, Concurrency

System Engineer

2010 - 2015
Tata Consultancy Services
  • Conducted gap analysis of the multiple source systems and integrated them with extract, transform, load (ETL) development using Informatica and Abnitio.
  • Worked on Informatica Integration Cloud Services (IICS) and Data Integration Hub (DIH).
  • Used trusted data to provide error-free reports in a timely and consistent manner.
Technologies: Informatica, Informatica Cloud, Data Engineering, Data Management, Data Analysis, Business Requirements, Data Pipelines, Business Intelligence (BI), ETL, Star Schema, Data, Databases, PL/SQL, Minimum Viable Product (MVP), Supply Chain, Database Design, Performance Tuning, Relational Databases, Database Optimization, Database Table Optimization, MySQL, Data Warehouse Implementation, ETL Tools

Experience

Pennymac - Finance | Data Lake, Data Governance, and AI Redesign

August 2023 – July 2025
ERP Modernization, Data Governance, and AI Transformation

• Led modernization for a US/UK financial enterprise, unifying ERP, governance, and AI to meet GDPR, HIPAA, and SOX.
• Addressed fragmented Oracle/SQL/Synapse/Databricks platforms, 50TB+ daily ingestion limits, siloed metadata, non-standard MLOps, and high-risk multi-terabyte migrations.
• Delivered a secure Azure-based architecture with automated lineage, audit-ready ERP migration, and GPU-enabled AI workloads.
• Migrated 10–50TB per cycle to Dynamics 365 F&O; built profiling, cleansing, validation, and compliance frameworks.
• Implemented unified metadata ingestion via Erwin DI across ADF, Synapse, ADLS, SQL Server, Postgres, and Databricks; automated impact analysis; integrated ServiceNow; and published Power BI governance KPIs (40% manual effort reduction).
• Built Azure ML pipelines with bias detection, SHAP, GPU clusters, and HyperDrive tuning (35% faster, +11% F1).
• Deployed ONNX models to AKS with <150ms inference, plus monitoring, drift detection, and retraining. Delivered FastAPI microservices.

Impact: 8% fewer readmissions, 92% accuracy, 99.9% uptime, and a scalable 50TB+ ingestion platform.

Nokia – Data Governance, Data Lake, and Enterprise Data Warehouse (EDW) Projects

Project Duration: January 2023 - July 2024
Role: Enterprise Architect/Senior Tech Lead

Use case:
A global networking organization needed to modernize its data governance framework and data lake architecture to achieve a “single source of truth” across customer, product, and operational datasets. The initiative aimed to unify metadata, master data, and reference data while enabling real-time integrations across Snowflake, Oracle, SQL Server, Salesforce, Marketo, Workday, and reporting platforms.

Key Deliverables:
• Unified Data Governance Framework: Business glossary, metadata management, data quality, reference management, and MDM.
• Single Source of Truth: Data Vault modeling in Snowflake, integrating customer, product, SKU master data, invoice, and order for the Customer360 view.
• API Integrations: Connected external systems (DNB, Axiom) for industry classification and hierarchy building.
• Visualization and Reporting: Power BI dashboards (DAX, hierarchical maps) and Tableau for lifecycle and order history insights.
• Tool Evaluation: Comparative assessment of Reltio, Informatica, and Ataccama for ARB board approval.

Shell – Data Science and Governance AI

ROLE
• Implemented a data strategy and governance framework with clearly defined roles and responsibilities.
• Provided thought leadership by addressing business problems and guiding on functional and technical aspects.
• Achieved compliance with the General Data Protection Regulation (GDPR) by collaborating with the information security team.
• Established a metadata catalog, including classification, dependencies, and impact, using Alation for data sources, including Azure Synapse, ADF, and Postgres.
• Conducted profiling using native scanners and created customized profiles for Synapse, Cosmos DB, Parquet, and Avro files, driving identical profiling for duplicate remediation.
• Built customized dashboards for business products and enrichment and access policy management via Business Process Model and Notation (BPMN) workflows and ServiceNow.
• Formed talented data science teams that created AI/ML and generative AI data products.
• Estimated the composition of hydrocarbons in oil using genetic algorithms with end-to-end product development using Scala-Spark.
• Implemented predictive maintenance for machines using sensor data.
• Built graph anomaly detection models utilizing tailor-made algorithms for rate engine applications.

Accolite – Data Quality and Enterprise Data Warehouse (EDW) Projects

• Implemented a centralized data quality and reconciliation framework.
• Developed a data quality framework using Informatica Data Quality (IDQ) and Python scripts.
• Created a reconciliation framework for the stock-keeping unit (SKU) and customer lifecycle.
• Designed conceptual and logical information models for the enterprise and business information architecture.
• Ensured compliance with DW/BI standards and guidelines for developing information models and database designs.
• Worked with the DBA to translate the logical information model into a preliminary logical database design or a physical information model for the target DBMS.
• Generated DDL and DML scripts to load data into BigQuery from GCS buckets.
• Formulated ETL and ELT strategies using SQL scripts to load data into BigQuery, leveraging GCP Composer airflow.
• Established BigQuery data sets, tables, and pipelines for storing processed results, configuring service storage, and using BigQuery in Cloud Shell in GCP.
• Built audit tables for reconciliation and metadata tracking.
• Utilized Airflow to orchestrate workflows between internal components in GCP.
• Used the Dataplex data governance tool to establish data lineage for all Composer and Data Fusion pipelines.

Future Focus – EDW Migration to Data Lake Project

Duration: 11 months
Team Size: 9
Role: Tech lead and data architect

CONTRIBUTIONS
• Identified point-to-point integrations, API gateways, and downstream analytics within the existing architecture during discussions with business analysts, subject matter experts, architects, and project owners.
• Designed the data lake in Snowflake and the landing zone in AWS S3.
• Created data ingestion processes with reusable frameworks.
• Developed data ingestion pipelines utilizing various AWS cloud services, including Lambda, Step Functions, CloudWatch Events, Simple Notification Service (Amazon SNS) notifications, S3, EC2, Python Boto3 SDK, Athena queries, IAM roles, policies, AWS Glue, and notebooks.
• Worked on cloud technologies, specializing in designing data lakes in Snowflake and MongoDB.
• Built data pipelines and real-time streaming using Kafka, Spark Streaming, and Flume, including topics and producers.
• Built the initial MVP on Kafka. Managed Confluent, Amazon MSK, and AutoMQ. Implemented the module using MSK due to the ability to handle taxonomy-driven data structures, heavy loads, and seamless integration with the AWS ecosystem.
• Implemented MapReduce jobs using Java APIs and Pig Latin.

Mars – Data Governance and Data Engineering Reordering Unclustered Domains

ROLE
• Developed reusable data pipelines with Azure Data Factory (ADF) and Databricks for delta processing across domains, ensuring code consistency.
• Organized Azure Data Lake Storage (ADLS) directories within BDAT domains and banners.
• Reorganized ADF pipelines and Databricks code based on the ADLS layer.
• Migrated pipelines for the curation layer and configured batch process parameters for bronze layer ingestion and silver layer processing using reusable code.
• Implemented data quality metrics at Hive Metastore (HMS) during the data transfer from the bronze to the silver layer using DQ Soda and Databricks.
• Established partner mobile device management (MDM) and customer master data management (MDM) to monitor the customer lifecycle from campaign to churn usage.
• Utilized Profisee MDM and configured Purview scans on ADLS, ADF, Databricks, Power BI, Soda DQ, and Profisee MDM.
• Addressed Purview limitation issues related to lineage using Atlas APIs, including duplicate asset creation, unresolved names, view lineage, DDL, and Power BI scanner.
• Customized workflow templates for self-read access to governance interface sources for access management.

Data Lake and Governance Migration Module

ROLE
• Identified the integration's data fabric and ontology and closed the gaps for real-time and batch integration using Informatica Intelligent Cloud Services (IICS), application integration, data integration, and mass ingestion for one-time loading.
• Aligned the roadmap of extract, transform, and load (ETL) and extract, load, and transform (ELT) and streaming using multiple statements of work (SOWs) on proof of concept for the technology stack of Fivetran, including Kafka and ADF.
• Discovered the resource skills required to fill them with the competent skills.
• Onboarded the resources at the client end from 1 to 9 associates within 6 months, managed deliverable tracking, and guided the technical team.
• Identified the enterprise architecture roadmap, designed the governance strategy, and created the technical and business model framework.
• Provided a platform for various applications by transforming and sharing financial messages of different formats.
• Identified the business stakeholders and approvers for the asset changes and data models.
• Captured the business glossary and hierarchical terms on the business and technical end.
• Categorized domains using BDAT terminology and identified sensitive data, PII and non-PII.

Metadata-driven Pipeline Automation for Azure Data Pipelines using Medallion Architecture

• Led digital-first migration to Azure stack with metadata-driven automation for ML workloads to unify ingestion, transformation, and AI model deployment.
• Delivered measurable business impact—72% reduction in deployment effort, 86% improvement in on-time delivery, and full traceability from raw ingestion layer to ML inference.
• Architected a scalable STM-based framework to auto-generate Azure Data Factory pipelines, Databricks notebooks, and Synapse views following a medalion architecture (bronze/silver/gold)
• Built GPT for business users to automatically create ADF pipelines and Databricks with simplex transformation for the reporting framework.
• Developed a Python-based STM parser (pandas, openpyxl) to extract source/target schema, transformation logic, partitioning rules, and type mappings.
• Automated ADF pipeline generation via REST API and ARM templates — dynamically creating linked services, datasets, and parameterized activities (Copy, Lookup, ForEach, Web, DatabricksNotebook) supporting schema drift.
• Implemented Azure DevOps YAML pipelines, ML model deployments best practices.
• Built predictive ML models in PyTorch and TensorFlow for healthcare use cases like patient readmission prediction and anomaly detection.

Healthcare: Enterprise Data Platform Modernization on GCP with Integrated Data Governance & MLOps

• Architected scalable Lakehouse on GCP using BigQuery, Dataproc, and Cloud Storage to ingest 50TB+ daily from transactional, IoT, and 3rd-party sources.
• Built resilient pipelines with Pub/Sub, Dataflow (Beam), and Composer (Airflow), supporting CDC via Datastream and schema evolution.
• Redesigned SAP MDM architecture to align with GCP metadata standards, enabling unified master data governance across cloud and on-prem systems.
• Implemented SAP MDM connectors for ingesting business-critical metadata into Dataplex, supporting entity resolution and golden record creation.
• Integrated Collibra with Dataplex and Data Catalog for lineage, glossary, and PII classification; extended metadata sync with SAP MDM for master data harmonization.
• Enforced fine-grained access via IAM, VPC-SC, and BigQuery column/row-level security, aligned with Cloud Identity for RBAC/ABAC.
• Deployed ML models on Vertex AI for forecasting, churn, and anomaly detection; automated feature store updates and retraining.
• Adopted MLOps with Vertex Pipelines, CI/CD via Cloud Build and GitHub Actions, and governed model registry workflows.
• Managed infra with Terraform across dev/test/prod; ensured compliance with GDPR/HIPAA via governance workshops

Healthcare | Metadata-driven Pipeline Automation for Medallion Architecture Using STM Framework

• Led digital-first migration with metadata-driven automation, integrating AWS services for machine learning (ML) workloads to unify ingestion, transformation, and AI model deployment.
• Architected a scalable STM-based framework to auto-generate AWS Glue, DynamoDB, Databricks notebooks, Synapse views, and Amazon SageMaker ML workflows.
• Supported Medallion Architecture (Bronze/Silver/Gold layers) with Unity Catalog integration for secure, governed data access.
• Generated PySpark notebooks from STM logic using Jinja2 templates.
• Uploaded notebooks via Databricks REST API and scheduled execution through ADF pipelines with dependency chaining.
• Built predictive ML models in Amazon SageMaker (PyTorch, TensorFlow) for healthcare use cases like patient readmission prediction and anomaly detection.
• Integrated CI/CD for AWS ML workflows (CodePipeline/CodeBuild) with model registry, approval gates, and rollback strategies.
• Implemented data governance using Collibra for lineage, classification, and glossary; AWS Lake Formation for fine-grained access and PII masking.
• Automated technical documentation by extracting metadata from Purview, Unity Catalog, and Amazon SageMaker to generate markdown lineage reports for compliance.

Education

2006 - 2010

Bachelor's Degree in Electronics and Computer Engineering

Nagarjuna University - Guntur, India

Certifications

AUGUST 2024 - PRESENT

Certified Data Management Professional – Associate Level - DMF

DAMA International

OCTOBER 2023 - PRESENT

AWS Certified Solutions Architect - Practitioner & Developer

Amazon Web Services

OCTOBER 2023 - PRESENT

Azure Solution Architect AZ-900, DP-900, DP-203, AZ-204, DP-100, AZ-305

Microsoft

AUGUST 2022 - PRESENT

Collibra Ranger

Collibra

APRIL 2021 - PRESENT

The Open Group Architecture Framework (TOGAF)

Open Group

Skills

Libraries/APIs

PySpark, API Development, REST APIs, PyTorch, TensorFlow, Open APIs, Fabric

Tools

Collibra, Informatica Sub Version, Microsoft Power BI, AWS Glue, Amazon SageMaker, Apache Airflow, BigQuery, Microsoft Graph, Azure Logic Apps, Microsoft Visual Studio, Microsoft Power Pages, Power BI Desktop, Amazon Athena, Amazon OpenSearch, Terraform, Git, Tableau, Microsoft AI, Microsoft Dynamics, Claude, Odoo, Kafka Streams, GPT Builder, Synapse, Microsoft Dynamics 365 for Finance and Operations, Azure ML Studio, dbt Cloud, AWS IAM, AWS Glue DataBrew

Languages

Snowflake, Python, SQL, Scala, Java

Frameworks

TOGAF, Jinja, ADF, Data Fabric, Spark, Bedrock

Paradigms

Requirements Analysis, Business Intelligence (BI), ETL, Agile Software Development, Azure DevOps, DevOps, Role-based Access Control (RBAC), Database Design, Model Context Protocol (MCP), HIPAA Compliance

Platforms

Azure, Amazon Web Services (AWS), Databricks, Google Cloud Platform (GCP), Microsoft Fabric, SharePoint, Azure SQL Data Warehouse, Reltio, Apache Kafka, Azure AI Studio, Qlik, Azure Service Fabric, Microsoft Dynamics 365, Salesforce, Kubernetes, Fabric Lakehouse, Azure Synapse, Azure Functions, AWS IoT

Storage

Master Data Management (MDM), Data Pipelines, Microsoft SQL Server, Redshift, Amazon S3 (AWS S3), Data Lakes, Databases, PL/SQL, Database Security, Data Lake Design, Azure Cloud Services, Azure SQL, Azure SQL Databases, SQL Stored Procedures, Azure Cosmos DB, Database Management, Relational Databases, Database Architecture, MySQL, SQL Server Integration Services (SSIS), ClickHouse, Microsoft Entra ID, Dell Boomi, Data Integration

Industry Expertise

Project Management

Other

Computer Engineering, Informatica Data Quality, MDM, Azure Databricks, purview, Enterprise Architecture, Data Engineering, Data Governance, Large Language Models (LLMs), Machine Learning, Electronics, Azure Data Lake, Azure Data Factory (ADF), Data Analysis, Agile Project Management, Technical Project Management, Data Architecture, Data Management, Analytical Thinking, Business Requirements, Data Warehousing, Data Modeling, Data Analytics, Engineering Management, IT Management, Leadership, API Integration, Amazon Kinesis, Machine Learning Operations (MLOps), ELT, Data Migration, Fivetran, Data Vault 2.0, Data Vaults, Star Schema, Enterprise Resource Planning (ERP), Data, Stakeholder Management, Amazon RDS, Amazon Glacier, Medallion Architecture, APIs, Architecture, Amazon Machine Learning, Cloud Infrastructure, Algorithms, Cloud, RESTFul APIs, Fractional CTO, Minimum Viable Product (MVP), Technical Leadership, Time Series, Supply Chain, Supply Chain Management (SCM), Cloud Architecture, Solution Architecture, Microsoft Purview, Metadata, CI/CD Pipelines, Performance Optimization, Data Orchestration, ETL Pipelines, Database Schema Design, Enterprise, Team Leadership, Performance Tuning, Document Management Systems (DMS), Content Management, Data Labeling, Reporting, Fact Tables, Database Optimization, Database Table Optimization, Data Warehouse Implementation, Row-level Security (RLS), Data Loss Prevention (DLP), ETL Tools, SaaS, Scalability, Concurrency, Azure Blob Storage, Data Science, Profisee MDM, Data Build Tool (dbt), WhereScape, Retrieval-augmented Generation (RAG), Artificial Intelligence (AI), LangChain, Amazon EMR Studio, Data Security, Google BigQuery, ServiceNow, SAP, Amazon Bedrock, Financial Engineering, CISSP, Front-end, Document Management, Microsoft 365, Zoho, Informatica, Data Quality, Informatica Cloud, EndNote, Generative Design, OpenAI GPT-4 API, Cognitive Architectures, Azure Cognitive Search, Pinecone, FAISS, SAP Master Data Management (MDM), Mixtral, Web Crawlers, Agentic AI, Dynamics CRM 365, Erwin DI, Oracle CR, General Data Protection Regulation (GDPR), SOX, API Gateways, DNB, Azure VDI, Azure IoT, ADLS GEN2, RISK AND COMPLIANCE

Collaboration That Works

How to Work with Toptal

Toptal matches you directly with global industry experts from our network in hours—not weeks or months.

1

Share your needs

Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2

Choose your talent

Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3

Start your risk-free talent trial

Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.

Top talent is in high demand.

Start hiring