Varun is available for hire
Hire VarunVarun Negi
Verified Expert in Engineering
Data Engineer and Developer
Rockville, MD, United States
Toptal member since April 27, 2026
Bio
Varun specializes in cloud-native data platforms on Azure and Databricks, with 17+ years of experience across healthcare, insurance, finance, and the public sector. He covers the full data lifecycle, including ingestion, migration, quality, MDM, and governance. As an accomplished data engineer professional, Varun drives engagements from discovery through delivery, translating complex requirements into scalable, governed data solutions.
Portfolio
U.S. Office of Personnel Management
Azure, Azure Data Factory (ADF), Informatica Intelligent Cloud Services (IICS)...
Weichert Realtors Southern Coast
Azure Data Factory (ADF), Azure, Azure SQL Managed Instance, Salesforce...
Crowe
Salesforce, Customer Data Platform (CDP), Dell Boomi, Workato...
Experience
- Microsoft Excel - 20 years
- Data Migration - 17 years
- SQL - 17 years
- Data Warehousing - 17 years
- Dimensional Modeling - 13 years
- Azure Data Factory (ADF) - 5 years
- PySpark - 4 years
- Databricks - 4 years
Work Experience
Data Engineer
2024 - 2025
U.S. Office of Personnel Management
- Developed end-to-end Databricks Lakehouse platforms processing billions of records using PySpark, Delta Lake, Autoloader, and Delta Live Tables with CDC logic, reducing pipeline runtime.
- Led domain-based migration strategy, modernizing legacy SQL Server and Informatica ETL pipelines into Databricks workflows, successfully migrating data domains including enrollment, providers, and claims.
- Implemented metadata-driven, governance-aware pipelines with built-in data quality validation, PHI access controls, encryption, and audit-ready permissions aligned with regulated public-sector compliance requirements.
- Designed Databricks catalogs, schemas, and compute strategies, including cluster policies and workload isolation, and cutting infrastructure costs through optimized job cluster configurations.
- Designed SCD-1 and SCD-2 star schema dimensional models in ER/Studio and integrated Databricks with Power BI, enabling self-serve analytics and large-scale bulk data consumption from curated gold layers.
- Automated multi-vendor file ingestion and validation using Python, generating submission summary reports and triggering email alerts to leadership, eliminating around 100% of manual file tracking effort across teams.
Technologies: Azure, Azure Data Factory (ADF), Informatica Intelligent Cloud Services (IICS), Databricks, PySpark, ER/Studio Data Architecture, Azure SQL Managed Instance, Python, Data Modeling, Data Architecture, Data Pipelines, Data Engineering, Business Intelligence (BI), Data Visualization, Data Analysis, Data Analytics, Delta Lake, CI/CD Pipelines, Architecture, Medallion Architecture, Role-based Access Control (RBAC), iPaaS, Data Lakehouse, Data Quality, Data Lineage, AI Agents, AI Copilots, Data Lakes, Data Contracts, Analytics
Enterprise Data Architect
2023 - 2024
Weichert Realtors Southern Coast
- Designed and implemented star schema data models in Erwin with optimized fact and dimension tables, reducing Power BI report rendering time by around 40% through indexing, partitioning, and query optimization across Azure SQL Managed Instance.
- Built end-to-end Azure Data Factory ETL pipelines ingesting Salesforce Service Cloud relocation data into the enterprise data warehouse, consolidating structured and semi-structured sources into a single analytics-ready layer.
- Engineered row-level security (RLS), data masking, and access controls on Azure SQL Managed Instance, ensuring PII data protection and compliance across all Power BI reporting layers.
- Optimized DAX calculations, measures, and data model relationships in Power BI, improving dashboard load performance and enabling self-serve analytics for business stakeholders across relocation services.
- Integrated Jenkins with Azure DevOps CI/CD pipelines to automate build triggers on code commits, reducing manual deployment effort and improving release consistency across data engineering workflows.
Technologies: Azure Data Factory (ADF), Azure, Azure SQL Managed Instance, Salesforce, erwin Data Modeler, Azure Data Lake, Microsoft Power BI, Data Build Tool (dbt), PySpark, Data Modeling, Data Architecture, Data Pipelines, Data Engineering, Business Intelligence (BI), Data Visualization, Data Analysis, Data Analytics, Delta Lake, CI/CD Pipelines, Architecture, Role-based Access Control (RBAC), iPaaS, Data Lakehouse, Data Quality, Data Lineage, AI Agents, AI Copilots, Data Lakes, Data Contracts, Analytics
Senior Data Architect
2022 - 2023
Crowe
- Designed and implemented complex enterprise data integration pipelines across NetSuite, SAP HANA, Oracle EBS, and Salesforce using Informatica CDI, Azure Data Factory, and Workato, ensuring seamless data flow and transformation at scale.
- Architected and delivered a Salesforce customer data platform (CDP) solution integrating customer data from multiple sources into a unified profile, enabling enterprise-wide analytics across healthcare, insurance, public sector, and retail clients.
- Built real-time data processing streams using Apache Kafka, modernizing event-driven data pipelines and improving data delivery latency for downstream consumers.
- Led migration of on-premises data infrastructure to cloud environments, enforcing data governance and regulatory compliance with GDPR and CCPA while improving processing efficiency and data accuracy.
- Served as technical SME and pre-sales architect, presenting data strategies to C-level executives and engineering teams across multiple industries, producing architectural diagrams, mapping sheets, and technical documentation.
Technologies: Salesforce, Customer Data Platform (CDP), Dell Boomi, Workato, Azure Data Factory (ADF), MuleSoft, Informatica Intelligent Cloud Services (IICS), Data Architecture, Data Pipelines, Data Engineering, Fivetran, Snowflake, Business Intelligence (BI), Data Visualization, Data Analysis, Data Analytics, Delta Lake, CI/CD Pipelines, Architecture, Medallion Architecture, Role-based Access Control (RBAC), iPaaS, Data Lakehouse, Data Quality, Data Lineage, Data Lakes, Data Contracts, Analytics
Data Architect
2021 - 2022
Salesforce
- Architected and delivered data migration, retention, and archiving solutions for enterprise Salesforce clients across the US and Canada, including end-to-end data transfer from Salesforce to Heroku PostgreSQL for historical analysis.
- Implemented data quality monitoring using Monte Carlo to automate anomaly detection across data pipelines, proactively identifying inconsistencies and improving overall data reliability.
- Designed reference architectures for event-driven workflows using MuleSoft Anypoint ESB and led data migration best practices across CPQ, Sales, Service, and EDA Cloud implementations.
- Established and enforced Salesforce platform best practices across sharing rules, profile and permission set management, validation rules, approval processes, and declarative automation using Flow and Process Builder.
- Managed cross-functional data engineering teams spanning solution architects, program architects, and developers, ensuring alignment of Salesforce configurations, customizations, and development with enterprise-wide business requirements.
Technologies: Salesforce, MuleSoft, Monte Carlo Analysis, Data Architecture, Data Pipelines, Data Engineering, Snowflake, Business Intelligence (BI), Data Visualization, Data Analysis, Data Analytics, Delta Lake, CI/CD Pipelines, Architecture, Medallion Architecture, Role-based Access Control (RBAC), iPaaS, Data Lakehouse, Data Quality, Data Lineage, Data Lakes, Data Contracts, Analytics
Senior Consultant
2020 - 2021
Informatica
- Delivered enterprise ETL, data quality, MDM, and data governance solutions for healthcare clients using Informatica IICS/IDMC.
- Helped customers with migrating their on-premises ETL on PowerCenter to IICS.
- Automated data quality profiling across their source systems using Informatica IDQ, cutting manual validation effort by around 30% and accelerating migration timelines.
- Identified and resolved 70% of critical data anomalies pre-migration using Informatica IDQ profiling, preventing downstream failures in claims and enrollment pipelines for one of the healthcare customers.
Technologies: Informatica PowerCenter, Informatica Data Quality, Master Data Management (MDM), Data Governance, SQL, Data Architecture, Data Pipelines, Data Engineering, Business Intelligence (BI), Data Visualization, Data Analysis, Data Analytics, CI/CD Pipelines, Architecture, Medallion Architecture, Role-based Access Control (RBAC), iPaaS, Data Lakehouse, Data Quality, Data Lineage, Data Lakes, Data Contracts, Analytics
Senior Data Modeler
2017 - 2020
UnitedHealth Group
- Led the joint application of design sessions and data modeling design sessions for the greenfield data warehouse implementation with various data domains.
- Helped the client in performing the entire application migration from IBM DataStage to Informatica PowerCenter, including the implementation of the Informatica Data Quality solution.
- Proposed a data as a service solution to the client and successfully implemented Address Doctor to fix missing/incomplete address data in various source vendor files.
Technologies: ETL, IBM InfoSphere (DataStage), Unix Shell Scripting, SQL, Windows PowerShell, erwin Data Modeler, Edifecs, Informatica PowerCenter, Dimensional Modeling, Slowly Changing Dimensions (SCD), Data Warehousing, Data Modeling, Data Architecture, Data Pipelines, Data Engineering, Business Intelligence (BI), Data Visualization, Data Analysis, Data Analytics, CI/CD Pipelines, Architecture, Medallion Architecture, Role-based Access Control (RBAC), Data Quality, Data Lineage, Data Lakes, Data Contracts, Analytics
Data Conversion Lead
2012 - 2017
Acentra Health
- Developed the reusable ETL code on DataStage that was utilized by multiple state and federal government clients, thereby saving duplicate efforts and achieving quicker results and faster production deployments.
- Converted and standardized legacy Medicaid data (member, provider, claims) using IBM DataStage for state MMIS programs (WA, MD, UT).
- Integrated and processed CMS EDI transactions (834, 835, 837, 810) using MuleSoft and X12 connectors.
- Created data reconciliation scripts in SQL for post data conversion validation, and integrated the ETL run with SQL scripts within a Unix shell script for an end-to-end one-click run script.
Technologies: ETL, IBM InfoSphere (DataStage), SQL, PL/SQL, Unix Shell Scripting, MuleSoft, Dimensional Modeling, Slowly Changing Dimensions (SCD), Data Warehousing, Data Modeling, Data Pipelines, Data Engineering, Fivetran, Snowflake, Business Intelligence (BI), Data Visualization, Data Analysis, Data Analytics, Medallion Architecture, Data Quality, Data Lineage, Data Contracts, Analytics
Senior Data Engineer
2008 - 2012
Tata Consultancy Services
- Integrated Enterprise data warehouse with other applications using DataStage jobs.
- Worked extensively with flat files, NCPDP D.0 standards, and XML files.
- Coordinated complete SDLC efforts in an Agile environment, from requirements gathering and development to operations.
- Worked as a senior developer and technical team lead, coordinating people in the US and India in different time zones (15+ people).
Technologies: ETL, IBM InfoSphere (DataStage), SQL, Unix Shell Scripting, Data Pipelines, Data Engineering, Business Intelligence (BI), Data Visualization, Data Analysis, Data Analytics, Data Lineage, Analytics
Experience
Databricks Lakehouse Ingestion and CDC Framework
A metadata-driven ingestion framework built on Azure Databricks using PySpark, Autoloader, and Delta Live Tables to process billions of records across structured and semi-structured sources. The framework implements CDC logic across bronze, silver, and gold layers with built-in data quality validation, access controls, and encryption. Pipeline configurations are metadata-driven, enabling teams to onboard new data sources without code changes, significantly reducing development effort and time-to-delivery.
MDM Golden Record Engine
An enterprise master data management (MDM) solution using Informatica MDM to perform entity resolution, fuzzy match-and-merge, and golden record creation across heterogeneous sources, including Salesforce, SQL Server, legacy case management systems, and on-premise data stores. The solution standardizes identity across millions of records, eliminating duplicates and establishing a single trusted source of truth for downstream analytics and reporting.
Legacy ETL to Lakehouse Migration Accelerator
A domain-based migration framework designed to modernize legacy SQL Server and Informatica ETL pipelines into Databricks workflows and Delta Live Tables using a CDC approach. The accelerator includes reusable pipeline templates, source-to-target mapping standards, and governance integration with Collibra for metadata management, enabling phased migration of critical data domains with minimal disruption to production systems.
Education
2004 - 2008
Bachelor's Degree in Electronics and Telecommunication
Govind Vallabh Pant University of Agriculture and Technology - Uttarakhand, India
Certifications
JULY 2024 - PRESENT
Databricks Lakehouse Fundamentals
Databricks
FEBRUARY 2023 - PRESENT
Tableau CRM and Einstein Discovery Consultant
Salesforce
MAY 2022 - PRESENT
Salesforce Application Architect
Salesforce
JULY 2021 - PRESENT
Cloud Data Integration
Informatica
JANUARY 2017 - PRESENT
MuleSoft Certified Developer, Integration and API Associate
Salesforce
Skills
Libraries/APIs
PySpark
Tools
Microsoft Excel, IBM InfoSphere (DataStage), Informatica PowerCenter, Microsoft Power BI, Tableau, Edifecs, Salesforce Sales Cloud
Languages
SQL, Python, Snowflake
Frameworks
Data Lakehouse, Windows PowerShell
Paradigms
ETL, Dimensional Modeling, Business Intelligence (BI), Role-based Access Control (RBAC)
Storage
PL/SQL, ER/Studio Data Architecture, Data Pipelines, Data Lakes, Dell Boomi, Master Data Management (MDM)
Platforms
Databricks, Azure, MuleSoft, Salesforce, Workato
Other
Azure Data Factory (ADF), Informatica Intelligent Cloud Services (IICS), Data Migration, Data Warehousing, Slowly Changing Dimensions (SCD), erwin Data Modeler, Informatica Data Quality, Data Modeling, Data Architecture, Data Engineering, Data Visualization, Data Analysis, Data Analytics, Delta Lake, CI/CD Pipelines, Architecture, Medallion Architecture, iPaaS, Data Quality, Data Lineage, Data Contracts, Analytics, Unix Shell Scripting, Azure Data Lake, Data Build Tool (dbt), salesforce data architect, Fivetran, Data Governance, Monte Carlo Analysis, Customer Data Platform (CDP), Azure SQL Managed Instance, Salesforce Service Cloud, AI Agents, AI Copilots
Collaboration That Works
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
1
Share your needs
Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2
Choose your talent
Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3
Start your risk-free talent trial
Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.
Top talent is in high demand.
Start hiring