Preethi is currently unavailable
Preethi B
Verified Expert in Engineering
Data Engineer and Developer
Nashville, TN, United States
Toptal member since July 10, 2024
Bio
Preethi is a versatile data engineer with extensive experience across various industries, specializing in Azure, AWS, and GCP. She designs, develops, and maintains robust data pipelines, utilizing Agile methodologies to ensure efficient project delivery. Preethi also excels in cloud application integration and cloud data integration, offering valuable insights for seamlessly integrating and optimizing data solutions.
Portfolio
Volvo Car USA - Data Management and Insights Team
Azure Data Factory (ADF), SQL, Snowflake, Data Migration, Metadata, Databricks...
Walmart
Azure, Hadoop, SQL, Apache Airflow, Snowflake, CI/CD Pipelines, Apache Spark...
Homesite Insurance
Apache Spark, Scala, Spark SQL, Amazon S3 (AWS S3), Exploratory Data Analysis...
Experience
- SQL - 9 years
- Azure - 9 years
- Azure Data Factory (ADF) - 9 years
- Python - 9 years
- Hadoop - 9 years
- ETL - 9 years
- CI/CD Pipelines - 8 years
- Snowflake - 8 years
Preferred Environment
SQL, Snowflake, PySpark, ETL Implementation & Design, Data Engineering, Microsoft Power BI, Google Cloud Platform (GCP), Google BigQuery, Fivetran, Apache Airflow
The most amazing...
...thing I've done is design, develop, and implement a real-time data analytics pipeline using Azure, GCP, streamlining data ingestion and integrating analytics.
Work Experience
Azure Data Engineer
2024 - 2026
Volvo Car USA - Data Management and Insights Team
- Implemented data integrations across batch, REST APIs, and streaming/time-series sources, designing ingestion patterns with checkpointing/idempotency and schema evolution handling.
- Built and operated production-grade Databricks pipelines in Azure using Jobs/Workflows (parameterized runs, retries, schedules, cluster policies) to deliver reliable data products beyond notebooks.
- Implemented a medallion architecture framework by designing and structuring metadata tables and configuration layers across bronze, silver, and gold zones, enabling dynamic orchestration, better lineage tracking, and seamless data quality governance.
- Enforced engineering rigor with Git-based development, PR/code reviews, and CI/CD pipelines to automate build, test, and deployment of Databricks code and configurations across environments.
- Designed analytics-first OLAP models in the lakehouse/warehouse (fact and dimension-style schemas, aggregates, and curated views) optimized for BI and customer-facing analytics workloads.
- Delivered operational reliability through monitoring and alerting, pipeline failure recovery (reruns/backfills), and performance tuning (Spark configs, partitioning, compaction, Z-ORDER/optimization) to meet SLA targets.
- Created end-to-end workflow orchestration connecting ADF pipelines, Snowflake stored procedures, and Airflow DAGs to automate multi-stage data loads from ingestion to business presentation layers.
- Reduced end-to-end processing time by optimizing Spark partitioning and join strategy, improving job runtime, and lowering compute cost.
- Increased analytics performance by designing curated OLAP tables/aggregates and applying Delta optimizations (OPTIMIZE/Z-ORDER where applicable).
- Improved data trust by embedding automated data quality checks (key integrity, duplicates, reconciliation) and publishing validation results for stakeholders.
Technologies: Azure Data Factory (ADF), SQL, Snowflake, Data Migration, Metadata, Databricks, PySpark, Data Integration, Delta Lake, Batch, APIs, Python, Data Reconciliation
Senior Data Engineer
2022 - 2024
Walmart
- Delivered Azure-based data pipelines and monitoring (ADF/Databricks/ADLS and alerts/logging) to support a large-scale migration, improving data freshness by 40% and maintaining stable scheduled loads with failure recovery.
- Built and optimized complex SQL transformations (CTEs, window functions, incremental loads) to deliver trusted KPI tables/views, reducing query runtime by 35% and fixing data mismatches through root-cause analysis.
- Implemented Looker/LookML models (views, explores, measures) and built operational dashboards, standardizing KPI definitions and reducing ad-hoc SQL/reporting requests by 25% through self-serve analytics.
- Implemented reusable LookML patterns (persistent derived tables, standardized dimensions/measures, symmetric aggregates), improving dashboard load time by 20%.
- Reconciled migrated datasets by building SQL validation suites (row counts, null rates, key integrity, aggregation checks), reducing post-release data defects by 30%.
Technologies: Azure, Hadoop, SQL, Apache Airflow, Snowflake, CI/CD Pipelines, Apache Spark, GitHub, ETL, Exploratory Data Analysis, Python, Microsoft Power BI, DAX, Looker Studio, Looker, Data Reconciliation
Senior Data Engineer
2020 - 2022
Homesite Insurance
- Designed and managed large-scale ETL pipelines supporting regulated insurance data, ensuring strong controls around data access, auditing, and reliability for compliance-driven reporting.
- Implemented secure data ingestion and transformation workflows aligned with PII protection and compliance standards, including role-based access controls and monitored data access patterns.
- Optimized distributed data processing workflows to improve data quality, reliability, and system uptime, reducing operational risk in compliance-sensitive environments.
Technologies: Apache Spark, Scala, Spark SQL, Amazon S3 (AWS S3), Exploratory Data Analysis, Informatica PowerCenter, SQL, Python, Spark, Tableau, Google Cloud Platform (GCP), Apache Airflow, Data Reconciliation
Senior Data Engineer
2018 - 2020
Merck Pharma
- Built and optimized GCP-based data pipelines ingesting millions of healthcare records daily from EHR and EMR systems, lab platforms, and patient portals using BigQuery, Cloud Functions, and Python.
- Designed secure ingestion and storage architectures for PHI data, implementing HIPAA-aligned controls including access restrictions, encryption, audit logging, and secure service-to-service communication.
- Developed scalable BigQuery schemas with optimized partitioning and clustering strategies to support high-volume analytical workloads while controlling query costs.
Technologies: SQL Server BI, Tableau, Google Cloud, Google BigQuery, Google Cloud Functions, Apache Airflow, GCP Security, Python
Data Engineer
2016 - 2018
Grapesoft Solutions
- Utilized Hadoop for distributed storage and processing of large-scale data, improving data processing capabilities, enabling the handling of terabytes of data, and significantly reducing query response times.
- Connected Tableau to various data sources, including SQL databases and Hadoop clusters, and created interactive dashboards and reports with intuitive and interactive visualizations, leading to a 25% increase in report utilization by stakeholders.
- Implemented indexing strategies and query refactoring while performing SQL Queries, which reduced query execution times by 50%, leading to quicker access to critical data and improved overall system performance.
Technologies: Spark SQL, Scala, Tableau, Hadoop, Redshift, Power Query, Exploratory Data Analysis, SQL, Python, Spark
Data Engineer
2014 - 2016
Avon Technologies Pvt Ltd
- Designed and implemented data models using relational databases (e.g., MySQL, PostgreSQL) or NoSQL databases (e.g., MongoDB), optimizing data storage and retrieval.
- Integrated data from multiple sources (e.g., APIs, flat files, databases) into a centralized data repository, ensuring data consistency and accuracy.
- Implemented version control for ETL scripts and maintained comprehensive documentation for data pipelines and processes, ensuring reproducibility and knowledge sharing.
- Created reports and dashboards using tools like Tableau or Power BI, providing actionable insights to stakeholders and business users.
Technologies: Data Warehouse Design, Scripting, SQL, Python, Software Development Lifecycle (SDLC), Exploratory Data Analysis, Spark, Software Development
Experience
Recommendation System
https://github.com/Preethi-68/Recommendation-SystemsUsing data to verify similarities is the secret to how Amazon, Flipkart, or YouTube suggest alternatives. The recommendation system is a similarity-based modeling technique that verifies the association of data points and basket preferences based on the same.
Outcomes include understanding association-based models, including the patterns of commonly used recommenders encountered daily, and building basic recommenders that suit business needs.
Outcomes include understanding association-based models, including the patterns of commonly used recommenders encountered daily, and building basic recommenders that suit business needs.
Exploratory Data Analysis
https://github.com/Preethi-68/Exploratory-Data-Analysis/tree/mainThe aim of this project is to perform a comprehensive exploratory data analysis (EDA) on the datasets to understand their underlying structure, identify patterns, detect anomalies, and summarize the main characteristics using both visual and quantitative techniques. The analysis will cover univariate, bivariate, and multivariate analysis to gain deeper insights into the data.
KEY FEATURES
• Univariate, bivariate, and multivariate analysis
• Categorical variable encoding
• Normalization and scaling
• Missing value handling
• Data visualization and storytelling
KEY FEATURES
• Univariate, bivariate, and multivariate analysis
• Categorical variable encoding
• Normalization and scaling
• Missing value handling
• Data visualization and storytelling
Model Deployment
https://github.com/Preethi-68/Model-Deployment-This project showcases the complete lifecycle of a machine learning model from development to deployment and productionalization.
I serialized the model using Python's pickle library, allowing efficient model storage and reuse.
To facilitate accessibility, I implemented an API using Flask, creating endpoints that handle JSON inputs and outputs for smooth integration. I containerized the Flask application using Docker for development and production by writing and optimizing Dockerfiles to create efficient Docker images.
Additionally, I orchestrated the deployment using Kubernetes, configuring deployments and services for scalability, load balancing, and rolling updates. I integrated monitoring and logging solutions to track the application's performance and quickly address any issues.
I serialized the model using Python's pickle library, allowing efficient model storage and reuse.
To facilitate accessibility, I implemented an API using Flask, creating endpoints that handle JSON inputs and outputs for smooth integration. I containerized the Flask application using Docker for development and production by writing and optimizing Dockerfiles to create efficient Docker images.
Additionally, I orchestrated the deployment using Kubernetes, configuring deployments and services for scalability, load balancing, and rolling updates. I integrated monitoring and logging solutions to track the application's performance and quickly address any issues.
Education
2010 - 2014
Bachelor's Degree in Computer Science
Gokaraju Rangaraju Institute of Engineering and Technology - Hyderabad, Telangana, India
Skills
Libraries/APIs
PySpark
Tools
GitHub, Tableau, Informatica PowerCenter, Spark SQL, SQL Server BI, Power Query, Azure Machine Learning, Apache Airflow, Microsoft Power BI, GCP Security, Looker
Languages
SQL, Python, Snowflake, Scala, Batch
Frameworks
Apache Spark, Hadoop, Spark, Flask
Paradigms
ETL, ETL Implementation & Design
Platforms
Azure, AWS IoT, Amazon EC2, Google Cloud Platform (GCP), Kubernetes, Docker, Databricks, Apache Kafka, Amazon Web Services (AWS)
Storage
Amazon S3 (AWS S3), Data Integration, Data Validation, Redshift, Azure Cosmos DB, Google Cloud
Other
Big Data, Azure Data Factory (ADF), CI/CD Pipelines, Informatica, Amazon EMR Studio, Informatica Cloud, Data Migration, Model Building, Data Warehouse Design, Scripting, Data Cleaning, Feature Engineering, Reporting, Normalization, Azure Databricks, Data Engineering, Data Reconciliation, Software Development, Exploratory Data Analysis, Software Development Lifecycle (SDLC), Visualization, Storytelling, Statistical Analysis, Scaling, Model Deployment, Data Modeling, Google BigQuery, Google Cloud Functions, Fivetran, DAX, Data Build Tool (dbt), Metadata, Document Management Systems (DMS), Architecture, Delta Lake, APIs, Looker Studio
Collaboration That Works
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
1
Share your needs
Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2
Choose your talent
Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3
Start your risk-free talent trial
Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.
Top talent is in high demand.
Start hiring