Eric is available for hire
Hire EricEric He
Verified Expert in Engineering
Azure Data Engineer and Software Developer
Location
Nashville, United States
Toptal Member Since
June 21, 2023
Eric is an Azure data engineer with 10 years of experience. He enjoys taking challenging roles as a senior or lead developer and data architect in highly technical companies where he can utilize his skills in Azure Stack and other BI tools to provide quality service and deliver effective solutions to customers and clients.
Portfolio
Crocs
Azure, Azure SQL, Azure Databricks, Apache Airflow, Azure Data Lake...
CCG
Azure, Azure SQL, Synapse, Azure Databricks, Azure Data Lake...
naviHealth
Azure SQL
Experience
Availability
Part-time
Preferred Environment
Azure, Azure Databricks, Azure Data Factory, Azure SQL, Snowflake, Azure Data Lake
The most amazing...
...thing I've done is redesign and lead a team to implement a metadata-driven architecture ETL system for Crocs using Azure Databricks, ADF, ADLS, and Snowflake.
Work Experience
Data Engineering Manager
2021 - 2023
Crocs
- Redesigned and implemented the metadata-driven architecture ETL system using Azure Databricks, ADLS, ADF, and Snowflake. The new system ingested petabyte-level data from 25+ source systems with a high throughput rate, full data auditing, and logging.
- Designed and implemented cost-tagging and alert systems in the newly designed ETL system for cost optimization and event-driven alerts.
- Led and guided offshore teams from India and Europe to develop a production job monitor dashboard in Power BI for level 1 and 2 support purposes. Managed onshore and offshore team members to provide global production support 24/7.
Technologies: Azure, Azure SQL, Azure Databricks, Apache Airflow, Azure Data Lake, Azure Data Factory, Snowflake, Microsoft Power BI
Azure Senior Consultant | Azure Data Architect
2020 - 2021
CCG
- Served as a data architect in the Vineyard Vines enterprise data platform project and led two data engineers to design and implement ETL processes to ingest and transform data sources into an existing data warehouse using Azure Stack.
- Led the development of the RaceTrac analytics platform project using Azure BI tools to ingest and transform the live feed data from all stores across the US for front-end business users to identify fraudulent activity and send out alerts.
- Acted as a data architect in the FormFree Azure Synapse project to help clients migrate their existing data platform to Azure Synapse, which enabled them to solve the storage limit problem on Azure SQL Database with the help of Synapse SQL pool.
- Served as the lead developer in the LazyDay enterprise data platform project to help clients migrate Azure SQL Database to Azure Delta Lake using Databricks, which enabled them to solve the challenge of compiling data.
Technologies: Azure, Azure SQL, Synapse, Azure Databricks, Azure Data Lake, Azure Data Factory, Google Cloud Platform (GCP)
Senior Business Intelligence Developer
2018 - 2020
naviHealth
- Gathered front-end requirements to develop T-SQL scripts to generate usable endpoint tables for amplifying statistical analysis capabilities and enabling data visualization on the front end.
- Created index, views, complex stored procedures, appropriate user-defined functions, and effective triggers to facilitate efficient data manipulation and consistency on the Microsoft SQL Server database platform.
- Maintained and troubleshot production system's SSIS packages, stored procedures, and optimized SQL Server performance.
- Acted as a key contributor to business intelligence and data management strategy, including information architecture, data architecture, master data management architecture, metadata architecture, and information delivery lifecycle.
Technologies: Azure SQL
Senior SQL Developer
2014 - 2016
Bank of America
- Worked with other developers, data architects, DBAs, project managers, and client business users to develop the initiative project of next generation ETL processes using Azure Data Lake, Azure Data Factory, and Power BI.
- Prepared an analysis document with the recommended solution. Analyzed, designed, developed, and tested dimensional models. Created, tested, and maintained models within SSIS and other ETL tools.
- Analyzed, developed, tested, maintained, and supported very complex data and process models, reports, and processes in the Microsoft ETL environment using SSIS and SSRS.
Technologies: SQL Server 2016, SQL Server Integration Services (SSIS)
BI Developer
2012 - 2014
Sabre
- Worked with the information/data architect and database designer/modeler to help implement the physical data model to support the mission critical OLTP and OLAP production databases.
- Performed the statistical analysis of data in the warehouse. Reviewed the customer's requirements and determined if the business processes for gathering, cleansing, and ensuring the quality of data were adequate.
- Designed and enhanced the SSIS ETL packages and processes and implemented data transformation rules and data validations in the enterprise data warehouses (EDW).
Technologies: SQL Server Integration Services (SSIS), SQL Server BI, Sabre Global Distribution System
Experience
Crocs | Business Analytics Next-gen Data Platform
I redesigned and implemented a metadata-driven architecture ETL system using Azure Databricks, Azure Data Lake Storage, Azure Data Factory, and Snowflake.
The newly designed ETL system ingests petabyte-level data from various source systems, such as Adobe, SFTP, SAP, SPS, SharePoint, and Salesforce, into Azure ADLS and Databricks Delta Table with high efficiency and throughput rate, as well as full data auditing and logging.
Designed and implemented cost-tagging and alert systems in the newly designed ETL system for cost optimization and sending alerts in case of process failure.
The newly designed ETL system ingests petabyte-level data from various source systems, such as Adobe, SFTP, SAP, SPS, SharePoint, and Salesforce, into Azure ADLS and Databricks Delta Table with high efficiency and throughput rate, as well as full data auditing and logging.
Designed and implemented cost-tagging and alert systems in the newly designed ETL system for cost optimization and sending alerts in case of process failure.
Education
2012 - 2014
Master's Degree in Industrial Engineering
Purdue University - West Lafayette, IN, USA
2008 - 2012
Bachelor's Degree in System Engineering
Zhejiang University - Hangzhou, China
Skills
Libraries/APIs
PySpark
Tools
SQL Server BI, Microsoft Power BI, Synapse, Apache Airflow, Adobe Analytics, Sabre Global Distribution System
Languages
Python, SQL, Snowflake
Paradigms
ETL, Database Design
Platforms
Databricks, Azure, Google Cloud Platform (GCP)
Storage
Azure SQL, SQL Server 2016, SQL Server Integration Services (SSIS)
Other
Big Data, Big Data Architecture, Azure Databricks, Azure Data Lake, Azure Data Factory, System Design, Statistics, Data Engineering
Collaboration That Works
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
1
Share your needs
Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2
Choose your talent
Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3
Start your risk-free talent trial
Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.
Top talent is in high demand.
Start hiring