Huajun Gao
Verified Expert in Engineering
DB and Data Lake Developer
Glen Huntly, Victoria, Australia
Toptal member since March 19, 2021
Huajun is an experienced senior developer who excels in end-to-end cloud data solutions, optimizing efficiency and accessibility. Their expertise in Azure, Snowflake, and Hadoop enables seamless migration and pipeline enhancement. With strong skills in CI/CD, infrastructure management, and Python programming, they ensure smooth development and efficient data exports. Clients can trust Huajun's reliable support and troubleshooting, delivering exceptional results.
Portfolio
Experience
- Spark - 4 years
- Databricks - 4 years
- ETL - 4 years
- Azure - 4 years
- Data Pipelines - 3 years
- Hadoop - 3 years
- Data Engineering - 3 years
- Azure Data Factory (ADF) - 2 years
Work Experience
Senior Developer
Coles Supermarkets
- Spearheaded the design and implementation of an end-to-end data solution, resulting in a 40% increase in data processing efficiency and enhanced accessibility.
- Developed and managed robust event-driven data processing pipelines, handling large volumes of data on a daily basis and achieving a 60% reduction in processing time.
- Orchestrated and optimized ETL pipelines, resulting in a 30% faster data landing onto the cloud data lake and substantial cost savings.
Consultant
Servian
- Developed comprehensive ETL pipelines, improving data accessibility and processing speed by 80% on an on-premises Hadoop system.
- Streamlined routine tasks through automation tools, improving productivity and operational efficiency.
- Documented and implemented CI/CD best practices, reducing code deployment time and enhancing development process efficiency.
Project History
Architect Azure Solution for Cloud Migration
Education
Master's Degree in Information Technology
Monash University - Australia
Certifications
Microsoft Certified Azure Fundamentals
Microsoft
Associate Cloud Engineer
Google Cloud
CCIE Routing & Switching
Cisco
Skills
Tools
Jenkins
Frameworks
Hadoop, Spark
Paradigms
ETL, Database Design
Platforms
Azure, Databricks
Languages
Bash, SQL, Java, Python, Scala, Python 3
Storage
Apache Hive, Data Pipelines, Google Cloud
Other
Azure Data Factory (ADF), Azure Data Lake, Networking, Data Engineering, Software Development, Azure Databricks
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring