Hang is available for hire
Hire HangHang Guo
Verified Expert in Engineering
Database Engineer and Developer
Seattle, WA, United States
Toptal member since May 20, 2021
Bio
Hang is a Senior Data Engineer with a proven track record of writing programs, building large-scale data systems, and extracting valuable insights from data. He excels in back-end database development, ETL, data pipeline creation, data warehousing, and business intelligence solutions, with deep expertise in performance tuning and data migration. Hang is proficient in AWS technologies, including Lambda, Step Functions, EMR, DynamoDB, Glue, S3, Redshift, SES, and API Gateway.
Portfolio
Amazon.com
Amazon S3 (AWS S3), AWS Glue, AWS Lambda, AWS Step Functions...
Amazon.com
Python, SQL, PySpark, Redshift, Amazon S3 (AWS S3), AWS Glue, AWS Lambda...
Honda
SQL, Python, SQL Server Integration Services (SSIS), JSON, APIs, C#...
Experience
- Python - 5 years
- SQL - 5 years
- Amazon S3 (AWS S3) - 4 years
- EMR - 4 years
- Amazon Web Services (AWS) - 4 years
- AWS Lambda - 3 years
- Spark - 3 years
- AWS Glue - 1 year
Preferred Environment
Linux, AWS Cloud Development Kit (CDK), Python, AWS Step Functions, AWS Lambda, Amazon S3 (AWS S3), EMR, Scala, SQL, Spark
The most amazing...
...thing was being recognized as a top-tier performer in two consecutive Amazon annual performance reviews (2024 and 2026) across two distinct internal teams.
Work Experience
Senior Data Engineer
2025 - PRESENT
Amazon.com
- Worked as a technical lead for strategically critical data infrastructure initiatives within Amazon's Devices & Services fintech organization, driving platform modernization and financial data visibility at scale.
- Designed a multi-account, multi-region framework standardizing data movement and quality validation across nine relational database clusters, two data warehouse clusters, and 11 AWS accounts—a consistent foundation for cross-organizational data migration.
- Led the architecture of a trusted data platform providing visibility into cost-of-goods-sold (COGS) data across seven product lines, enabling financial insights for 600+ programs.
- Built agentic AI applications using Amazon Bedrock and the Strands framework, powering intelligent multi-step automation for internal customer-facing products.
Technologies: Amazon S3 (AWS S3), AWS Glue, AWS Lambda, AWS Step Functions, AWS Cloud Development Kit (CDK), AWS Database Migration Service (DMS), Amazon RDS, Redshift, Amazon API Gateway, Amazon Elastic Container Service (ECS), Electronic Medical Records (EMR), Spark, SQL, Amazon EventBridge, Data Lakes, Python, TypeScript, React, Java, JavaScript, Amazon DynamoDB, DB, Database Administration (DBA), Query Optimization, Database Architecture, Dashboards, Data Cleaning, Data Transformation, Data Validation, Database Schema Design, Financial Reporting Dashboards, Workflow Automation
Data Engineer II
2022 - 2025
Amazon.com
- Achieved a 90% reduction in delivery latency. Led the design and development of data pipelines for a marketing measurement ML model, significantly improving efficiency and performance.
- Saved millions through optimization. Optimized slow-running Spark jobs, resulting in substantial cost savings and drastically reducing software product delivery latency.
- Enhanced team capabilities and efficiency. Mentored junior engineers and interns, provided critical data engineering support for data scientists and economists, and designed automation solutions for ETL workflows.
- Demonstrated advanced data warehousing skills by developing and maintaining large-scale distributed data warehouses, ensuring robust data management and accessibility.
Technologies: Python, SQL, PySpark, Redshift, Amazon S3 (AWS S3), AWS Glue, AWS Lambda, Amazon Web Services (AWS), Data Engineering, Amazon Elastic Container Service (ECS), Docker, Spark, AWS Step Functions, AWS Cloud Development Kit (CDK), EMR, Scala, React, JavaScript, ETL, NoSQL, Apache Airflow, Data Modeling, pgAdmin, Jupyter Notebook, Linux, PostgreSQL, Apache Spark, Data Pipelines, Data Migration, Transact-SQL (T-SQL), Apache Kafka, Hadoop, Apache Flink, Amazon Aurora, Amazon RDS, Data Science, Data Warehouse Design, Data Warehousing, Data Analytics, Data Architecture, Data Lake Design, Data Strategy, Distributed Systems, Unix, Data Analysis, DevOps, Pandas, NumPy, Database Design, DB, Database Administration (DBA), Query Optimization, Database Architecture, Dashboards, Data Cleaning, Data Transformation, Data Validation, Database Schema Design, Financial Reporting Dashboards, Workflow Automation
Senior Database and Data Warehouse Engineer
2019 - 2022
Honda
- Maintained and supported 150+ OLTP database ETL processes and 20+ OLAP data warehouse ETL processes. Achieved a 99.9% resolution rate for production issues, effectively addressing process failures from both application and analytics teams.
- Completed a sensitive data migration project involving PII and non-PII data. Managed secure data feeds to the cloud by decrypting sensitive information and processing the reverse feed from the cloud, ensuring data integrity and security.
- Developed a validation process to identify and load missing data into the data warehouse. This addressed data inaccuracies that had caused concerns for the business and analytics teams, restoring data integrity and trust.
- Identified and resolved the root cause of performance issues in a large back-end database store procedure returning hierarchical data for API usage. Eliminated the timeout issue and unblocked the downstream application workflow.
- Migrated a database and data warehouse to the cloud, thoroughly testing and ensuring all database and ETL programs functioned correctly post-migration, maintaining data integrity and operational efficiency.
- Developed helper programs in Python and C# to automate repeatable tasks, including database server interactions, FTP/SFTP, file system and security, cloud operations, and Spark processes, greatly reducing manual effort and increasing efficiency.
Technologies: SQL, Python, SQL Server Integration Services (SSIS), JSON, APIs, C#, Amazon Web Services (AWS), Data Engineering, Docker, Spark, SQL Server Management Studio (SSMS), Jupyter Notebook, Apache Spark, Databricks, Data Pipelines, Data Migration, Transact-SQL (T-SQL), MySQL, MySQL DBA, Microsoft SQL Server, SQL Server 2014, SQL Server 2017, Data Science, Data Warehouse Design, Data Warehousing, Data Analytics, Data Analysis, Snowflake, Pandas, Database Design, DB, Database Administration (DBA), Query Optimization, Database Architecture, Dashboards, Data Cleaning, Data Transformation, Data Validation, Database Schema Design, Financial Reporting Dashboards, Workflow Automation
Database Engineer
2019 - 2019
Honda
- Completed an ETL project in two weeks and deployed it to production; it loads large text files to a relational database and it had been left unfinished by the previous contractor due to its complexity.
- Provided business intelligence solutions by creating and maintaining business intelligence reports and data visualizations.
- Worked with the project team on database design, database programs, and development.
Technologies: SQL, Python, C#, SQL Server Integration Services (SSIS), Data Modeling, Dimensional Modeling, SQL Server Management Studio (SSMS), Jupyter Notebook, Data Migration, Transact-SQL (T-SQL), MySQL, MySQL DBA, Microsoft SQL Server, SQL Server 2014, SQL Server 2017, Data Science, Data Warehouse Design, Data Warehousing, Data Analytics, Snowflake, Database Design, DB, Database Administration (DBA), Query Optimization, Database Architecture, Data Transformation, Data Validation, Database Schema Design, Workflow Automation
Database Engineer
2018 - 2019
Helm360
- Assisted a senior database engineer with database development.
- Designed a relational data model to meet project needs.
- Contributed to the development of ETL pipelines and business intelligence reports.
- Wrote SQL functions and Stored Procedures for use by the API and ETL data pipelines.
Technologies: SQL, SQL Server Integration Services (SSIS), C#, Python, Data Modeling, Dimensional Modeling, Microsoft Power BI, Data Migration, Transact-SQL (T-SQL), Microsoft SQL Server, SQL Server 2014, SQL Server 2017, Data Warehouse Design, Data Warehousing, Database Design, DB, Database Administration (DBA), Query Optimization, Database Architecture, Data Transformation, Data Validation, Database Schema Design
Experience
Large-scale Cross-account Database Migration Platform
https://d17xl7hcexkoa6.cloudfront.net/projects/large-scale-data-migration.htmlI designed and led a large-scale, standardized migration platform that consolidated production data across Amazon's fintech organization, migrating nine relational database clusters and two data warehouse clusters across 11 AWS accounts and multiple regions into a governed target environment. It was built on AWS Database Migration Service (DMS) with full-load plus change-data-capture (CDC) replication for continuous, low-downtime cutover, backed by automated row-level data validation to guarantee parity between source and target.
Agentic AI Crew — Multi-agent Software Development Harness
https://d17xl7hcexkoa6.cloudfront.net/projects/agentic-ai-crew.htmlA multi-agent harness, powered by Claude Code, where a specialized crew of eight AI agents builds software collaboratively end-to-end—a manager that assigns and organizes the work, plus a system designer, database developer, software engineer, test engineer, deployment engineer, code reviewer, and an AI-optimizer that tunes the crew after every iteration. Each agent is scoped to least-privilege permissions and a curated set of MCP tools, keeping responsibilities and access tightly bound. The crew runs roughly 99% of day-to-day development, design, testing, and deployment autonomously. At the same time, I stay the human-in-the-loop—monitoring the harness and reviewing code and design at critical checkpoints.
Large-scale Spark Performance Tuning on Amazon EMR
https://d17xl7hcexkoa6.cloudfront.net/projects/spark-emr-performance-tuning.htmlI diagnosed and tuned large-scale Apache Spark jobs running on Amazon EMR that were slow and expensive at scale. By profiling workloads and fixing the real bottlenecks—data skew, excessive shuffles, suboptimal partitioning, and executor and memory configuration—and right-sizing EMR clusters, I dramatically cut job runtime and compute costs, saving millions of dollars annually while reducing software delivery latency.
Data Science MLOps Platform
https://d17xl7hcexkoa6.cloudfront.net/projects/data-science-mlops-platform.htmlI built an MLOps platform that gave the data science team a self-service lab for training models and running experiments at scale. AWS Batch orchestrates parallel model training and experiment execution; AWS Lambda validates results; S3 and the AWS Glue Data Catalog store and catalog historical experiment results, keeping them queryable for analysis; and CloudWatch alarms keep the platform reliable in production. The lab supports 10,000+ parallel experiments and model executions.
Inventory Data Flow and Database
TASKS ACCOMPLISHED
• Built and supported an auto inventory and data management database (OLTP).
• Created an ETL program to manage inventory and offer data flow across different environments.
• Developed database programs for the use of the Web API.
• Built and supported an auto inventory and data management database (OLTP).
• Created an ETL program to manage inventory and offer data flow across different environments.
• Developed database programs for the use of the Web API.
Support and Development of a Powersports Website Database and ETL
TASKS ACCOMPLISHED
• Supported and enhanced a back-end database and ETL process for the main web application API.
• Supported cross-environment data publishing and data validation.
• Troubleshot database performance issues caused by large database programs.
• Supported and enhanced a back-end database and ETL process for the main web application API.
• Supported cross-environment data publishing and data validation.
• Troubleshot database performance issues caused by large database programs.
Cloud Sensitive Data Migration
The project involved migrating sensitive PII information from an on-premise database to the cloud. As the main developer, I created a custom ETL package.
CUSTOM ETL PACKAGE PROCESS
• Extracted and decrypted the PII information.
• Transformed and applied the cloud business rule to the data and sent it to the destination.
• Retrieved data from the cloud and built an automation process to populate cloud data back to the on-premise database.
CUSTOM ETL PACKAGE PROCESS
• Extracted and decrypted the PII information.
• Transformed and applied the cloud business rule to the data and sent it to the destination.
• Retrieved data from the cloud and built an automation process to populate cloud data back to the on-premise database.
Weblog Data ETL Pipeline
TASKS ACCOMPLISHED
• Created ETL data pipeline loads for JSON weblogs from AWS S3 using PySpark; it then writes the Parquet files back to AWS S3.
• Developed a program that quickly reads and analyzes these Parquet files using PySpark.
The company's consumer application team uses the weblog ETL/analyze process.
• Created ETL data pipeline loads for JSON weblogs from AWS S3 using PySpark; it then writes the Parquet files back to AWS S3.
• Developed a program that quickly reads and analyzes these Parquet files using PySpark.
The company's consumer application team uses the weblog ETL/analyze process.
Jira System's Data ETL and Generation of a Campaign Calendar
PROJECT PROCESS
• Extracted the data from the old ticketing system and loaded it into the Jira ticketing system.
• Extracted the data from the Jira ticketing system daily and loaded it into a data warehouse.
• Transformed the data into a format required by the business.
• Generated a marketing campaign calendar based on the data set, and then the calendar was emailed.
• Extracted the data from the old ticketing system and loaded it into the Jira ticketing system.
• Extracted the data from the Jira ticketing system daily and loaded it into a data warehouse.
• Transformed the data into a format required by the business.
• Generated a marketing campaign calendar based on the data set, and then the calendar was emailed.
Enhancement of a Data Warehouse Lead System
This is the process that loads the feeds sent by 3rd-party vendors to a data warehouse. The old process used fuzzy logic to map the lead information between the OLTP and OLAP and sent the result to the analytics team, which caused data inaccuracies in rare cases.
We decided to enhance the logic by introducing a unique identifier between the two systems. The process involves redesigning a few tables belonging to the data warehouse and modifying the ETL process.
We decided to enhance the logic by introducing a unique identifier between the two systems. The process involves redesigning a few tables belonging to the data warehouse and modifying the ETL process.
Data Pipeline of ML Model
TASKS ACCOMPLISHED
• Set up an RDS database and ETL workflow (S3 and Lambda) to store ML model metadata.
• Built an automation workflow to ingest and merge transaction data with 3rd-party data in an EMR environment, driven by AWS Step Functions and Lambda.
• Encrypted PII using an internal library.
• Set up an RDS database and ETL workflow (S3 and Lambda) to store ML model metadata.
• Built an automation workflow to ingest and merge transaction data with 3rd-party data in an EMR environment, driven by AWS Step Functions and Lambda.
• Encrypted PII using an internal library.
Validation API Gateway
Developed an API gateway for a web app to pull validation results, handling authentication with AWS Cognito and backed by AWS Lambda. I built the infrastructure using AWS CDK, ensuring secure and efficient user authentication.
Education
2013 - 2015
Master's Degree in Information Technology
University of La Verne - La Verne, CA, United States
2006 - 2010
Bachelor's Degree in Business Administration
Guang Dong University of Technology - Guang Zhou, China
Certifications
NOVEMBER 2023 - PRESENT
React Fundamentals
Udacity
AUGUST 2021 - PRESENT
Machine Learning Engineer
Udacity
MAY 2021 - PRESENT
Full-stack Web Developer
Udacity
MARCH 2021 - PRESENT
Data Engineering
Udacity
Skills
Libraries/APIs
PySpark, Pandas, NumPy, Flask-RESTful, React
Tools
pgAdmin, AWS Cloud Development Kit (CDK), AWS Step Functions, Apache Airflow, Microsoft Power BI, PyCharm, AWS Glue, Amazon Elastic Container Service (ECS), Amazon Elastic MapReduce (EMR), Amazon Cognito, AWS CloudFormation, AWS Batch, Claude Agent SDK
Languages
Python, SQL, Scala, Transact-SQL (T-SQL), C#, C#.NET, JavaScript, TypeScript, Java, Snowflake, Python 3
Frameworks
Spark, Apache Spark, Hadoop, Flask
Paradigms
ETL, Database Design, Dimensional Modeling, DevOps
Platforms
Jupyter Notebook, AWS Lambda, Amazon Web Services (AWS), Unix, Linux, Apache Kafka, Amazon EC2, Docker, Databricks, Apache Flink
Storage
Amazon S3 (AWS S3), SQL Server Management Studio (SSMS), PostgreSQL, Amazon Aurora, Microsoft SQL Server, SQL Server 2014, SQL Server 2017, Data Lake Design, DB, Database Architecture, Data Validation, SQL Server Integration Services (SSIS), JSON, MySQL, Database Administration (DBA), Redshift, NoSQL, Cassandra, Data Pipelines, Data Lakes, Amazon DynamoDB
Other
Data Modeling, Data Analysis, Data Engineering, EMR, Amazon RDS, Data Migration, Data Science, Data Warehouse Design, Data Warehousing, Data Analytics, Data Architecture, Data Strategy, Distributed Systems, Query Optimization, Data Cleaning, Data Transformation, Database Schema Design, Workflow Automation, Big Data, MySQL DBA, Dashboards, Financial Reporting Dashboards, APIs, Business Administration, Machine Learning, Message Queues, API Gateways, Healthcare Management Systems, AWS Database Migration Service (DMS), Amazon API Gateway, Electronic Medical Records (EMR), Amazon EventBridge, Networking, Virtual Private Cloud (VPC), AWS Transit Gateway, AWS Secrets Manager
Collaboration That Works
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
1
Share your needs
Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2
Choose your talent
Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3
Start your risk-free talent trial
Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.
Top talent is in high demand.
Start hiring