Senior Data Engineer
2022 - PRESENTCurvo Labs- Developed and maintained the company's data platform, including core processing and serving pipelines with all infrastructure as code.
- Designed and developed a nonlinear regression with a Spark data pipeline to calculate potential savings likelihoods.
- Created back-end APIs for product features and standard libraries for other developers.
Technologies: Scala, TypeScript, Python, SQL, AWS Step Functions, AWS Lambda, Terraform, AWS Glue, Spark, NestJS, PostgreSQL, Docker, CircleCI, Bash, Git, Data EngineeringData Engineer
2020 - 2022Techcombank- Designed and implemented a system that migrated data from on-premise databases to a cloud data lake that scaled to the scheduled pulling of thousands of tables with different change-data-capture patterns and throughput requirements.
- Created operations processes and implemented validation for ad-hoc file upload.
- Gathered requirements; designed and built an end-to-end BI performance tracking dashboard that ingested and processed data from multiple heterogeneous sources.
Technologies: Scala, Java, Python, SQL, Spark, AWS Glue, AWS Lambda, Apache Airflow, PostgreSQL, JanusGraph, Docker, Microsoft Power BI, Apache Beam, Bash, Git, Data Engineering, Data Visualization
Cuong Dang
Data Visualization Developer in Waverly, IA, United States
Member since November 14, 2022
Cuong is a well-rounded data engineer who delivers complete solution packages, including design, code, documentation, testing, and infrastructure. He is passionate about theory and practice and commits to lifelong learning, especially in computer science and mathematics. One of Cuong's highlightable experiences includes a scalable system that migrated data from on-premise databases to a cloud data lake. He is also proficient in back-end development.
Cuong is now available for hire
Portfolio
- Curvo LabsScala, TypeScript, Python, SQL, AWS Step Functions, AWS Lambda, Terraform...
- TechcombankScala, Java, Python, SQL, Spark, AWS Glue, AWS Lambda, Apache Airflow...
Experience
- SQL 5 years
- Python 3 years
- Data Engineering 2 years
- Spark 2 years
- Scala 2 years
- Data Visualization 2 years
- Terraform 1 year
- C 1 year
Availability
Part-time
Preferred Environment
Scala, SQL, AWS Step Functions, Terraform, Spark, AWS Glue, AWS Lambda, Bash
The most amazing...
...system I've designed and implemented is a data system migrating thousands of tables from on-premise to the cloud for one of the largest banks in the country.
Employment
Experience
- On-premise to Cloud Data Migration System
A data migration system that scaled to scheduled pulling of thousands of tables from multiple on-premise databases to the cloud data lake's landing zone. I was the primary designer and implementer of the system.
The goal was to maximize automation that allows Data Operations staff to add source data tables to be loaded quickly. The system was composed of the main components:
• Directed Acyclic Graph (DAG) Generator—induced Airflow DAGs.
• Extractor—pulled data from sources based on different specified change-data-capture (CDC) patterns.
• Uploader—picked up data from buffer storage and loaded them to the cloud landing zone.
• Validator— validated files in file upload jobs.
Skills
Languages
Scala, TypeScript, Python, SQL, Java, C, SchemeFrameworks
Spark, NestJSTools
AWS Step Functions, AWS Glue, Microsoft Power BI, CircleCI, Git, Terraform, Apache Airflow, Apache BeamPlatforms
AWS Lambda, DockerStorage
PostgreSQL, JanusGraphOther
Data Engineering, Data Visualization
Education
- Bachelor's Degree in Economics2008 - 2014Yonsei University - Seoul, South Korea
