Senior Data Engineer
2022 - PRESENTCurvo Labs- Developed and maintained the company's data platform, including core processing and serving pipelines with all infrastructure as code.
- Designed and developed a nonlinear regression with a Spark data pipeline to calculate potential savings likelihoods.
- Created back-end APIs for product features and standard libraries for other developers.
Technologies: Scala, TypeScript, Python, SQL, AWS Step Functions, AWS Lambda, Terraform, AWS Glue, Spark, NestJS, PostgreSQL, Docker, CircleCI, Bash, Git, Data EngineeringData Engineer
2020 - 2022Techcombank- Designed and implemented a system that migrated data from on-premise databases to a cloud data lake that scaled to the scheduled pulling of thousands of tables with different change-data-capture patterns and throughput requirements.
- Created operations processes and implemented validation for ad-hoc file upload.
- Gathered requirements; designed and built an end-to-end BI performance tracking dashboard that ingested and processed data from multiple heterogeneous sources.
Technologies: Scala, Java, Python, SQL, Spark, AWS Glue, AWS Lambda, Apache Airflow, PostgreSQL, JanusGraph, Docker, Microsoft Power BI, Apache Beam, Bash, Git, Data Engineering, Data Visualization