Senior Machine Learning Engineer2020 - 2021Agoda
Technologies: Spark, Python, Scala, Apache Hive, Hadoop, Object-oriented Programming (OOP), Data Structures, Agile Software Development
- Developed data pipelines to process massive data (100+ TB/run) from suppliers' data that is often incorrect and fragmented to map rooms with 99.9% certainty by incorporating machine learning models and various matching algorithms.
- Worked closely with data scientists to train and integrate new machine learning models into current property mapping streamline targeting properties in the Chinese language.
- Debugged various memory, performance, and scalability issues in Spark Jobs.
Senior Machine Learning Engineer, Technical Lead2020 - 2021Agoda
Technologies: Scala, Python, Spark, Apache Hive, Machine Learning, Big Data, Data Pipelines, React, SQL, Technical Leadership, Engineering Management, Agile Software Development
- Designed and implemented new pipelines for current Spark Jobs where each task can be executed independently thus increasing the stability of production processes by over 50%, and at the same time, requiring to no dev changes to current processes.
- Refactored Spark Job into a producer/consumer pattern which gave the ability to control how much resources each run should consume. It also allowed the execution of several instances in parallel, thus increasing the processing throughput by 25%.
- Architected and developed a web application with a workflow for the Operations team that converted the process of adjusting the filtering algorithm in the room mapping streamline Spark Job into a self-service process, saving 2-3 development hours per week.
- Collaborated with product owners to develop various hypotheses for displaying mapping data in the UI and implemented A/B experiments to verify hypotheses resulting in 100+ bookings increase per day.
- Optimized core ETL job that processes around 300 million records/day, reducing memory usage by 85%.
Software Engineer2020 - 2020Clients (via Toptal)
Technologies: wxWidgets, Perl, C++
- Designed and developed a new C++ UI and framework to replace the client's legacy process to optimize the DeltaV workflows for automation, optimizing operational efficiency and providing a better user experience.
- Deployed software as a Windows application with Microsoft Installer to encapsulate all existing process dependencies, removing manual setup and configuration for end-users.
- Packaged the application into a Microsoft Installer that sets up all its dependencies, allowing end-users to immediately get going.
- Assessed various licensing models and implementation roadmaps leveraging different solution stack to develop final recommendations for the client.
Lead Software Engineer2016 - 2019FactSet
- Initiated the project to create a unified web application, Portfolio Reporting Batcher, to replace four legacy batching apps, delivering a one-stop solution for clients' batching workflows.
- Designed a long running infrastructure to offload Portfolio Reporting Batcher's complex operations to background processes, significantly increasing the success rate of job runs by 40%.
- Designed a long-running infrastructure to offload portfolio reporting batcher complex operations to background processes, significantly increasing job runs' success rate by 40% (Node.js, Redis).
- Created a centralized Batch API to orchestrate and manage 50,000+ reports per job, reducing code redundancy by over 30%.
- Led the design and development of an automated conversion system to seamlessly migrate 10,000+ client documents and jobs onto the next-gen platform, saving product managers six months of manual work.
- Collaborated with project managers and other engineering groups to plan and prioritize projects for engineering teams.
Senior Software Engineer2011 - 2016FactSet
- Built a prototype web application over one weekend and convinced senior management to invest in migrating the legacy Portfolio Publisher to a new web app.
- Redesigned legacy Portfolio Batcher infrastructure from running jobs in-process to running with distributed services, improving scalability and stability without disrupting clients’ production processes.
- Created a unified API to replace the different legacy frameworks used by three Portfolio Publisher applications, reducing the number of client issues by more than 50%.
- Optimized the Publisher Document Manager application, shortening the runtime of loading 9000+ documents from an hour to two minutes.
Software Engineer2007 - 2011FactSet
Technologies: C++, Data Structures
- Developed both the front end and back end of the portfolio publishing applications to implement major enhancements and solve client issues.
- Built private wealth manager application from scratch, introducing FactSet into the wealth management space, now becoming the foundation of the FactSet next-generation wealth solutions.
- Optimized portfolio lookup widgets, improving performance in speed and space by more than 50%.