Yi Sheng Chan

An automated, scalable, machine learning system that processes hundreds of GBs of raw behavioral data and predicts the probabilities of user churning. The system is written in Python and runs fully automated daily batch jobs on AWS. It includes security compliance, networking, data processing, model training, and model serving by Web API. Each working component (i.e., data store, web service, data pipeline, data quality, and model predictions) is monitored using various metrics and linked to PagerDuty in order to meet the service level agreement for production.

A highly complex and mission-critical system for fighting fraud in a unicorn fintech company. The system sources data from various data providers via APIs; stores the data in GraphDB, Apache Kafka, and a relational database in real time; and marks a transaction as fraud or non-fraud within one second, which is a service level agreement.

Lambda architecture for a recommendation system as a service for a SaaS company. The system includes a batch layer for hourly aggregation of data, generation of a list of recommendations for each user, and a speedy layer for real-time data consumption and generation of recommendations for each user or session. The system is written in Python, Scala, Spark, and Spark Streaming with HDFS, and it uses HBase and Apache Kafka for storing the data and model output.