Machine Learning Developer in Singapore, Singapore

Our team worked on the problem of whether an ad's landing page is good for the user's query.

Latency is important. We cannot spend too much time computing features for our machine learning system.

One way around this is to precompute feature values. The number of possibilities is too many, so we try to reduce it using various techniques, e.g., clustering. The project involves writing MapReduce pipelines to precompute these features. It also involves a lot of A/B testing and data analysis.

Dgraph is a graph database. It is essentially an inverted index that stores relationships. I implemented key features such as filtering, sorting, and pagination. By replacing lock-free hash maps with regular hash maps and simplifying the code, I reduced data loading time by 60%. I also helped implement the caching layer, essentially a diff over the posting list stored in RocksDB.

My work on the Badger LSM key-value store is likely the fastest Go-based key-value store available. The Dgraph project was able to use Badger instead of RocksDB to avoid cgo issues.

This is one of my PhD projects. I have some new ideas recently (2017) and would like to wrap this up soon.

The FFT operation takes O(n log n) time. However, if we know that the signal is sparse in frequency space, (i.e. only a few modes), then we want to be able to perform this FFT in O(s) time, ignoring log factors, where "s" is the sparsity or the number of modes we want to recover.