Deep Learning Developer in Houston, TX, United States

I built a superlearner ensemble of over 100 models to classify six different kinds of toxicity in Wikipedia forums using PyTorch. These models included a variety of Deep Pyramidal Convolutional Neural Networks (DPCNN), Hierarchical Attention Networks (HAN), Recurrent Neural Networks (RNN), and Naive-Bayes Support Vector Machines (SVMs) all trained with various combinations of Word2Vec, GloVe, and FastText word vectors. With this ensemble I held 1st place out of over 4000 competing teams for over a month, scoring 0.9875 ROC AUC and roughly 99% accuracy. In addition, I investigated novel data augmentation techniques using text translation and embeddings. Learning from my fellow Kaggle competitors, to augment my text data, I used the Google Translate API to translate text samples to some intermediary language, then back to English.

To help with my various PyTorch projects, I built my own custom front-end for PyTorch that behaves a lot like the library Keras on the user-side. However, the underlying implementation of the front-end is significantly more efficient and allows for use of PyTorch's API for building and designing dynamic neural networks. This library covers all aspects of the deep learning pipeline including loading and processing data, data augmentation, streamlined model design and construction, and training and inference workflows. This library is well planned and built on principles of functional and object-oriented programming. Currently, I am working on increasing my unit and integration test coverage as well as completing all documentation so I can add it to the PyTorch ecosystem.

I developed an AI to play chess using my novel Dynamic Policy Net model. The model's weights are initialized with games of grandmasters from the FICS chess server and uses the REINFORCE learning algorithm to learn from self-play. The novelty of the model comes from its design, allowing it to generate a probability distribution over a variably sized action space. The overall training algorithm and design is inspired by the work of Google DeepMind on the Go-playing AI AlphaGo. My deep chess bot plays at a 1650 elo (top 4%) only looking ahead to the next move.

I adapted deep learning techniques to classify images of leaves out of 99 species with only about 10 images per class. Images were binary images that contained the shape of each leaf. Since the dataset was so small with such a large number of output classes, standard convolutional networks did not train properly. Instead, I combined a convolutional network with extracted shape features from each image. In addition, I used affine transforms for image augmentation to further improve the robustness of my model. My final model placed 37th out of 1600 competitors, scoring 99.9% accuracy. My solution was featured on Kaggle’s No Free Hunch blog for its creativity and insightfulness.

My teammates and I investigated the use of recurrent and convolutional neural networks to generate word embeddings using character-based information. The characters were passed as a sequence into the neural encoder to produce the word vector for the given word. Once the word vector was obtained, the rest of the workflow was similar to Google's skip-gram Word2Vec model. We predicted which words occur in the same context as our given word using our word vector and the negative contrastive loss. The recurrent neural encoders performed better than skip-gram Word2Vec on tasks that required understanding morphological structure in words.

I researched a novel algorithm for training a non-differentiable neural network with binary activations. Due to a non-disclosure, I cannot reveal specific details about the algorithm, but the work showed promise because it paralleled much of what is observed in biological brains.