Julián Peller

I am a Kaggle Notebook Grandmaster, the first and only from my country. I have published various popular notebooks in Kaggle—with more than 3600 upvotes and 5400 forks—reaching the rank of 7th of 200,000 on that scoreboard.

My main contributions are:

• PyTorch transformers models for the token classification NLP competition "Feeback Prize—Evaluation Student Writing": Sentence Classifier baseline, ShortFormers w/Chunks. Other notebooks for this competition: Roberta Intra-task pre-training, Topic Modeling with LDA.

• A straightforward and high-scoring Naive Bayes model that opened the path to simple linear models which surprisingly ended up performing similarly to transformers in the NLP competition "Jigsaw Rate Severity of Toxic Comments."

• A series of notebooks covering various preprocessing steps and explaining different public models in the NLP ML QA competition "Chai—Hindi and Tamil QA."

• PyTorch models for the instance segmentation competition "Sartorius—Cell Instance Segmentation.": Masked R-CNN, U-net, Classifier, and Masked R-CNN.

• PyTorch models for the object detection competition "TensorFlow—Help Protect the Great Barrier Reef": FasterRCNN, DETR, and an intelligent cross-validation strategy proposal.

Prod2Vec is an innovative application of Word2vec to the recommender systems domain. Word2vec uses a large corpus of language sentences to learn low dimensional words embedding. The key insight is that: similar words have similar words around them.

Prod2Vec uses the same concept, but we want embeddings for products instead of word embeddings. Instead of sentences (sequences of words), we use different sequences of products like user navigation or purchase histories to learn them.

With the low dimensional embeddings, we can calculate similarities and neighborhoods among products, which is the core of an item-item recommender model.

In Navent, I implemented this recommender system from scratch using Gensim. A few tweaks were necessary once the POC successfully reached a reasonable computation time. First, K-means was used to cluster the items in groups to avoid calculating the similarities among all the products. Second, Annoy—an approximate neighborhood library by Spotify—which trades some accuracy for performance, replaced the usual nearest neighbors module.

The model obtained a 17.5% increase in CTR against the previous productive model during the A/B test and is currently in production.

In Trivial Crack, users can create content themselves. As it is called, the factory is very engaging, but on the other hand, it opens the door to low quality or even nocive content, so there is a manual moderation process to filter this content out.

My onboarding project at Etermax was an image moderation tool; a binary classifier built using Fast.ai. The tool allowed us to provide the images to the moderator ordered by the likelihood of approval and, finally, to fully automatize this tedious task. Moreover, the model increased the throughput of the content-creation pipeline and provided us with a more extensive set of high-quality questions with images (known to be more engaging).

Soon the project became productive, and I became enthusiastic about Fast.ai. After this, I took the rest of the courses and started a personal side project with ULMFit.

The productive model uses a simple ResNet34 and handles duplicated images with contradictory labels using p-hashes. It reaches 90% accuracy with a simple training trick that runs a few epochs over shrunk versions of the final-sized images, a kind of intermediate pre-training before the final fine-tuning.

I answered various Python and data science questions in Stack Overflow. Currently, I am in the top 10% answerers of Python and ML in the platform and the top 20% answerers regarding Pandas and scikit-learn.

GPT2 is a deep learning language model released in 2019 and 2020 by OpenAI. It obtained a lot of attention because, in the original release, they showed a handful of human-level language generation examples of an impressive length and complexity. Furthermore, they did not publish the pre-trained weights worried about the harm potential of such an advanced language model. They used this milestone to propose a broad discussion about the dangers of advanced NLP.

My experiment consisted of running the GPT-2 with the largest available version by that time (774M weights) over the blog post examples' conditional inputs to check if the actual output was as good as it was shown. I used the most advanced known sampling mechanisms for language generation: Top-k and Top-p sampling, with different configurations. Sadly, no results were as good as the ones in the blog post and, although results were good and impressive, they were not that impressive. There are some challenging theoretical problems involved in NLG with Deep Networks, as the author of Top-p sampling shows in his paper. For these experiments, I used Hugging Face's library transformer.

Recently at Etermax, we have developed a win-rate predictor. Given a user and a question, the win-rate predictor model should assess whether the user will or won't answer the question correctly and with what probability.

We used MLflow for hyper-parameter tuning and publishing and exposing the productive model trained from Airflow. The model is stored by MLflow in S3 and picked up by a Flask-Gunicorn API. Initially, we used the API service provided by MLflow, but since a request to Dynamo was needed before accessing the model itself, we had an extra HTTP request, which made everything too slow. On the other hand, it was not easy to integrate with New Relic, so we branched out a customized version of MLflow API.

We started with a latency of 400 ms and a requirement to answer in less than 80 ms to keep it as an online model. Using Locust.io as a stress test framework and New Relic to disaggregate the times, we could reach 20 ms of response time under everyday stress compromising minimal accuracy.