Tobias Plötz

Within a cross-functional team of data scientists, full-stack developers, and business experts, I was driving the conception and technical implementation of a multi-stage computer vision system for turning depictions of chemical compounds into a machine-readable form. The system works by first detecting drawings of molecules within each page of a document, e.g., a patent. Then, the drawing is parsed into its constituents which are finally merged into a molecular graph representing the chemical compound.

Our solution is powered by modern deep learning techniques and enables the recognition of small and very large molecules. Our solution allows searches for already patented molecules by automatically screening millions of patents and extracting chemical information therein.

We focused on quickly building an MVP and iterating from there in an agile framework during development.

I built a machine learning system that automatically detects relevant documents crawled from the internet regularly.

As a special twist to this project, the systems need to adapt to each user's preference of what constitutes a relevant document while avoiding a lengthy training process and providing highly accurate predictions from the start.

To meet these goals, I employed weakly supervised learning techniques that effectively allowed us to set up an accurate user-specific classifier in less than an hour. Thus, the system could provide value to the user right away, cutting down on manual screening efforts while continuously improving through user feedback.

In this research project, I proposed a novel, a differentiable nearest neighbor lookup that can be used as a layer in any neural network. This research was published as a conference paper at NeurIPS 2018, one of the most renowned venues for machine learning research.

In my paper, the neural nearest neighbors layer is used mainly in the context of image restoration. Other researchers have applied the layer to a multitude of different problems like 3D reconstruction, graph learning, or deriving model explanations. Neural nearest neighbors generalize the well-known transformer module that powers state-of-the-art research in natural language processing and computer vision.

I have built a semantic search engine based on state-of-the-art open source NLP models to search different corpora with up to multi-million documents. This approach drastically improves the search experience upon keyword-based search by allowing users to pose natural language queries. The system also can extract answers based on relevant documents using either open source NLP models or GPT-3. In this project, I was instrumental in driving the implementation of a reusable technology stack, as well as consulting internal stakeholders on potential applications of the system.