Data Scientist and Machine Learning Developer in Oakland, CA, United States

Led a computer vision team of three data scientists and two engineers at the utility company PG&E. We provided AI-powered assistance for remote manual inspections of drone-captured images of the electrical grid. We also trained and deployed object detection and classification models at scale, using PyTorch models deployed with Seldon and Kubernetes in AWS. Our model pipelines ran on hundreds of thousands of images. We succeeded with multiple use cases, including 1) using model predictions to prioritize high-risk structures, 2) integrating model predictions into the inspection UI to reduce inspector error, and 3) replacing manual form questions with AI predictions to reduce manual labor. Some of the challenges we overcame were 1) optimizing model performance, 2) measuring model performance on production data, 3) sharing code across multiple model deployments, 4) deploying models to GPU, and 5) efficiently storing and analyzing tens of millions of predictions.

As the lead data scientist, my project goal was to establish a robust data science practice within the asset management group of San Diego Gas and Electric. This involved teaching by example how to manage the data science lifecycle using a specific use case to scope, build, and validate the team’s first ML app. I implemented many technical best practices such as modular coding, packaging, testing, workflow management, CI/CD, and non-technical best practices such as Agile methodologies.

The specific use case was to predict devices in the electrical grid that are nearing failure. This involved joining many disparate data stores with information on more than 200,000 transformers, including metadata pertaining to GIS, customer outage, and service records; time series of weather variables; and time series of electrical loads. The end-to-end pipeline involved cleaning, filtering, and joining data and training custom artificial neural nets (CNNs, LSTMs, autoencoders) using metadata and time series of weather and load data. The pipeline ran as a Python app using Luigi to manage the workflow, with CI/CD configured in ADO. We demonstrated accuracy that outperformed existing baselines and established previously unknown mechanistic insights.

Served as technical lead and scrum master on a team analyzing videos of high-resolution neural tissue microscopy, with members from biology, robotics, software, and data science. The goal was to build, validate, and enable CI/CD for the pipeline, and use it to measure the effects of drug perturbations, lab protocols, and genetic modifications on the activity of artificial neural tissue.

The key challenge was representing extremely high spatiotemporal resolution data via low-D, biologically interpretable metrics. We built a 12-module semi-automated pipeline (a DAG), including supervised and unsupervised CV methods constrained by biological priors, to clean and standardize the data, including auto-triggered QC that seamlessly integrated with pre- and post-processing.

Deployed as a streaming app in AWS on over 10 TB of data, it reduced QC-ed videos' failure rate by 75% and enabled us to increase the temporal resolution 10x.

For analytics, I designed two novel ML-based methods to deconfound experimental variables. I employed these pipelines to achieve the following critical endpoints for investors—demonstrating distinct effects of three neuromodulator drugs and demonstrating significant accuracy in predicting disease.

Acted as a lead data scientist for QuickBooks Online's self-help recommendation app, which required a multi-team collaboration. The goal was to surface the most relevant help articles to customers and enable them to resolve their problems from within the product. My role was to build and integrate the ML engine.

For data exploration, extraction, and feature engineering, I liaised with data science and data engineering teams to understand the multiple sources of relevant data. I wrote efficient PySpark code to ingest and transform high volumes of clickstream (billions of rows), customer profile data, and help article databases.

For model training, I employed a novel deep learning approach consisting of shared layers, LSTMs, and merging temporal sequences with static features.

To productionize the model, I led a team consisting of other DS contributors as well as front-end and back-end developers, and members of performance testing and A/B testing teams. Together we integrated the model with the existing click data streams, built I/O specs, ensured adequate stability and response latency, and measured significant improvements in customer engagement (55% higher clickthrough on articles) and support metrics (10% lower call rates).