Clayton is available for hire
Hire ClaytonClayton Lemons
Verified Expert in Engineering
Software Developer
Location
Ithaca, NY, United States
Toptal Member Since
April 23, 2020
Clayton is a transformative software engineer and leader with over 15 years of experience in the software industry, innovating at the intersection of data, AI/ML, and cloud engineering. As both an individual contributor and a visionary leader, he is a dynamic force in transforming challenges into user-centric, high-quality software solutions. Renowned for his technical mastery, strategic foresight, and principled approach to software, Clayton elevates teams to collaborate and perform their best.
Portfolio
U.S. Department of Veterans Affairs
Big Data, Databases, ETL Tools, Data Management, IT Strategy, Cloud
Elevance Health
Code Review, Coaching, Career Coaching, Technical Leadership, Software Design...
Elevance Health
JupyterLab, Jupyter Notebook, Spark, PySpark, MLflow, Feast, Pachyderm...
Experience
Availability
Part-time
Preferred Environment
Full-stack, TypeScript, PostgreSQL, Docker, Scala, Python, Kubernetes, Amazon Web Services (AWS), React, Machine Learning Operations (MLOps)
The most amazing...
...solution I've architected and developed is an internal AI/ML platform deployed on Kubernetes, leveraging EKS, JupyterHub, Spark, MLflow, Feast, and Pachyderm.
Work Experience
Enterprise Strategy Architect
2023 - PRESENT
U.S. Department of Veterans Affairs
- Produced a detailed technical strategy document for a large-scale, cloud-based data and analytics platform.
- Revised existing and created new architecture diagrams in support of the technical strategy document.
- Planned the pilot of a novel synthetic data generation system.
Technologies: Big Data, Databases, ETL Tools, Data Management, IT Strategy, Cloud
Senior Director of AI Technology
2023 - 2023
Elevance Health
- Managed an 8-member engineering team skilled in AI/ML, data science, back-end API development, and DevOps, fostering a diverse technical environment.
- Mentored eight engineers in technical and leadership skills, enabling two engineers to achieve technical leadership roles.
- Oversaw the maintenance of a cloud-native, elastic AI/ML platform built on JupyterHub and a data pipeline that processed HL7 FHIR resources for over 70 million patients, handling billions of claims and clinical records.
- Headed a cross-functional team to develop a research-focused data science platform on Google Kubernetes Engine, ensuring secure access to deidentified data for over 70 million patients.
- Designed a secure solution using Kasm to prevent data exfiltration from container-based JupyterLab workspaces running on the data science platform.
- Containerized a customized JupyterLab environment for the data science platform equipped with an extensive set of AI/ML tools.
- Architected a secure, multi-tenant persistent storage system to support both user and shared project data in JupyterLab workspaces, employing Google Cloud Storage buckets mounted with gcsfuse.
- Engineered a deployment strategy for an innovative synthetic data generation system.
- Served as the cloud engineering lead for an LLM-powered, internal knowledge management tool for CSR queries, providing solutions for both the LLM's development and its deployment and integration with the tool's front-end application.
- Implemented Ray on Kubernetes with autoscaling and GPU support and showcased its ability to fine-tune a 70 billion parameter Llama 2 LLM using the Fully Sharded Data Parallel (FSDP) technique.
Technologies: Code Review, Coaching, Career Coaching, Technical Leadership, Software Design, Software Architecture, Feedback Review, Agile, Scrum, Software Project Management, Kubernetes Operations (kOps), Director, Software Engineering, Cross-functional Team Leadership, Strategic Planning & Execution, Idea Synthesization and Application, Staff Management & Development, Goal Management, Project Coordination, Business Requirements, Open-source LLMs, Large Language Models (LLMs), New Product Development, Google Kubernetes Engine (GKE), Google Cloud Platform (GCP), Google Cloud Storage, JupyterLab, Jupyter Notebook, React, Python, FastAPI, Terraform, Cross-functional Collaboration, Datadog, Ray, PyTorch, Llama 2, Fine-tuning, Data-level Security, Data Exfiltration Prevention, Git, Cloud Deployment, Full-stack Development, Leadership, Team Leadership, Remote Team Leadership, Snowflake, Protegrity, Retrieval-augmented Generation (RAG), ChatGPT, Chatbots
AI Software Engineering Lead
2021 - 2023
Elevance Health
- Led the architecture and team efforts to build and deploy a cloud-native AI/ML platform on AWS for enhanced data science on scalable infrastructure, integrating technologies such as Kubernetes, JupyterHub, Spark, MLflow, Feast, and Pachyderm.
- Secured cloud accounts for the AI/ML platform deployment, working in close collaboration with internal committees and DevSecOps teams to ensure compliance and governance alignment.
- Spearheaded the deployment of JupyterHub on Amazon EKS in close collaboration with DevSecOps engineers, leveraging EFS, EBS, S3, and custom Docker images to meet specific user environment needs.
- Developed a single sign-on (SSO) solution integrating JupyterHub and MLflow via Auth0, facilitating a seamless authentication experience for users.
- Developed a high-performance computing solution for users of the AI platform, seamlessly integrating on-demand, scalable GPU resources and elastic, Kubernetes-hosted Spark jobs with JupyterHub.
- Proposed and contributed a security solution to Pachyderm, improving the security posture of its "JupyterLab Pachyderm Mount Extension" and making it easier for users to integrate.
- Influenced Pachyderm's development roadmap by identifying numerous performance issues and suggesting new features and optimizations, several of which were implemented.
- Conducted comprehensive training sessions for data scientists and engineers on the effective use of the AI/ML platform, enhancing team capabilities.
- Directed a cross-functional team on the development and operationalization of a predictive model for type 2 diabetes on the AI/ML platform, successfully advocating for the use of Feast and MLflow to achieve MLOps best practices.
- Led the architectural design and development of a new FHIR generation pipeline to replace an old one, cutting processing time from three weeks to 24 hours for 70 million patients and substantially lowering operational costs.
Technologies: JupyterLab, Jupyter Notebook, Spark, PySpark, MLflow, Feast, Pachyderm, Amazon EFS, Amazon EKS, Amazon EBS, AWS ELB, AWS IAM, AWS CLI, Amazon S3 (AWS S3), AWS Auto Scaling, Python 3, Python, GPU Computing, Distributed Computing, Big Data, Machine Learning Operations (MLOps), Technical Leadership, Architecture, Software Design, Machine Learning, GitLab CI/CD, GitLab, Helm, Artificial Intelligence (AI), Single Sign-on (SSO), Artifactory, OCI Artifact Registry, Docker, Docker Hub, Docker Compose, DevOps, HL7 FHIR Standard, Optimization, Git, Cloud Deployment, API Integration, Team Leadership, Remote Team Leadership, Leadership, Protegrity, Snowflake
AI Solutions Engineer Executive Advisor
2020 - 2021
Anthem
- Conducted comprehensive research to identify Pachyderm as an enterprise-grade COTS software solution that met specific needs for pipeline orchestration, distributed processing, incremental processing, and data tracking and provenance.
- Drove the procurement process for Pachyderm, successfully navigating licensing, negotiations, and acquisition.
- Directed and assisted a team of cloud engineers with the deployment of Pachyderm within the enterprise's AWS cloud infrastructure, specifically leveraging Amazon EKS.
- Oversaw the operationalization of Pachyderm, establishing robust processes and best practices for building and executing large-scale data pipelines.
- Standardized and documented data engineering best practices for the organization.
- Designed an internal, patient-focused health trajectory data structure, significantly enhancing data scientists' ability to rapidly analyze data and develop AI-driven health models.
- Led the design and development of a Pachyderm pipeline that hydrated the above health trajectory data structure with data from over 70 million patients, encompassing 2+ TB of data.
Technologies: Spark, PySpark, Kubernetes, Docker, Amazon EKS, Python 3, Python, Apache Airflow, Pachyderm, Data Pipelines, CI/CD Pipelines, GitLab CI/CD, GitLab, Flake8, Pytest, Black, pre-commit, Orchestration, Data Structures, Big Data, Data Science, Code Review, Debugging, Software Testing, Distributed Computing, Distributed Software, Negotiation, Procurement, COTS, Enterprise SaaS, Provenance, Data Lineage, Artificial Intelligence (AI), IT Security, Technical Leadership, Git, Cloud Deployment, Snowflake, Protegrity
AI ETL Solutions Engineer via Toptal
2020 - 2020
Anthem AI - Telehealth/PIP
- Spearheaded the design and execution of a complex data pipeline that enables the seamless delivery of AI-driven health insights from an on-prem server to a cloud-based application.
- Engineered a Python-based API and storage framework for the storage and retrieval of AI-driven health insights on Amazon S3, utilizing compression, Base64 encoding, and indexing for flexibility and efficiency.
- Enhanced data science operations by providing expert ETL and ML pipeline engineering support to data scientists in the form of code reviews, debugging, pair programming, and performance optimization.
- Maintained on-prem ETL pipeline components built using Hive, PySpark, and Airflow.
- Advocated successfully to leadership for the transformation of an on-prem ETL pipeline to a cloud-native solution, leveraging Snowflake, Kubernetes, PySpark, and Pachyderm.
Technologies: Python, SQL, ETL, PySpark, Data Analytics, Data Modeling, Data Profiling, Spark, Apache Hive, Hadoop, Data Pipelines, GitLab CI/CD, CI/CD Pipelines, Apache Airflow, Kubernetes, Pachyderm, Snowflake, Amazon S3 (AWS S3), APIs, Frameworks, Storage, Artificial Intelligence (AI), Machine Learning, Big Data, PyTorch, Python 3, Code Review, Debugging, Pair Programming, Advisory, Amazon Web Services (AWS), AWS Step Functions, AWS Lambda, Bitbucket, Git, Cloud Deployment
Research Software Engineer (Machine Learning)
2018 - 2020
GrammaTech
- Accelerated the static and binary analysis of a large-scale codebase by implementing a data and ML pipeline using Python, MongoDB, and JavaScript.
- Reduced computational costs for a binary analysis program by implementing a pupil-style ML model with scikit-learn.
- Crafted data analysis techniques in Python to detect security issues in JavaScript functions, such as swapped callback and error arguments in higher-order continuation-style functions.
- Leveraged the Doc2Vec model to vectorize function call sites and definitions, streamlining the detection of swapped arguments through semantic similarity of parameter and argument names.
- Developed the back end of a feature for binary scanning in a SaaS binary analysis tool.
Technologies: JavaScript, GitLab, Docker, C++, MongoDB, Keras, TensorFlow, Python, Scikit-learn, Machine Learning, Statistical Analysis, Bash Script, GitLab CI/CD, CI/CD Pipelines, NumPy, Big Data, Parallel Programming, Git
Software Engineer I – III
2013 - 2020
National Instruments
- Earned recognition for outstanding performance, receiving the "Rookie of the Year" award, multiple fast-track promotions, and the opportunity to lead a key project.
- Standardized and streamlined the firmware downloading framework across multiple devices in the NI-DCPower and NI-DMM product families.
- Designed and implemented over 20 features in the NI-DCPower and NI-DMM driver APIs for Windows, with many improvements directly visible to the end user.
- Led the research and definition of three major features for a key product, collaborating with project managers across hardware teams and various stakeholders to ensure alignment and address technical requirements comprehensively.
- Designed and led the development of an internal programming language and compiler that targeted a proprietary instruction set for power supply output control, enabling flexible device behavior reconfiguration and complex output control.
- Implemented a client-server system that enables remote management of NI-DCPower API driver sessions, facilitating debugging and introspection.
- Improved developer workflow efficiency by implementing a Sublime Text plugin to integrate Perforce.
- Developed a VS Code extension that integrates NI's custom build system with Microsoft's C/C++ extension, enabling advanced features like semantic code completion.
- Researched the Tarantula fault localization technique, successfully created a prototype for select NI codebases, and showcased the findings at an internal engineering conference.
- Mentored more than 10 interns and junior engineers.
Technologies: Compiler Design, API Design, Firmware, Development, Windows, Visual Studio, Python, C++, Perforce, Windows Kernel Drivers, TypeScript, Sublime Text 3, Visual Studio Code (VS Code), Leadership, Team Leadership
Web Developer
2012 - 2013
CleanTelligent Software
- Optimized several database queries and storage layouts, including the file storage system for customer photos, which reduced several API response times to just milliseconds.
- Implemented the back-end API for a customizable report generation tool.
- Applied a new UI theme to several parts of the website.
Technologies: Apache Struts 2, HTML, CSS, JavaScript, Jakarta Server Pages (JSP), Java, PostgreSQL, CVS, Full-stack Development
Software Engineer Intern
2012 - 2012
National Instruments
- Designed an essential kernel driver feature that streamlined driver communication with embedded storage devices on over 10 commercial products.
- Developed a code generation tool to support the driver feature, which automatically leveraged Python and Mako templates to generate C++ and LabVIEW code from metadata.
- Investigated and presented the advantages and disadvantages of various metadata schema formats for the code generation tool, then led a consensus meeting to select the most suitable one.
Technologies: LabVIEW, Ruby, C++, Python, Mako, Ruby ERB, Perforce, Windows Kernel Drivers
Web Developer
2011 - 2012
CleanTelligent Software
- Pinpointed and documented multiple user experience inconsistencies across related functionalities and addressed the corresponding issues successfully.
- Broadened the capabilities of a crucial job scheduling tool by integrating additional back-end queries and introducing new user interface elements on the front end.
- Resolved over 30 bugs throughout the CleanTelligent website's front end and back end, enhancing overall performance and user experience.
Technologies: JavaScript, Apache Struts 2, HTML, CSS, Jakarta Server Pages (JSP), Java, PostgreSQL, CVS, Full-stack Development
Experience
Capstone Project for the Coursera Course "Functional Programming in Scala"
https://github.com/claytonlemons/fp-in-scala-capstoneFor my capstone project in a Scala functional programming course, I developed a Scala/Spark project to extract and process global temperature data. The goal was to create interactive, zoomable visualizations showcasing temperature patterns and deviations. This involved leveraging Scala for the functional programming aspects and Spark for handling large datasets efficiently, enabling dynamic and informative visualizations of climate trends worldwide.
NI-DCPower Soft Front Panel Debug
http://www.ni.com/en-us/innovations/white-papers/14/introducing-debug-driver-session-technology.htmlI developed a Python/C++ client-server system for remote debugging and introspection of NI-DCPower API sessions across separate processes. The system features an optional server launched by the primary NI-DCPower session and a client that mirrors the session's API, essentially acting as a proxy. This system serves as the underpinning of the NI-DCPower Soft Front Panel Debug feature.
I implemented the project by creating a DLL that initiates an Apache Thrift server for command relay between remote and driver sessions over localhost, enabling seamless integration of debugging tools. Thrift's flexibility was key for future extensions to support connections from non-local processes.
I collaborated closely with LabVIEW and C# developers, who consumed the client API in order to implement the NI-DCPower Soft Front Panel Debug feature.
I implemented the project by creating a DLL that initiates an Apache Thrift server for command relay between remote and driver sessions over localhost, enabling seamless integration of debugging tools. Thrift's flexibility was key for future extensions to support connections from non-local processes.
I collaborated closely with LabVIEW and C# developers, who consumed the client API in order to implement the NI-DCPower Soft Front Panel Debug feature.
Subforce
https://github.com/claytonlemons/SubforceI created a Sublime Text plugin to integrate Perforce VCS features directly within Sublime Text while at National Instruments, aimed at boosting team workflow efficiency. This involved researching both Perforce and Sublime Text APIs and crafting features to streamline common developer tasks. This effort enhanced the development experience for my team members, who preferred using Sublime Text for their coding activities. I also worked with Sublime Text's Package Control community to get the plugin added.
GMF Aquatics Website
I developed a prototype website for GMF Aquatics to showcase and sell their aquarium products, utilizing Amazon's merchant API. This pro bono project was intended to assist a friend in marketing their products more effectively. The website was engineered using CakePHP on the back end for scaffolding and rapid development. For the front end, I employed a combination of Bootstrap for a responsive design, Backbone.js for a structured web application, and Mustache for template rendering. This blend of technologies resulted in a simple but effective platform for highlighting and exploring aquarium products.
Skills
Languages
Python, C++, Bash, TypeScript, PHP, CSS, Ruby, Java, Scala, GraphQL, HTML, JavaScript, Bytecode, Python 3, Embedded C, C, Bash Script, SQL, Snowflake, C#
Platforms
Kubernetes, Docker, Amazon Web Services (AWS), Cloud Native, Windows, NetBeans, Software Design Patterns, Linux, AWS Lambda, Jupyter Notebook, OCI Artifact Registry, Director, Google Cloud Platform (GCP), Visual Studio Code (VS Code)
Other
Windows Kernel Drivers, Software Architecture, Back-end, Machine Learning, Cloud, Development, Full-stack, Firmware, API Design, Concurrent Computing, Thread Scheduling, Processing & Threading, Code Validation, Data Mining, Software Project Management, Data Engineering, Programming, Operating Systems, Data Structures, Software Design, Security, Asymmetric Encryption, Server Development, Non-blocking I/O, Natural Language Processing (NLP), Compilers, Digital Signal Processing, Computer Engineering, Algorithms, Machine Learning Operations (MLOps), Statistical Analysis, CI/CD Pipelines, Big Data, Data Analytics, Data Modeling, Data Profiling, Pachyderm, APIs, Frameworks, Storage, Artificial Intelligence (AI), Code Review, Debugging, Advisory, Flake8, Black, pre-commit, Orchestration, Distributed Software, Negotiation, Procurement, COTS, Enterprise SaaS, Provenance, Data Lineage, IT Security, Technical Leadership, Functional Design, Kubernetes Operations (kOps), Open Source, Container Orchestration, Troubleshooting, Scheduling, Site Reliability Engineering (SRE), System Administration, Containerization, JupyterLab, MLflow, Feast, AWS Auto Scaling, GPU Computing, Architecture, Single Sign-on (SSO), Optimization, Coaching, Career Coaching, Feedback Review, Software Engineering, Cross-functional Team Leadership, Strategic Planning & Execution, Idea Synthesization and Application, Staff Management & Development, Goal Management, Project Coordination, Business Requirements, Open-source LLMs, Large Language Models (LLMs), New Product Development, FastAPI, Cross-functional Collaboration, Ray, Llama 2, Fine-tuning, Data-level Security, Data Exfiltration Prevention, Mako, Version Control, Plugin Development, Client-server Model, DLL, Device Drivers, Full-stack Development, API Integration, ETL Tools, Data Management, IT Strategy, Leadership, Team Leadership, Remote Team Leadership, Protegrity, Retrieval-augmented Generation (RAG), ChatGPT, Chatbots
Frameworks
Spark, Flask, Hadoop, Jakarta Server Pages (JSP), Apache Struts 2, CakePHP, Bootstrap, Apache Thrift
Libraries/APIs
TensorFlow, Keras, NumPy, SciPy, Pandas, Scikit-learn, PySpark, React, PyTorch, Ruby ERB, Backbone.js, Mustache, Windows API
Tools
Sublime Text 3, Visual Studio, Gensim, Git, Perforce, GitLab, GitLab CI/CD, LabVIEW, Apache Airflow, Amazon EKS, Pytest, Helm, Logging, AWS Step Functions, Bitbucket, Amazon EBS, AWS ELB, AWS IAM, AWS CLI, Artifactory, Docker Hub, Docker Compose, Google Kubernetes Engine (GKE), Terraform, CVS
Paradigms
Functional Programming, Concurrent Programming, Compiler Design, Distributed Programming, Software Testing, Parallel Programming, ETL, Pair Programming, Data Science, Distributed Computing, DevOps, HL7 FHIR Standard, Agile, Scrum
Storage
PostgreSQL, MongoDB, Cloud Deployment, Databases, Apache Hive, Data Pipelines, Amazon S3 (AWS S3), Amazon EFS, Google Cloud Storage, Datadog
Education
2014 - 2016
Master of Science Degree in Software Engineering
The University of Texas at Austin - Austin, TX, USA
2007 - 2013
Bachelor of Science Degree in Computer Science
Brigham Young University - Provo, UT, USA
Certifications
FEBRUARY 2024 - FEBRUARY 2026
CKS: Certified Kubernetes Security Specialist
The Linux Foundation
NOVEMBER 2023 - NOVEMBER 2026
CKA: Certified Kubernetes Administrator
The Linux Foundation
OCTOBER 2023 - OCTOBER 2026
CKAD: Certified Kubernetes Application Developer
The Linux Foundation
DECEMBER 2019 - PRESENT
Parallel Programming in Scala
Coursera
DECEMBER 2019 - PRESENT
Functional Programming in Scala Capstone
Coursera
DECEMBER 2019 - PRESENT
Functional Programming Principles in Scala
Coursera
DECEMBER 2019 - PRESENT
Functional Program Design in Scala
Coursera
DECEMBER 2019 - PRESENT
Big Data Analysis with Scala and Spark
Coursera
APRIL 2019 - PRESENT
Advanced Python
Collaboration That Works
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
1
Share your needs
Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2
Choose your talent
Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3
Start your risk-free talent trial
Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.
Top talent is in high demand.
Start hiring