
Brian de la Motte
Verified Expert in Engineering
Data Engineer and Developer
Meridian, ID, United States
Toptal member since July 13, 2021
Brian is a staff engineer and two-time CTO with 19 years of experience writing code, leading teams, and delivering systems that others could not. He is the founding team engineer behind an AI meetings platform's audio capture and transcription stack, shipping Rust, Go, Swift, and Kotlin across four operating systems. Brian previously founded a consultancy that served 70 companies, and re-platformed a federal Hadoop environment that became the first in its agency to pass Authority to Operate.
Portfolio
Experience
- Bash - 14 years
- SQL - 14 years
- Python - 8 years
- Data Engineering - 7 years
- Spark - 6 years
- Ansible - 5 years
- Apache Airflow - 2 years
- Data Build Tool (dbt) - 2 years
Work Experience
CEO | CTO | Software and Cloud Architect
Hidden Mountain Data
- Built the firm to $2.0 million in invoiced revenue across 54 clients, peaking at $944,000 in 2024 and growing roughly 8x in one year, serving 70 companies across 51 industries.
- Acted as Fractional CTO for several clients, including Onsite Analytics, GymGuard, Always Connect Solutions, and Logistics Ocean.
- Cultivated and led multidisciplinary teams to execute end-to-end digital transformation initiatives in logistics, HR, and property maintenance industries.
- Integrated real-time data pipelines and AI/ML capabilities, including predictive analytics and computer vision, across projects to deliver actionable insights and operational efficiencies.
- Ran an AI lead-qualification system over the AWS IQ marketplace that screened 16,128 inbound requests in two years and converted them into 31% of the client base.
Data Engineer
Syngenta - IT, Seeds Research & Development
- Migrated a compute-intensive data pipeline from on-prem to the AWS cloud.
- Containerized software applications so that they'd run anywhere for maximum portability.
- Troubleshoot bottlenecks in high-performance computing workflows.
Data Transformation Engineer
PepsiCo Global - Main
- Taught and mentored several PepsiCo contractors and employees across multiple teams on how to use dbt more efficiently.
- Introduced and implemented CI/CD and Slim CI processes to the entire eCommerce division of Pepsi, which increased productivity by 800% and at the same time drastically reduced resources used in the data warehouse.
- Introduced SQLFluff, a SQL linter, to all of Pepsi's eCommerce divisions. Linted around 700 different SQL dbt data models with over 20 linting rules. Got an entire eCommerce division using the same SQL dialect and SQL style.
- Implemented incremental models or data loads in place of full data refreshes, reducing the time it took for bigger data pipelines to run and increasing developer productivity.
- Introduced dbt best practices for data engineers, analytics engineers, and data analysts.
Senior Data Engineer
Netlify
- Maintained business-critical data pipelines which handled 2TB of data a day with over 60 different data pipelines.
- Migrated the company's data process from ETL to ELT using DBT and Spark on Databricks. This simplified the number of programming languages used from four to two and allowed for easier debugging and idempotent data pipelines.
- Introduced and productionized the use of Apache Airflow for data workflow orchestration and to kick off extractions and loads that previously were extremely tedious and difficult to monitor.
- Switched from error-prone data extractions for some data sources over to Fivetran. After Fivetran was set up, introduced several new datasets, and integrated them into our data warehouse for analytical uses.
- Authored and maintained big data compute jobs by leveraging PySpark and SparkSQL on Databricks and building out a data lakehouse in the process.
- Secured data warehouse clusters by automating the creation of credentials through Ansible and Terraform.
- Built a cost-of-goods data model that leveraged source billing data from major cloud providers into one conformed model in order to answer questions about the company's cloud spending.
Data Engineer Curriculum Writer (Subject Matter Expert)
Thinkful
- Designed a 6-month curriculum plan for teaching data engineers.
- Wrote core modules and led meetings to brainstorm the best topics to cover.
- Created hands-on labs for students to go through to gain practical experience in the role of a data engineer.
Senior Data Engineer Contractor
Offer Up
- Migrated dozens of data pipelines from Apache Airflow to Google Cloud Composer in a very short time span.
- Assisted in a mass data warehouse migration from Snowflake to BigQuery for a total of 1PB (petabyte) of data being migrated between AWS and Google Cloud.
- Reconciled data records in old and new data warehouses to ensure that the data pipelines worked as expected.
- Rewrote analytical SQL queries that were Snowflake-specific to use BigQuery's syntax which involved lateral flattens and other types of complicated joins on nested data and semi-structured data.
Data Engineer
Brushfire
- Built out a new data warehouse in AWS for analytical and data mining workloads (the first data warehouse of the company).
- Implemented business intelligence and data visualizations for the company and client KPIs.
- Built repeatable, scalable ETL data pipelines in the cloud.
- Identified data performance issues in queries, indexes, data modeling, and so on.
- Migrated from .NET to .NET Core, then transitioned the entire web stack from Azure VMs to Kubernetes (complete with cluster and pod autoscaling and CI/CD with Azure DevOps).
Principal Big Data Consultant
zData, Inc
- Architected big data and cloud solutions in a cloud-based enterprise Linux environment utilizing compute resources such as EC2, S3, AWS Auto Scaling, Lamba functions, DynamoDB, Couchbase, SQS, and Elastic MapReduce (EMR).
- Automated repeatable deployments of big data software using Ansible, AWS CloudFormation, and Terraform.
- Secured distributed clusters via security groups, firewalls, authorization and authentication policies. Kerberized Hadoop and other distributed clusters for strong authentication.
- Led various software development projects in back-end web APIs and cloud-based web applications in Java, Python, and Elixir.
- Architected distributed, fault-tolerant, and highly available systems using on-premise or cloud-based hardware.
- Built a custom Apache Ambari stack containing installation and management capabilities for Pivotal Greenplum, Pivotal HAWQ, and Chorus in Python.
- Developed a parallel backup and restore solution for large compute clusters in the AWS cloud in Java.
DevOps Engineer
Melaleuca
- Built an internal tool to dynamically discover WCF endpoints and make async requests to see if they are online or not. The tool was used to warm up the services to trigger JIT compilation and to actively monitor which endpoints were down.
- Wrote a custom DevOps dashboard that the entire team used to monitor how deployments were going and the health of the cloud infrastructure.
- Automated several complicated workflows in SharePoint to reduce the number of repetitive tasks for the team.
Firmware Engineer | Web Developer | Network Operations Manager | Senior Support Technician
Linora
- Wrote the back-end and front-end code for router control panels and for "MeshView" which was a cloud-based (before the cloud) management portal for WiFi networks.
- Managed a small team of network operators and support technicians to take inbound level I and level II calls; was responsible for hiring, firing, teaching, and scheduling the team.
- Wrote firmware for WiFiRanger and BlueMesh Networks product firmware; worked with kernel modules, internal and external radios, USB modem connectivity, controlling remote radios over ethernet, failover, and failback logic.
- Assisted in the management, monitoring, and software for monitoring over 10,000 IoT and routers in the field.
Project History
Cost-of-goods Data Model
http://netlify.comI designed it from end to end and built all the data pipelines. This project was used internally to forecast the company's future cloud costs and to drill down into the specifics to see how money was being spent.
Big Data Platform
Out of this came an idea of dynamic templating so several clusters could be spun up in parallel and used for development or production.
The government had the most stringent requirements involving things being approved for FEDRAMPED and before using the cluster for official government use, we had to get the official "Authority to Operate" approval. This involved turning every security bell and whistle you could think of, including vormetric disk encryption, SSL on everything, security groups, firewalls, authorization, SSSD, Kerberos, RedHat IdM integration, cross-forest Active Directory trusts.
After getting the ATO, we helped several teams within the SSA get onboarded to the BDP to use Hive and Apache Spark for various ML and analytical workloads. Our work was so successful, the NIH wanted to use our cluster for some work too.
The ML work involved parameter tuning and installing TensorFlow on Spark, XGBoost, and other popular libraries.
Hadoop Data Lake
Massive Data Warehouse Migration
http://offerup.comVirtual Machine to Kubernetes Migration
http://brushfire.comFinally, after getting the cluster pod autoscaling and horizontal pod autoscaling working correctly, I set up several CI /CD pipelines for production, staging, and development branches to automatically build and deploy the updated images to the production, staging, and development websites.
Out of all this, the company saved about $2,000 per month on their hosting costs as the autoscaling was much more resilient, and the new cluster was cheaper than paying for the raw VMs.
Empire Unmanned
Modern API Rewrite
https://www.recallinfolink.com/Cross-platform Audio Capture and Transcription Platform
• Engineered the pipeline in Rust: multi-device capture, WebRTC AEC3 echo cancellation, resampling, and streaming ASR with fault classification, plus audio-in and words-out reconciliation that guarantees data integrity from microphone to database so a dropped word is detectable rather than invisible.
• Recovered 20% of captured audio that was being silently discarded by fixing resampling and downmix defects in the 44.1kHz path.
• Shipped on-device transcription on three platforms - iOS SpeechAnalyzer, Android Kroko/sherpa ASR, and a native macOS capture service, with a canonical-transcript policy that resolves conflicts when sources disagree.
• Designed the underlying PostgreSQL data model, ERDs, and architecture decision records, and the Goose migration pipeline that still runs every schema change.
• Took the product to market on a Tauri shell, then replaced it with native Swift, Kotlin, and Windows implementations, and launched both mobile apps through App Store and Google Play review.
PropTech Platform Rescue and Back end Re-platform
• Took over a stalled codebase and restarted it: the project had produced 22 commits in the four months before I arrived and shipped 902 in my first six months.
• Directed a team of six engineers delivering roughly 1,440 commits across three repositories in nine months. Consolidated three legacy applications into a single Next.js and TypeScript codebase, retiring a 228-commit React front end and two abandoned mobile apps.
• Put the production database schema under version control for the first time in the company's history with Prisma and a migration pipeline that ran on deploy.
• Directed a complete back end re-platform from AWS Lambda and MySQL onto NestJS, PostgreSQL, and ECS Fargate - 433 commits in 18 weeks, 23 controllers, 34 data models, 41 test suites, and a hand-written cutover migrating live production data.
• Shipped an AI document-understanding pipeline turning a phone photo of an appliance serial plate into structured manufacturer, model, and warranty records using AWS Textract and OpenAI.
QR-based Equipment Maintenance Platform
• Structured the engagement myself, trading a 25% discount on the build fee for a 10% equity stake, and delivered the MVP on a contractual hard deadline with the team demobilized on schedule.
• Directed a team of seven engineers and a project manager that shipped roughly 920 commits across three codebases.
• Designed the core product mechanic: a member scans a QR sticker on a machine with their phone camera - no app and no login, and the report becomes a tracked repair ticket tied to that machine's make, model, and warranty history.
• Oversaw a NestJS and PostgreSQL back end with true multi-tenant authorization, a Vue 3 operator console, preventive-maintenance scheduling, and SMS and email notification pipelines.
I now serve as a technical advisor and built the AI product surface solo: conversational equipment-fault diagnosis, where an operator describes a fault in plain language and receives a structured diagnosis, using OpenAI and the Vercel AI SDK on Next.js and React 19.
Education
Bachelor's Degree in Computer Science
Boise State University - Boise, ID, United States
Certifications
AWS Certified Solutions Architect - Associate
Amazon Web Services (AWS)
Skills
Libraries/APIs
PySpark, REST APIs, Pandas, Stripe, jQuery, React, Vue
Tools
Apache Airflow, Ansible, Microsoft Access, Terraform, BigQuery, Azure Kubernetes Service (AKS), Tableau, Azure DevOps Services, Apache ZooKeeper, Apache Ambari, AWS CloudFormation, Cloudera, Amazon Simple Queue Service (SQS), Amazon Elastic Container Service (ECS), Amazon Elastic Block Store (EBS), Amazon Redshift Spectrum, AWS Glue, Amazon Athena, dbt Cloud
Languages
Python, SQL, Bash, Transact-SQL (T-SQL), Snowflake, Java, C, Scala, C#, Elixir, HTML, PHP, CSS, JavaScript, R, TypeScript, Rust, Go, Swift, Kotlin
Frameworks
Spark, Apache Spark, .NET, .NET Core, Hadoop, Angular, Windows PowerShell, Phoenix, NestJS, Next.js
Paradigms
ETL, Agile, Scrum, Business Intelligence (BI), Azure DevOps, Test-driven Development (TDD), Dimensional Modeling, High-performance Computing (HPC)
Platforms
Databricks, Amazon Web Services (AWS), Google Cloud Platform (GCP), DigitalOcean, Azure, Kubernetes, Linux, Apache Kafka, Amazon EC2, Hortonworks Data Platform (HDP), SharePoint, Salesforce, Oracle Database
Storage
Data Pipelines, Data Lakes, PostgreSQL, Database Architecture, Databases, Microsoft SQL Server, Redshift, Google Cloud Storage, MySQL, MongoDB, SQL Server DBA, AWS Data Pipeline Service, Amazon S3 (AWS S3), Apache Hive, HBase, Data Lake Design, Greenplum, Couchbase, Amazon DynamoDB, Google Cloud, SQL Server Integration Services (SSIS), Elasticsearch, Data Integration
Other
Big Data, ELT, CI/CD Pipelines, Data Architecture, Data Engineering, Data Analytics, Data Analysis, Data Warehouse Design, APIs, API Integration, Google BigQuery, Integration, Migration, Solution Architecture, Large-scale Data Migration, Agentic Deployment, Artificial Intelligence (AI), Cloud, Data Build Tool (dbt), Data Modeling, Data Migration, Dashboard Design, Dashboards, Software Development Lifecycle (SDLC), Programming, Quantum Computing, Ethics, Algorithms, Fivetran, Big Data Architecture, EMR, Autoscaling, Kerberos, SSH, USB, Firmware, Linux Kernel Modules, WiFi, Windows Communication Foundation (WCF), Active Directory Federation, Vormetric, Web Security, Cloud Architecture, Mainframe, Web Scraping, Amazon Kinesis, Data Transformation, Data Warehousing, Fiserv
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring