Data/ML Engineer2019 - PRESENTDatabricks
Technologies: Azure, Amazon Web Services (AWS), Scala, Python, AWS, SQL, Spark
- Developed an application to store and track the changes in the hyperparameters used in training models as well as the data the model was trained on. This application saves model metadata and provides access to them using API calls.
- Built an optical character recognition pipeline that converted images to a table.
- Increased querying performance of a 75TB data lake table. The reports that pulled from this table had an SLA of 30 seconds. By applying Spark performance tuning techniques, I was able to make the query time to less than five seconds.
Senior Data Engineer2017 - 2018Copart
Technologies: Azure, Apache Kafka, Pentaho, Python, SQL
- Developed a real-time data pipeline to move application logs to a more consumable form for reporting.
- Built a global data warehouse to serve as a single source of truth for company-wide open operational metrics.
- Migrated the company's ETL architecture to the cloud.
Software Developer2015 - 2018Brocks Solution
Technologies: Azure, DataWare, SQL, Python
- Developed a real-time data pipeline to stream data from IoT devices (bag tag scanners) at airports to create baggage handling reports for business executives.
- Led the implementation of analytics into the company's enterprise baggage handling system. software.
- Created dashboards to report data on baggage handling operations.