Douglas is currently unavailable

Douglas Oliveira

Verified Expert in Engineering

Machine Learning Developer

Fortaleza - State of Ceará, Brazil

Toptal member since May 8, 2026

Expertise

Data Engineering AWS Snowflake Databricks Machine Learning Artificial Intelligence Deep Learning Prompt Engineering Python AWS IAM Apache Airflow Cloud Engineering Terraform

Bio

Douglas is a data consultant holding a PhD in AI and 19 advanced technical certifications, with deep expertise spanning data engineering, large language models, applied machine learning, and project management. He has extensive experience leading analytical and AI-driven projects, from designing and building ETL and data transformation workflows to deploying generative AI and LLM solutions into production.

Portfolio

Apple

Snowflake, Python, Prompt Engineering, RAG Systems, Databricks

Motive

Snowflake, Data Build Tool (dbt), AWS IoT, Terraform, RAG Systems

Loadsmart

AWS IoT, Snowflake, Data Build Tool (dbt), Apache Airflow, Terraform

Experience

Machine Learning - 9 years
Data Engineering - 9 years
Amazon Web Services (AWS) - 8 years
Artificial Intelligence (AI) - 7 years
AWS IoT - 7 years
Databricks - 6 years
Snowflake - 6 years
RAG Systems - 4 years

Preferred Environment

Snowflake, AWS IAM, Databricks

The most amazing...

...solution I've developed is a production LLM data pipeline that automated analytics workflows and improved decision-making at scale.

Work Experience

AI Data Engineer

2024 - PRESENT

Apple

Defined best practices for prompt versioning, model monitoring, and evaluation, establishing standards adopted across multiple AI initiatives.
Implemented data transformations and validation, ensuring data quality using Snowflake and dbt.
Optimized RAG retrieval by applying embedding best practices and infrastructure tuning across the platform.

Technologies: Snowflake, Python, Prompt Engineering, RAG Systems, Databricks

AI Data Engineer

2023 - 2023

Motive

Built an end-to-end data infrastructure platform from scratch using Snowflake, dbt, AWS, and Terraform.
Developed and deployed retrieval-augmented generation (RAG) pipelines integrating LLMs with internal knowledge bases, enabling semantic search and AI-driven document Q&A.
Reduced token consumption by 50% in an LLM-based automated essay grading system by applying prompt engineering and prompt compaction techniques, preserving accuracy.

Technologies: Snowflake, Data Build Tool (dbt), AWS IoT, Terraform, RAG Systems

AI Data Manager

2022 - 2023

Loadsmart

Integrated LLM-powered capabilities into pricing workflows, leveraging RAG pipelines for competitive intelligence extraction and automated insights generation from unstructured market data.
Built scalable AWS pipelines leveraging Amazon Simple Storage Service (S3), AWS Lambda, and Amazon Managed Workflows for Apache Airflow (MWAA) for pricing data ingestion and processing across the platform.
Spearheaded six developers delivering data products for pricing optimization and managed infrastructure with Terraform.

Technologies: AWS IoT, Snowflake, Data Build Tool (dbt), Apache Airflow, Terraform

Machine Learning Lead

2021 - 2022

Feedzai

Delivered data and machine learning solutions for credit card fraud detection, enabling clients to maximize platform value.
Owned end-to-end analytics projects, including data extraction, exploration, modeling, and evaluation, using AWS, Google Cloud Platform, Azure, and Snowflake.
Architected and supported data pipelines and feature datasets for the machine learning lifecycle from training through deployment.

Technologies: Google Cloud Platform (GCP), Python, Snowflake, AWS IoT, Machine Learning, Microsoft Azure

Data Coordinator

2020 - 2021

Cielo Brazil

Directed MLOps modernization using AWS, Google Cloud Platform, and Airflow, improving pipeline reliability and deployment workflows.
Built Snowflake-based data models and pipelines to support analytics prioritization and impact measurement.
Managed a team of six developing ML solutions with production-grade data pipelines for logistics, credit risk, and CRM.

Technologies: AWS IoT, Google Cloud Platform (GCP), Apache Airflow, Snowflake

Data Science Lead

2018 - 2019

Apple

Defined the scope of the data science team, establishing processes, KPIs, and necessary resources to deliver impact.
Directed a team of four data scientists allocated to HWETA projects, responsible for developing data science solutions.
Served as the technical point of contact with on-site management to keep track of the progress of team projects.

Technologies: Python, Data Science, Machine Learning, Deep Learning

Experience

End-to-end Data Infrastructure with RAG-powered LLM Pipelines

As an AI data engineer at Motive/Orlo, I architected and built an end-to-end data infrastructure platform from scratch using Snowflake, dbt, AWS, and Terraform. The platform supported scalable analytics, machine learning workloads, and AI-driven products across the organization.

A core part of the project was developing and deploying RAG pipelines that integrated large language models with internal knowledge bases. This enabled semantic search capabilities and AI-driven document Q&A, giving teams fast and accurate access to institutional knowledge.

I also designed and shipped an LLM-based automated essay grading system. By applying prompt engineering and prompt compaction techniques, I reduced token consumption by 50% while preserving grading accuracy, significantly lowering operating costs.

My responsibilities spanned data modeling, pipeline orchestration, infrastructure-as-code with Terraform, embedding strategy, vector search tuning, evaluation frameworks, and production monitoring. The result was a reliable, cost-efficient data and AI platform powering internal tooling and customer-facing AI features.

Education

2012 - 2016

PhD in Artificial Intelligence

Florida Institute of Technology - Melbourne, FL, USA

Snowflake

APRIL 2024 - PRESENT

AWS Certified Cloud Practitioner

Amazon Web Services (AWS)

Skills

Tools

AWS IAM, Apache Airflow, Terraform

Languages

Snowflake, Python

Platforms

Databricks, AWS IoT, Amazon Web Services (AWS), Google Cloud Platform (GCP)

Other

Data Engineering, Data Science, RAG Systems, Artificial Intelligence (AI), Machine Learning, Microsoft Azure, Deep Learning, Prompt Engineering, Data Build Tool (dbt), Cloud Computing, Data Architecture

Collaboration That Works

How to Work with Toptal

Toptal matches you directly with global industry experts from our network in hours—not weeks or months.

Share your needs

Discuss your requirements and refine your scope in a call with a Toptal domain expert.

Choose your talent

Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.

Start your risk-free talent trial

Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.

Top talent is in high demand.

Start hiring