Alexandre is currently unavailable
Alexandre França de Magalhães
Verified Expert in Engineering
Data Architect and Developer
Salvador - State of Bahia, Brazil
Toptal member since February 4, 2022
Bio
Alexandre is a data solutions architect and engineer with over eight years of professional experience. His primary experiences include designing and building data lakes and warehouses, as well as processing data using available resources such as Spark, SQL from databases, and Pandas. Alexandre is familiar with Azure and the AWS stack, but is open to working with other cloud platforms as well.
Portfolio
M37 Ltd
Data Engineering, ETL, Python, Big Data, Retrieval-augmented Generation (RAG)...
Malachyte, Inc.
Python, Data Engineering, Apache Airflow, Data Pipelines...
PepsiCo
Data Engineering, Python, Advertising, Media, Azure, Over-the-top Content (OTT)...
Experience
- SQL - 8 years
- Data Warehousing - 8 years
- Data Engineering - 8 years
- Data Modeling - 8 years
- Spark - 8 years
- Python - 8 years
- ETL - 8 years
- Data Management - 5 years
Preferred Environment
Spark, SQL, Azure, Databricks, Python, Amazon Web Services (AWS), Apache Airflow, Azure Data Factory (ADF)
The most amazing...
...project I've developed was a data lake architecture from scratch with cloud, on-premise, and API data sources.
Work Experience
Lead Data Engineer
2025 - PRESENT
M37 Ltd
- Developed large-scale unstructured data extraction pipelines using Python (text extraction libraries + AI models) to convert SEC filings into structured Markdown and JSON outputs.
- Processed and analyzed 10,000+ financial documents from publicly traded U.S. companies, enabling automated insights and improving data accessibility for downstream analytics.
- Implemented and managed an Apache Airflow orchestration service from scratch, migrating all existing data pipelines and onboarding the engineering team to the new workflow platform.
Technologies: Data Engineering, ETL, Python, Big Data, Retrieval-augmented Generation (RAG), Apache Airflow, MongoDB, OpenAI
Lead Data Engineer
2025 - 2025
Malachyte, Inc.
- Designed and implemented the company’s first data lake on GCP, establishing the foundational data engineering infrastructure and scalable ingestion pipelines.
- Developed robust batch and streaming data pipelines leveraging BigQuery, BigTable, Kafka, and Python to enable real-time analytics and reliable data processing.
- Engineered Python-based microservices for high-performance data processing, system integration, and specialized business use cases within the data platform.
Technologies: Python, Data Engineering, Apache Airflow, Data Pipelines, Google Cloud Platform (GCP), Artificial Intelligence (AI), Apache Kafka, ETL, Machine Learning, Terraform, GitHub Actions, Docker
Data Architect | Lead Engineer
2022 - 2024
PepsiCo
- Developed PepsiCo's global media data hub from scratch to centralize customer and performance data across various media campaigns in a corporate and centralized environment. The sources were mixed between APIs and cloud storage.
- Handled PySpark code optimization to enhance performance and standardization.
- Built simple machine learning models on Databricks with MLflow to track performance, metrics, and artifacts.
Technologies: Data Engineering, Python, Advertising, Media, Azure, Over-the-top Content (OTT), Databricks, Samba, Roku, YouTube, Dynamic Data, Snowflake, APIs, Azure Data Factory (ADF), Data Warehousing, Data Lakes, Azure Data Lake, Amazon S3 (AWS S3), Google Cloud Storage, Azure Blobs, PySpark, Spark, MLflow, Machine Learning, Google Cloud Platform (GCP), Big Data, Google BigQuery, Data Build Tool (dbt), Docker, Azure DevOps
Lead Data Engineer
2022 - 2022
BCG - Gamma
- Developed data pipelines with data scientists to productionize experiments, data extractions, data modeling, data cleaning, and quality checking on multiple cloud environments.
- Managed large datasets using Spark as a processing tool.
- Developed SQL queries to query, analyze, and manipulate data on many platforms, such as Spark, Hive, and relational data sources.
Technologies: SQL, PySpark, Automation, Azure, Amazon Web Services (AWS), Microsoft Azure, Apache Airflow, Pandas, Docker, Amazon S3 (AWS S3), Spark, Python, Parquet, CSV, JSON, Delta Lake, CI/CD Pipelines, Azure Blobs, Data Extraction, Data Cleaning, Hue, Apache Hive, Amazon Elastic MapReduce (EMR), Amazon Redshift, AWS Glue
Senior Data Engineer
2021 - 2022
Via Varejo
- Refactored the fraud analysis pipeline for performance improvements to be ready for Black Friday in 2021, achieving constant execution times on increased batch data loads.
- Worked on developments at the company's fraud data marts.
- Developed various pipelines to solve ingestions and data processing necessities.
Technologies: Azure, Databricks, Azure Data Factory (ADF), Azure Data Lake, Azure Synapse, Azure Event Hubs, Data Engineering, Data Warehousing, Data Modeling, ETL, ETL Tools, SQL, Data Management
Senior Data Engineer
2021 - 2021
Radix
- Developed generic ingestion data pipelines for relational data sources, accelerating the process of new ingestions using simple configuration files.
- Developed a Delta Lake architecture from scratch for secure and efficient data processing.
- Worked on developing and maintaining a corporate data warehouse on the Azure Synapse platform.
Technologies: Azure, Azure Data Factory (ADF), Databricks, Azure Data Lake, Azure Synapse, Google Cloud Storage, Data Engineering, Data Modeling, Data Warehousing, ETL, ETL Tools, SQL, PySpark, Synapse, Apache Kafka, Microsoft Azure, Amazon Web Services (AWS)
Senior Data Engineer
2020 - 2021
Bridgestone
- Supported and enhanced corporate data lakes built on Azure Cloud Services with on-premise data sources, such as SQL Server, Oracle, and Kafka streams for sensor data.
- Developed SSIS packages for data pipelines with SQL, PL/SQL, and T-SQL.
- Managed the third-party team in charge of on-site software and data support demands.
Technologies: Azure, Azure Data Factory (ADF), Databricks, Oracle, Pandas, SQL Server 2016, Data Engineering, Data Warehousing, ETL, ETL Tools, Data Modeling, SQL, PySpark, Oracle PL/SQL
Software Developer
2019 - 2020
Chemtech
- Developed data extractions and features to help data science teams train and validate machine learning models built on top of Python, Pandas, and Scikit-learn technologies for various projects in the company.
- Developed data pipelines for multiple client companies to serve data lake and warehouse architectures.
- Tracked and developed user histories using Jira as a reporting tool.
Technologies: Oracle, SQL Server 2016, Python, Pandas, ETL Tools, ETL, Data Engineering, Data Warehousing, Data Modeling, Oracle PL/SQL, Transact-SQL (T-SQL), Data Pipelines, Microsoft Azure, Amazon Web Services (AWS), HDFS, Spark
Software Developer
2017 - 2019
Braskem
- Developed SQL scripts for data ETL in a corporate data warehouse.
- Created complex queries for production reports serving business analyst needs.
- Developed C# back-end applications for the manufacturing of execution systems.
Technologies: Python, SQL, Oracle, SQL Server 2016, MongoDB, Data Warehousing, ETL Tools, ETL, Data Engineering, Data Modeling, Oracle PL/SQL, Amazon Web Services (AWS), Microsoft Azure, Data Pipelines, Pandas, C#
Experience
Media Data Hub for Global Food and Beverage Company
A cloud-based solution that acquires data from media campaign performance globally and distributes it through a centralized data hub that can be accessed by a myriad of tools. It exists as a cornerstone for initiatives that need campaign performance measurements, such as ROI management and AI/machine learning predictions.
Data Lakehouse for an Educational Company
I developed an Azure Data Lake and Data Warehouse with dynamically configurable ingestion pipelines for various data sources such as on-premise Oracle and SQL Server databases, Google Cloud Platform (GCP) storage, and external API providers. The cloud infrastructure was previously nonexistent, so I established the hierarchical storage patterns and partitioning for each data source, processing most of the data on the Databricks platform with a data lakehouse configuration.
Data Lake for Rubber and Tire Industry
Evolved and migrated corporate data lake from on premisses Hadoop to Azure ADLS2. Developed pipelines for data ingestion and started the data modeling for a data warehousing solution where the processing was performed with Spark Databricks. The data warehouse was hosted on a synapse-dedicated SQL pool.
Refactoring of Fraud Detection Pipeline for Retail Company
I refactored an existing fraud detection pipeline used to analyze credit card purchases for abnormal behavior with an ML model. The main source of the problem lies in many calculations that could and should be executed in parallel. With this change, it was also necessary to rethink the calculated values. This reduced execution time from two hours to 20 minutes, consistent even on last year's Black Friday.
Education
2013 - 2018
Bachelor's Degree in Engineering
Federal University of Bahia - Salvador, Brazil
Certifications
DECEMBER 2025 - PRESENT
Astronomer Certification DAG Authoring for Apache Airflow 3
Astronomer
SEPTEMBER 2022 - PRESENT
Certified Data Engineer Associate
Databricks
Skills
Libraries/APIs
PySpark, Pandas, REST APIs
Tools
Spark SQL, Apache Airflow, Synapse, Hue, Amazon Elastic MapReduce (EMR), AWS Glue, Terraform
Languages
SQL, Python, Transact-SQL (T-SQL), Batch, C#, Snowflake
Frameworks
Spark, Hadoop, Data Lakehouse
Paradigms
ETL, Automation, Samba, Azure DevOps
Platforms
Azure, Databricks, Oracle, Azure Synapse, Azure Event Hubs, Apache Kafka, Amazon Web Services (AWS), Docker, YouTube, Google Cloud Platform (GCP)
Storage
Data Pipelines, SQL Server 2016, Oracle PL/SQL, MongoDB, Data Lake Design, Data Lakes, HDFS, Amazon S3 (AWS S3), JSON, Azure Blobs, PostgreSQL, Apache Hive, Google Cloud Storage
Other
Azure Data Factory (ADF), Azure Data Lake, Data Engineering, Data Modeling, ETL Tools, Data Warehousing, Data Management, Data Cleaning, Microsoft Azure, Data Build Tool (dbt), Streaming, Parquet, CSV, Delta Lake, CI/CD Pipelines, Data Extraction, Advertising, Media, Over-the-top Content (OTT), Roku, Dynamic Data, APIs, MLflow, Machine Learning, Data Curation, Azure Databricks, Data Governance, Cloud Storage, Big Data, Google BigQuery, Amazon Redshift, Artificial Intelligence (AI), GitHub Actions, Retrieval-augmented Generation (RAG), OpenAI
Collaboration That Works
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
1
Share your needs
Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2
Choose your talent
Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3
Start your risk-free talent trial
Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.
Top talent is in high demand.
Start hiring