Daniel Brum, Developer in Porto Alegre - State of Rio Grande do Sul, Brazil
Daniel is available for hire
Hire Daniel

Daniel Brum

Verified Expert  in Engineering

Data Engineer and Developer

Porto Alegre - State of Rio Grande do Sul, Brazil

Toptal member since October 24, 2023

Bio

Daniel is a data engineer and architect specializing in Azure's cloud services. He's keen on collaborating with data analysts and data scientists to provide them with accurate data for dashboards and experiments, leveraging his clear communication skills and problem-solving mindset. As a certified Azure data engineer and a Databricks data engineer professional, Daniel's tech stack includes Databricks, Azure Data Factory, Azure Synapse, Azure Analysis Services, and Microsoft Power BI.

Portfolio

Toptal Clients
Data Engineering, Data Warehousing, ETL, Data Lakes, Microsoft SQL Server, SQL...
Meta IT North America
Azure Synapse, Azure Analysis Services, Microsoft Power BI, Azure DevOps...
Gavb
Databricks, Azure Databricks, Spark, Python, PySpark, Azure Data Factory (ADF)...

Experience

  • SQL - 9 years
  • Python - 7 years
  • Azure - 5 years
  • Data Engineering - 5 years
  • Microsoft Power BI - 5 years
  • Data Modeling - 5 years
  • Azure Synapse - 5 years
  • Databricks - 4 years

Availability

Part-time

Preferred Environment

Azure, Databricks, Azure Synapse, Microsoft Power BI, Azure Data Factory (ADF), Spark, Python, Apache Airflow, SQL, Agile

The most amazing...

...data platform I've designed and implemented supports more than 700 data engineers, analysts, and scientists daily.

Work Experience

Data Architect | Data Engineer

2023 - PRESENT
Toptal Clients
  • Designed and fully implemented data platforms in the Azure cloud.
  • Deployed and managed Azure subscriptions, resource groups, as well as resources for data lakehouse and data warehouse implementations with a focus on scalability, security, maintainability, and cost.
  • Leveraged tools and technologies such as Databricks (Apache Spark), Microsoft Fabric, Data Factory, Azure Synapse, Power BI, SQL, Python, DAX, and PowerShell to develop metadata-driven data pipelines and enable several data analytics use cases.
  • Developed Python code to extract source system data incrementally through REST and SOAP APIs.
  • Created Spark SQL code to transform and integrate the data in an enterprise-level dimensional model to drive Power BI reports.
  • Ensured code quality using static analysis tools such as Ruff and SQLfluff.
  • Automated infrastructure and code deployments using Azure DevOps pipelines, Terraform, Bash, and PowerShell.
Technologies: Data Engineering, Data Warehousing, ETL, Data Lakes, Microsoft SQL Server, SQL, Azure, Microsoft Power BI, Agile, APIs, Databricks, Azure Databricks, Spark, Python, Data Modeling, Computer Science, Terraform, Azure Blob Storage API, Azure PaaS, Azure Data Lake, Azure Virtual Networks, Scrum, Scrum Master, Product Owner, Product Ownership, Scrum Product Owner, Certified Scrum Product Owner (CSPO), Certified ScrumMaster (CSM), Azure DevOps, PySpark, Git, Data Lakehouse, DAX, Regular Expressions, SSRS Reports, SQL Server Reporting Services (SSRS), Power BI Report Builder, Microsoft Power Automate, Microsoft Excel, Data Pipelines, Cloud, Agile Project Management, IT Project Management, DevOps, Data Architecture, Delta Lake, Tabular Editor, DAX Studio, Data Warehouse Design

Azure Data Engineer

2022 - 2023
Meta IT North America
  • Designed and implemented DW and BI projects using Microsoft Azure, Microsoft SQL Server, Azure Synapse, Azure Data Factory, Azure Analysis Services, and Microsoft Power BI.
  • Developed and optimized generic data pipelines to efficiently ingest a large volume of raw data and reliably transform it into dimensional models.
  • Set up build and release pipelines on Azure DevOps for continuous integration and deployment.
  • Optimized and developed DAX measures using Tabular Editor and DAX Studio to drive reports and dashboards in Microsoft Power BI.
Technologies: Azure Synapse, Azure Analysis Services, Microsoft Power BI, Azure DevOps, Azure Data Factory (ADF), Azure Data Lake, Azure, Azure SQL, Azure SQL Databases, Azure SQL Data Warehouse, Dedicated SQL Pool (formerly SQL DW), Git, T-SQL (Transact-SQL), Oracle EBS, Oracle E-Business Suite (EBS) R12, Microsoft Dynamics 365, Microsoft Dynamics 365 for Finance and Operations, Salesforce, DAX, Regular Expressions, DevOps, Agile Project Management, IT Project Management, Data Engineering, Data Architecture, Data Modeling, Microsoft SQL Server, Tabular Editor, DAX Studio, ETL, Data Warehousing, Data Warehouse Design, Cloud, Data Pipelines, Computer Science, Azure PaaS, Azure Resource Manager (ARM), Product Owner, Product Ownership, Scrum Product Owner, Certified Scrum Product Owner (CSPO), Certified ScrumMaster (CSM), Microsoft Power Automate, SQL

Data Engineering Specialist

2021 - 2022
Gavb
  • Consulted with external clients on the design of cloud-native big data platforms using modern data warehouses, lakehouses, and cloud-scale analytics.
  • Implemented big data solutions using technologies such as Databricks, Delta Lake, Apache Airflow, Azure Data Factory, Azure Synapse, and Microsoft Power BI.
  • Shared best practices for information security, data governance, and cost and performance optimization.
  • Developed data pipelines and customized frameworks for automatic batch generation and streaming pipelines through metadata.
Technologies: Databricks, Azure Databricks, Spark, Python, PySpark, Azure Data Factory (ADF), Azure DevOps, Azure SQL, Azure SQL Databases, Azure SQL Data Warehouse, Dedicated SQL Pool (formerly SQL DW), Git, GitHub, Apache Airflow, Data Lakes, Azure Data Lake, Data Lakehouse, Microsoft Power BI, Google Cloud Composer, SAP, SAP HANA, SAP Business Warehouse (BW), Regular Expressions, DevOps, Agile Project Management, IT Project Management, Data Engineering, Data Architecture, Data Modeling, Attunity Replicate, Delta Lake, ETL, Data Warehousing, Data Warehouse Design, Cloud, Data Pipelines, Computer Science, Azure PaaS, Azure Resource Manager (ARM), Azure Virtual Machines, Azure Virtual Networks, Product Owner, Product Ownership, Scrum Product Owner, Certified Scrum Product Owner (CSPO), Microsoft Excel, SQL, Azure Blob Storage API, Agile, Scrum, Scrum Master, Certified ScrumMaster (CSM)

Data Engineer

2020 - 2021
DLL Group
  • Worked as a tech lead and project manager to modernize the existing on-premises data warehouse, migrating it to the Azure cloud and using modern data warehouse architecture.
  • Leveraged several technologies, including Azure Synapse, Azure Data Factory, Azure Analysis Services, and Microsoft Power BI.
  • Delivered a robust solution with dozens of data models, complete with CI/CD, monitoring, disaster recovery, cost optimizations, and rich documentation.
Technologies: Azure, SQL, Azure Data Factory (ADF), Azure SQL Data Warehouse, Dedicated SQL Pool (formerly SQL DW), Azure Synapse, Azure Analysis Services, Microsoft Power BI, Azure Databricks, DAX, T-SQL (Transact-SQL), Azure DevOps, Git, Salesforce, Agile, Scrum, Scrum Product Owner, Scrum Master, Agile Project Management, IT Project Management, DevOps, Data Engineering, Data Architecture, Data Modeling, ETL, Data Warehousing, Data Warehouse Design, Cloud, Data Pipelines, Computer Science, Azure PaaS, Azure Resource Manager (ARM), Azure Virtual Networks, Product Owner, Product Ownership, Certified Scrum Product Owner (CSPO), Certified ScrumMaster (CSM), Microsoft Excel, Azure Blob Storage API

Systems Analyst

2019 - 2020
DLL Group
  • Worked closely with business users to analyze, develop, and support new integrations, automation tools, ETL jobs, reports, and APIs.
  • Discovered, reported, and helped fix multiple security incidents affecting internal systems.
  • Obtained the appointment of "top performer" for DDL Brazil in 2019.
Technologies: SQL Server Integration Services (SSIS), SQL, T-SQL (Transact-SQL), APIs, SQL Server Reporting Services (SSRS), SSRS Reports, Agile, Scrum Product Owner, Scrum Master, Scrum, SharePoint, Azure DevOps, Git, DevOps, ETL, Data Warehousing, Data Pipelines, Computer Science, Product Ownership, Product Owner, Certified Scrum Product Owner (CSPO), Certified ScrumMaster (CSM), Microsoft Excel

IT Intern

2017 - 2019
DLL Group
  • Closed hundreds of level-two support tickets for local systems.
  • Developed and documented an internal tool for data anonymization using SQL Server Integration Services (SSIS).
  • Analyzed, developed, and supported integrations, automation tools, and ETL jobs.
  • Created reports and dashboards for business stakeholders using SQL Server Reporting Services (SSRS).
Technologies: SQL, T-SQL (Transact-SQL), SQL Server Integration Services (SSIS), SQL Server Reporting Services (SSRS), AutoIt, Azure DevOps, Git, ETL, Data Warehousing, Data Pipelines, Computer Science, Microsoft Excel

Experience

Data Lakehouse Implementation for a Consumer Packaged Goods Company

I worked as a data engineering lead and data architect in a team of twenty engineers to design and implement a new global data platform for a large consumer packaged goods company. This project involved transitioning from a centralized modern data warehouse architecture into a data mesh-like distributed data lakehouse architecture.

After nine months, my team delivered an internal portal to create and catalog new data products. We generated the necessary infrastructure using Terraform and the incremental data pipelines using Azure Data Factory and Azure Databricks. Today, more than 700 data engineers, analysts, and scientists use the new global data platform.

Education

2023 - 2023

Specialization in Data Engineering

Pontifical Catholic University of Minas Gerais - Minas Gerais, Brazil

2015 - 2021

Bachelor's Degree in Computer Science

Federal University of Rio Grande do Sul - Rio Grande do Sul, Brazil

Certifications

SEPTEMBER 2022 - SEPTEMBER 2024

Databricks Certified Data Engineer Professional

Databricks

JUNE 2022 - JUNE 2025

Google Cloud Certified Associate Cloud Engineer

Google Cloud

AUGUST 2021 - AUGUST 2023

Microsoft Certified: Azure Data Engineer Associate

Microsoft

JULY 2021 - PRESENT

Microsoft Certified: Azure Fundamentals

Microsoft

JULY 2021 - PRESENT

Microsoft Certified: Azure Data Fundamentals

Microsoft

FEBRUARY 2020 - FEBRUARY 2022

Certified ScrumMaster (CSM)

Scrum Alliance

NOVEMBER 2019 - NOVEMBER 2021

Certified Scrum Product Owner (CSPO)

Scrum Alliance

Skills

Libraries/APIs

PySpark, Azure Blob Storage API

Tools

Microsoft Power BI, Git, GitHub, Microsoft Excel, Apache Airflow, Terraform, Google Cloud Composer, Oracle E-Business Suite (EBS) R12, Microsoft Dynamics 365 for Finance and Operations

Languages

Python, SQL, T-SQL (Transact-SQL), AutoIt

Frameworks

Spark, Data Lakehouse

Paradigms

Agile, Scrum, ETL, Azure DevOps, Agile Project Management, DevOps

Platforms

Azure, Databricks, Azure Synapse, Azure PaaS, Azure SQL Data Warehouse, Dedicated SQL Pool (formerly SQL DW), Azure IaaS, Google Cloud Platform (GCP), Microsoft Dynamics 365, Salesforce, SAP HANA, SharePoint, Microsoft Power Automate

Storage

Azure SQL, Azure SQL Databases, Data Lakes, SQL Server Integration Services (SSIS), Microsoft SQL Server, Data Pipelines, Azure Table Storage, SQL Server Reporting Services (SSRS), Attunity Replicate, Google Cloud, Azure Cosmos DB

Other

Azure Databricks, Azure Analysis Services, Azure Data Factory (ADF), Data Modeling, Data Engineering, Computer Science, Azure Data Lake, Azure Resource Manager (ARM), Regular Expressions, APIs, Data Architecture, Delta Lake, Data Warehousing, Data Warehouse Design, Cloud, Azure Virtual Machines, Azure Virtual Networks, Scrum Master, Product Owner, Product Ownership, Scrum Product Owner, Certified Scrum Product Owner (CSPO), Certified ScrumMaster (CSM), DAX, SSRS Reports, IT Project Management, Tabular Editor, DAX Studio, Power BI Report Builder, Oracle EBS, SAP, SAP Business Warehouse (BW)

Collaboration That Works

How to Work with Toptal

Toptal matches you directly with global industry experts from our network in hours—not weeks or months.

1

Share your needs

Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2

Choose your talent

Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3

Start your risk-free talent trial

Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.

Top talent is in high demand.

Start hiring