Daniel Brum
Verified Expert in Engineering
Data Engineer and Developer
Porto Alegre - State of Rio Grande do Sul, Brazil
Toptal member since October 24, 2023
Daniel is a data engineer and architect specializing in Azure's cloud services. He's keen on collaborating with data analysts and data scientists to provide them with accurate data for dashboards and experiments, leveraging his clear communication skills and problem-solving mindset. As a certified Azure data engineer and a Databricks data engineer professional, Daniel's tech stack includes Databricks, Azure Data Factory, Azure Synapse, Azure Analysis Services, and Microsoft Power BI.
Portfolio
Experience
- SQL - 9 years
- Python - 7 years
- Azure - 5 years
- Data Engineering - 5 years
- Microsoft Power BI - 5 years
- Data Modeling - 5 years
- Azure Synapse - 5 years
- Databricks - 4 years
Preferred Environment
Azure, Databricks, Azure Synapse, Microsoft Power BI, Azure Data Factory (ADF), Spark, Python, Apache Airflow, SQL, Agile
The most amazing...
...data platform I've designed and implemented supports more than 700 data engineers, analysts, and scientists daily.
Work Experience
Data Architect | Data Engineer
Toptal Clients
- Designed and fully implemented data platforms in the Azure cloud.
- Deployed and managed Azure subscriptions, resource groups, as well as resources for data lakehouse and data warehouse implementations with a focus on scalability, security, maintainability, and cost.
- Leveraged tools and technologies such as Databricks (Apache Spark), Microsoft Fabric, Data Factory, Azure Synapse, Power BI, SQL, Python, DAX, and PowerShell to develop metadata-driven data pipelines and enable several data analytics use cases.
- Developed Python code to extract source system data incrementally through REST and SOAP APIs.
- Created Spark SQL code to transform and integrate the data in an enterprise-level dimensional model to drive Power BI reports.
- Ensured code quality using static analysis tools such as Ruff and SQLfluff.
- Automated infrastructure and code deployments using Azure DevOps pipelines, Terraform, Bash, and PowerShell.
Azure Data Engineer
Meta IT North America
- Designed and implemented DW and BI projects using Microsoft Azure, Microsoft SQL Server, Azure Synapse, Azure Data Factory, Azure Analysis Services, and Microsoft Power BI.
- Developed and optimized generic data pipelines to efficiently ingest a large volume of raw data and reliably transform it into dimensional models.
- Set up build and release pipelines on Azure DevOps for continuous integration and deployment.
- Optimized and developed DAX measures using Tabular Editor and DAX Studio to drive reports and dashboards in Microsoft Power BI.
Data Engineering Specialist
Gavb
- Consulted with external clients on the design of cloud-native big data platforms using modern data warehouses, lakehouses, and cloud-scale analytics.
- Implemented big data solutions using technologies such as Databricks, Delta Lake, Apache Airflow, Azure Data Factory, Azure Synapse, and Microsoft Power BI.
- Shared best practices for information security, data governance, and cost and performance optimization.
- Developed data pipelines and customized frameworks for automatic batch generation and streaming pipelines through metadata.
Data Engineer
DLL Group
- Worked as a tech lead and project manager to modernize the existing on-premises data warehouse, migrating it to the Azure cloud and using modern data warehouse architecture.
- Leveraged several technologies, including Azure Synapse, Azure Data Factory, Azure Analysis Services, and Microsoft Power BI.
- Delivered a robust solution with dozens of data models, complete with CI/CD, monitoring, disaster recovery, cost optimizations, and rich documentation.
Systems Analyst
DLL Group
- Worked closely with business users to analyze, develop, and support new integrations, automation tools, ETL jobs, reports, and APIs.
- Discovered, reported, and helped fix multiple security incidents affecting internal systems.
- Obtained the appointment of "top performer" for DDL Brazil in 2019.
IT Intern
DLL Group
- Closed hundreds of level-two support tickets for local systems.
- Developed and documented an internal tool for data anonymization using SQL Server Integration Services (SSIS).
- Analyzed, developed, and supported integrations, automation tools, and ETL jobs.
- Created reports and dashboards for business stakeholders using SQL Server Reporting Services (SSRS).
Experience
Data Lakehouse Implementation for a Consumer Packaged Goods Company
After nine months, my team delivered an internal portal to create and catalog new data products. We generated the necessary infrastructure using Terraform and the incremental data pipelines using Azure Data Factory and Azure Databricks. Today, more than 700 data engineers, analysts, and scientists use the new global data platform.
Education
Specialization in Data Engineering
Pontifical Catholic University of Minas Gerais - Minas Gerais, Brazil
Bachelor's Degree in Computer Science
Federal University of Rio Grande do Sul - Rio Grande do Sul, Brazil
Certifications
Databricks Certified Data Engineer Professional
Databricks
Google Cloud Certified Associate Cloud Engineer
Google Cloud
Microsoft Certified: Azure Data Engineer Associate
Microsoft
Microsoft Certified: Azure Fundamentals
Microsoft
Microsoft Certified: Azure Data Fundamentals
Microsoft
Certified ScrumMaster (CSM)
Scrum Alliance
Certified Scrum Product Owner (CSPO)
Scrum Alliance
Skills
Libraries/APIs
PySpark, Azure Blob Storage API
Tools
Microsoft Power BI, Git, GitHub, Microsoft Excel, Apache Airflow, Terraform, Google Cloud Composer, Oracle E-Business Suite (EBS) R12, Microsoft Dynamics 365 for Finance and Operations
Languages
Python, SQL, Transact-SQL (T-SQL), AutoIt
Frameworks
Spark, Data Lakehouse
Paradigms
Agile, Scrum, ETL, Azure DevOps, Agile Project Management, DevOps
Platforms
Azure, Databricks, Azure Synapse, Azure PaaS, Azure SQL Data Warehouse, Dedicated SQL Pool (formerly SQL DW), Azure IaaS, Google Cloud Platform (GCP), Microsoft Dynamics 365, Salesforce, SAP HANA, SharePoint, Microsoft Power Automate
Storage
Azure SQL, Azure SQL Databases, Data Lakes, SQL Server Integration Services (SSIS), Microsoft SQL Server, Data Pipelines, Azure Table Storage, SQL Server Reporting Services (SSRS), Attunity Replicate, Google Cloud, Azure Cosmos DB
Other
Azure Databricks, Azure Analysis Services, Azure Data Factory (ADF), Data Modeling, Data Engineering, Computer Science, Azure Data Lake, Azure Resource Manager (ARM), Regular Expressions, APIs, Data Architecture, Delta Lake, Data Warehousing, Data Warehouse Design, Cloud, Azure Virtual Machines, Azure Virtual Networks, Scrum Master, Product Owner, Product Ownership, Scrum Product Owner, Certified Scrum Product Owner (CSPO), Certified ScrumMaster (CSM), DAX, SSRS Reports, IT Project Management, Tabular Editor, DAX Studio, Power BI Report Builder, Oracle EBS, SAP, SAP Business Warehouse (BW)
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring