
Paweł Mitruś
Verified Expert in Engineering
Data Architect and Developer
Warsaw, Poland
Toptal member since September 10, 2021
Paweł is a data engineer and architect with several years of experience building data platforms with a range of technologies, including Azure and Microsoft. Apart from traditional ETLs, data lakes, and data warehouses, he is also proficient with various business intelligence tools and services. For the past few years, Paweł's focused on cloud projects, sourcing from both on-premise and cloud locations. Recently, Paweł's been working as a lead architect on a major data mesh implementation.
Portfolio
Experience
- SQL - 8 years
- Azure - 6 years
- Azure Data Factory (ADF) - 4 years
- Azure SQL - 4 years
- Dedicated SQL Pool (formerly SQL DW) - 3 years
- Microsoft Power BI - 3 years
- Databricks - 3 years
- Azure SQL Data Warehouse - 3 years
Availability
Preferred Environment
Azure, Databricks, SQL, PySpark, Azure Data Factory (ADF), Microsoft Power BI, Azure SQL, Azure SQL Data Warehouse, Dedicated SQL Pool (formerly SQL DW), SQL Server BI, Azure Analysis Services
The most amazing...
...role was as a lead architect on a data mesh project that involved over 40 developers and 20 different domain teams to integrate it into the platform.
Work Experience
Solution Architect
Lingaro
- Led a team of 6-8 tech leads to design and develop a data mesh platform that consisted of several microservices; also helped to plan automation in context of CI/CD.
- Delivered about 20 different training sessions (internally and externally in conferences) about best practices and anti-patterns regarding the Databricks platform that aimed to upskill participants.
- Designed and developed a custom ETL framework with a WYSIWYG editor, that non-developers can use to onboard their own ETL pipelines in a self-service manner. The framework is similar to ADF Data Flows which was also executed on Databricks.
- Helped to optimize the performance of Spark applications by applying best practices and mitigating future issues.
- Conducted multiple Azure Monitor analyses that aimed at finding misused services, e.g., in big data batch processing, knowing how the ratio of several markers should look like and performing an analysis resulted in $200,000 in savings.
- Consulted in multiple "traditional" data lake, data warehouse (DWH), and online analytical processing (OLAP) projects and helped to plan architecture for specific requirement sets and establish and configure environments (Azure).
Freelance Lead Analytics Developer and Product Designer
Azum
- Designed monitoring-and-analytics features for sports activities that were uploaded to the Azum platform from users' devices.
- Described and helped to understand developers how FIT, TCX, and GPX files containing activities details should be processed and how to interpret it.
- Helped to organize the process of gathering requirements, specifying them, and handing them over to the development team in a Scrum manner.
Solution Architect
ITMAGINATION
- Led several teams, as a solution architect, on different projects with 11-15 developers and successfully delivered over ten data analytics platforms with over 500 end-users in total.
- Planned and executed a major migration from SQL Server 2008R2 to a 2016 BI platform that consisted of 15 different areas.
- Optimized a data warehouse refresh from 12 to four hours, mostly by applying appropriate data structures and indexes but also partitioning tables.
- Implemented a data quality panel into an existing SSIS framework that gathered information about rows read/inserted to enable tracking row counts through different data layers (staging, data warehouse, and semantic).
Data Developer
ITMAGINATION
- Helped to design data warehouse star schemas and fact and dimension tables (Ralph Kimball) by analyzing the client's requirements together with the team and also as an individual.
- Built and released data warehouse (DWH) and business intelligence (BI) projects which included integrations with SSIS, a data warehouse hosted on SQL Server 2012-2016, an OLAP database as SSAS (multidimensional and tabular), and reports in SSRS.
- Developed an MDM system based on SQL Server 2012 MDS which included training data stewards (clients) on how to use both the app and Excel form.
- Delivered a couple of training sessions regarding PowerQuery, PowerPivot, PowerReport, and advanced use of pivot tables in Microsoft Excel (self-service BI).
Experience
Data Mesh
Technology Stack: Azure, Databricks (Python), Airflow, Azure SQL, Azure Data Lake Gen2, App Services
Azure Data Analytics Platform
My role mostly involved consulting on the architecture and helping plan the implementation. I also helped out to resolve performance problems and adjust cloud utilization to lower the overall costs.
Technology Stack: Azure, Data Factory, Databricks, Azure SQL, Azure SQL Data Warehouse (Synapse), Databricks, Azure Data Lake Gen2, Event Hub, Azure Analysis Services, Power BI
Global Business Intelligence
The development work lasted for over two years and involved 5-7 developers. We implemented ETL in batch mode once per day so users could access the data warehouse (DWH), OLAP database, or predefined reports. Due to the immaturity of the Azure PaaS services, we decided to host the solution mostly on VMs (IaaS).
Technology stack: Azure, MS SQL Server 2016 (SSIS, SSRS), Azure Analysis Services, PowerBI
Education
Engineer's Degree in Computer Science
Warsaw University of Technology - Poland, Warsaw
Certifications
Azure Solutions Architect
Microsoft
Agile PM
APMG International
Professional Scrum Master 1 (PSM1)
Scrum.org
Microsoft Certified Professional
Microsoft
Skills
Libraries/APIs
PySpark, REST APIs
Tools
SQL Server BI, Microsoft Power BI, Visual Studio, Azure App Service, Azure Logic Apps
Languages
SQL, T-SQL (Transact-SQL), Python
Paradigms
ETL, Scrum, Agile, Kimball Methodology, Azure DevOps, DSDM
Platforms
Azure, Databricks, Azure SQL Data Warehouse, Visual Studio Code (VS Code), Dedicated SQL Pool (formerly SQL DW), Azure Event Hubs
Storage
Azure SQL, Data Lakes, Data Pipelines, SQL Server Analysis Services (SSAS), SQL Server Integration Services (SSIS), SQL Server Reporting Services (SSRS), SQL Server DBA, JSON
Frameworks
Apache Spark, Django
Other
Azure Data Factory (ADF), Architecture, Cloud, Data Engineering, Data Modeling, Data Architecture, Azure Analysis Services, Azure Data Lake, Domain-driven Design (DDD), Cloud Infrastructure, Azure Resource Manager (ARM), Big Data, Data Analytics, Distributed Systems, Azure Virtual Machines, Data Mesh
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring