Paweł is available for hire
Hire PawełPaweł Mitruś
Verified Expert in Engineering
Data Architect and Developer
Location
Warsaw, Poland
Toptal Member Since
September 10, 2021
Paweł is a data engineer and architect with several years of experience building data platforms with a range of technologies, including Azure and Microsoft. Apart from traditional ETLs, data lakes, and data warehouses, he is also proficient with various business intelligence tools and services. For the past few years, Paweł's focused on cloud projects, sourcing from both on-premise and cloud locations. Recently, Paweł's been working as a lead architect on a major data mesh implementation.
Portfolio
Lingaro
Azure, ETL, Data Lakes, Databricks, Azure Data Factory, Azure Analysis Services...
Azum
Python, Django, Domain-driven Design (DDD), SQL, Microsoft Power BI, Scrum...
ITMAGINATION
Azure, Cloud Infrastructure, Azure Data Factory, Azure Analysis Services...
Experience
Availability
Part-time
Preferred Environment
Azure, Databricks, SQL, PySpark, Azure Data Factory, Microsoft Power BI, Azure SQL, Azure SQL Data Warehouse, Dedicated SQL Pool (formerly SQL DW), SQL Server BI, Azure Analysis Services
The most amazing...
...role was as a lead architect on a data mesh project that involved over 40 developers and 20 different domain teams to integrate it into the platform.
Work Experience
Solution Architect
2019 - PRESENT
Lingaro
- Led a team of 6-8 tech leads to design and develop a data mesh platform that consisted of several microservices; also helped to plan automation in context of CI/CD.
- Delivered about 20 different training sessions (internally and externally in conferences) about best practices and anti-patterns regarding the Databricks platform that aimed to upskill participants.
- Designed and developed a custom ETL framework with a WYSIWYG editor, that non-developers can use to onboard their own ETL pipelines in a self-service manner. The framework is similar to ADF Data Flows which was also executed on Databricks.
- Helped to optimize the performance of Spark applications by applying best practices and mitigating future issues.
- Conducted multiple Azure Monitor analyses that aimed at finding misused services, e.g., in big data batch processing, knowing how the ratio of several markers should look like and performing an analysis resulted in $200,000 in savings.
- Consulted in multiple "traditional" data lake, data warehouse (DWH), and online analytical processing (OLAP) projects and helped to plan architecture for specific requirement sets and establish and configure environments (Azure).
Technologies: Azure, ETL, Data Lakes, Databricks, Azure Data Factory, Azure Analysis Services, Azure SQL, Azure SQL Data Warehouse, Dedicated SQL Pool (formerly SQL DW), Microsoft Power BI, Azure DevOps, Azure App Service, Azure Logic Apps, Architecture, Cloud, SQL, PySpark, Python, Distributed Systems, SQL Server DBA, Azure Data Lake, Azure Event Hubs, Visual Studio Code (VS Code), Cloud Infrastructure, Azure Resource Manager (ARM), Azure Virtual Machines, Scrum, Agile, Data Engineering, Data Modeling, Data Pipelines, JSON, REST APIs, T-SQL (Transact-SQL), Apache Spark, Big Data, Data Analytics, Data Architecture, Kimball Methodology
Freelance Lead Analytics Developer and Product Designer
2019 - 2021
Azum
- Designed monitoring-and-analytics features for sports activities that were uploaded to the Azum platform from users' devices.
- Described and helped to understand developers how FIT, TCX, and GPX files containing activities details should be processed and how to interpret it.
- Helped to organize the process of gathering requirements, specifying them, and handing them over to the development team in a Scrum manner.
Technologies: Python, Django, Domain-driven Design (DDD), SQL, Microsoft Power BI, Scrum, Agile, Data Engineering, Data Modeling, JSON, Data Architecture
Solution Architect
2017 - 2019
ITMAGINATION
- Led several teams, as a solution architect, on different projects with 11-15 developers and successfully delivered over ten data analytics platforms with over 500 end-users in total.
- Planned and executed a major migration from SQL Server 2008R2 to a 2016 BI platform that consisted of 15 different areas.
- Optimized a data warehouse refresh from 12 to four hours, mostly by applying appropriate data structures and indexes but also partitioning tables.
- Implemented a data quality panel into an existing SSIS framework that gathered information about rows read/inserted to enable tracking row counts through different data layers (staging, data warehouse, and semantic).
Technologies: Azure, Cloud Infrastructure, Azure Data Factory, Azure Analysis Services, Azure SQL Data Warehouse, Dedicated SQL Pool (formerly SQL DW), SQL Server BI, SQL Server DBA, SQL Server Integration Services (SSIS), SQL Server Analysis Services (SSAS), SQL Server Reporting Services (SSRS), Microsoft Power BI, Azure Resource Manager (ARM), Azure Virtual Machines, Databricks, Architecture, Azure SQL, Cloud, SQL, Azure Data Lake, Visual Studio, ETL, Data Lakes, Azure DevOps, Scrum, Agile, Data Engineering, Data Modeling, Data Pipelines, T-SQL (Transact-SQL), Data Analytics, Data Architecture, Kimball Methodology
Data Developer
2014 - 2017
ITMAGINATION
- Helped to design data warehouse star schemas and fact and dimension tables (Ralph Kimball) by analyzing the client's requirements together with the team and also as an individual.
- Built and released data warehouse (DWH) and business intelligence (BI) projects which included integrations with SSIS, a data warehouse hosted on SQL Server 2012-2016, an OLAP database as SSAS (multidimensional and tabular), and reports in SSRS.
- Developed an MDM system based on SQL Server 2012 MDS which included training data stewards (clients) on how to use both the app and Excel form.
- Delivered a couple of training sessions regarding PowerQuery, PowerPivot, PowerReport, and advanced use of pivot tables in Microsoft Excel (self-service BI).
Technologies: SQL Server BI, SQL Server DBA, SQL Server Integration Services (SSIS), SQL Server Reporting Services (SSRS), SQL Server Analysis Services (SSAS), SQL, Visual Studio, ETL, Scrum, Agile, Data Engineering, Data Modeling, Data Pipelines, T-SQL (Transact-SQL), Data Analytics, Data Architecture, Kimball Methodology
Experience
Data Mesh
I worked as a lead architect in a data mesh implementation (the technical part can be found at Martinfowler.com/articles/data-mesh-principles.html) in the FMCG field. I worked with the tech leads of smaller development teams and discussed and agreed upon the low-level architecture. I also shared my expertise in Databricks utilization, as it stands for the processing engine of the platform.
Technology Stack: Azure, Databricks (Python), Airflow, Azure SQL, Azure Data Lake Gen2, App Services
Technology Stack: Azure, Databricks (Python), Airflow, Azure SQL, Azure Data Lake Gen2, App Services
Azure Data Analytics Platform
A data analytics platform hosted on Azure that was built mostly for self-service in terms of accessing datasets and building reports. The majority of the advanced users were using Databricks for their prototypes before handed over to implementation. The solution consisted of both batch and near real-time processing.
My role mostly involved consulting on the architecture and helping plan the implementation. I also helped out to resolve performance problems and adjust cloud utilization to lower the overall costs.
Technology Stack: Azure, Data Factory, Databricks, Azure SQL, Azure SQL Data Warehouse (Synapse), Databricks, Azure Data Lake Gen2, Event Hub, Azure Analysis Services, Power BI
My role mostly involved consulting on the architecture and helping plan the implementation. I also helped out to resolve performance problems and adjust cloud utilization to lower the overall costs.
Technology Stack: Azure, Data Factory, Databricks, Azure SQL, Azure SQL Data Warehouse (Synapse), Databricks, Azure Data Lake Gen2, Event Hub, Azure Analysis Services, Power BI
Global Business Intelligence
I worked as an architect for a business intelligence platform, sourcing mostly MS Dynamics AX that was deployed for several regions all over the world.
The development work lasted for over two years and involved 5-7 developers. We implemented ETL in batch mode once per day so users could access the data warehouse (DWH), OLAP database, or predefined reports. Due to the immaturity of the Azure PaaS services, we decided to host the solution mostly on VMs (IaaS).
Technology stack: Azure, MS SQL Server 2016 (SSIS, SSRS), Azure Analysis Services, PowerBI
The development work lasted for over two years and involved 5-7 developers. We implemented ETL in batch mode once per day so users could access the data warehouse (DWH), OLAP database, or predefined reports. Due to the immaturity of the Azure PaaS services, we decided to host the solution mostly on VMs (IaaS).
Technology stack: Azure, MS SQL Server 2016 (SSIS, SSRS), Azure Analysis Services, PowerBI
Skills
Languages
SQL, T-SQL (Transact-SQL), Python
Tools
SQL Server BI, Microsoft Power BI, Visual Studio, Azure App Service, Azure Logic Apps
Paradigms
ETL, Scrum, Agile, Kimball Methodology, Azure DevOps, DSDM
Platforms
Azure, Databricks, Azure SQL Data Warehouse, Visual Studio Code (VS Code), Dedicated SQL Pool (formerly SQL DW), Azure Event Hubs
Storage
Azure SQL, Data Lakes, Data Pipelines, SQL Server Analysis Services (SSAS), SQL Server Integration Services (SSIS), SQL Server Reporting Services (SSRS), SQL Server DBA, JSON
Other
Azure Data Factory, Architecture, Cloud, Data Engineering, Data Modeling, Data Architecture, Azure Analysis Services, Azure Data Lake, Domain-driven Design (DDD), Cloud Infrastructure, Azure Resource Manager (ARM), Big Data, Data Analytics, Distributed Systems, Azure Virtual Machines, Data Mesh
Frameworks
Apache Spark, Django
Libraries/APIs
PySpark, REST APIs
Education
2011 - 2015
Engineer's Degree in Computer Science
Warsaw University of Technology - Poland, Warsaw
Certifications
DECEMBER 2019 - PRESENT
Azure Solutions Architect
Microsoft
DECEMBER 2018 - PRESENT
Agile PM
APMG International
DECEMBER 2018 - PRESENT
Professional Scrum Master 1 (PSM1)
Scrum.org
FEBRUARY 2017 - PRESENT
Microsoft Certified Professional
Microsoft
Collaboration That Works
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
1
Share your needs
Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2
Choose your talent
Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3
Start your risk-free talent trial
Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.
Top talent is in high demand.
Start hiring