Cloud Data Engineer2020 - PRESENTRoche
Technologies: Azure, Python, SQL, MSSQL, Databricks, Spark, Streaming
- Created data pipelines for IoT solutions in Python, PySpark, Azure Data Factory, and Azure Synapse.
- Validated and migrated on-prem Python to Azure-specific offerings.
- Documented the process of ingesting and transforming data.
Senior Data Engineer/Database Lead2019 - 2020Aberdeen
Technologies: SQL Server, SSIS, C#, Python, AWS, MSSQL, TSQL, PySpark, Spark
- Created new ways to clean up the data, using web scraping and algorithms (Python, PySpark).
- Designed master data.
- Designed and implemented ETL processes.
- Handled SQL Server administration including data partitioning, index maintenance, security, and backup policies.
- Automated QA testing in Python.
- Maintained the Elasticsearch cluster.
- Successfully migrated whole product to AWS (re-designing the data product - using AWS EMR, s3, RDS, Lambda, Athena, Glue).
- Integrated heterogeneous data sources.
- Implemented business logic in Python.
Business Intelligence Developer2019 - 2019Ryanair
Technologies: SQL, SSIS, C#, SSAS, MDX, DAX, MSSQL, T-SQL, SQL Server
- Prepared PoC solutions in Azure(Databricks, ADF), graph databases, Power BI - what allowed BI team to.
- manage relationships between ETL processes easier.
- Created ETL processes, which allowed company to integrate data during LaudaMotion takeover and use Data Science algorithms to measure fuel consumption - using hadoop, Impala, python scripts for 3rd party API integration.
- Modeled new data marts focused on integrating big data solutions based on web analytics and marketing with consumer-related Data Warehouse.
- Developed Data Quality checks.
ETL & Automation Analyst2018 - 2018Ryanair
Technologies: VBA, SQL, Python, R, Java, MDX, Tableau, PowerBI, TSQL, MSSQL, SQL Server
- Query performance tuning - highly improved ETL processes performance required for reporting solutions in Marketing team.
- Data integration automation (R, Python, SQL, MDX) - eliminated human factor from ETL process completely (full automation).
- Resolved Tableau issues.
- Automated internal processes to get the data.
- Set up query performance tuning.
Business Intelligence Consultant2017 - 2018Tech Data
Technologies: SQL, TSQL, SSAS, SSIS, VBA, Python, Powershell, R, C#, MSSQL
- Resolved ETL issues.
- Created new ETL data flows.
- Developed advanced TSQL solutions.
- Automated tasks using Powershell/TSQL/SSIS.
- Designed and maintained reporting environment.
- Prepared reports and dashboards for high-level management.