Director | Data Engineering
2019 - 2021IRI- Developed Azure Data Factory pipelines to integrate data from Apache Hive, HDFS, OAuth 2 APIs, and various flat-file types into Azure SQL.
- Managed a team of onshore and offshore big data developers, assigning tasks and tracking the progress on Jira.
- Oversaw data strategy and recommendations for new data sources and ongoing projects.
- Mentored big data engineers to help them develop their skills.
- Architected new data models and upgraded old data warehouses as per client request or technology change.
Technologies: Python, Apache Hive, Hadoop, Azure Synapse, Azure Data Factory, Spark SQL, Bash Script, SQL, Azure SQL, Databricks, Data Engineering, ETL, Data Modeling, Databases, Azure, Data, Data Architecture, Business Intelligence (BI), Data Pipelines, Apache Airflow, Data Integration, Big Data, T-SQL, Data Migration, Snowflake, Data Building Tool (DBT), Apache Kafka, ELT, SSIS, Data TransformationETL Architect
2016 - 2019IRI- Developed SQL-based data warehouses on-premise and on the cloud.
- Integrated various data sources from flat files to cloud-based data sources like Snowflake, AWS and data lakes into Azure Data Warehouse, and Apache Hive on Hadoop.
- Created scalable data pipelines and improved efficiencies on the existing ones.
- Trained and upskilled new data developers and participated in code reviews.
- Maintained system documentation of all business data components and strategies.
Technologies: SQL Server Integration Services (SSIS), Azure Synapse, Azure Data Factory, Databricks, PySpark, SQL, Oracle, Apache Hive, Hadoop, Data Warehouse Design, Data Engineering, ETL, Data Modeling, SQL Stored Procedures, Databases, Data, Data Architecture, Business Intelligence (BI), Data Pipelines, Data Integration, Big Data, BigQuery, JavaScript, T-SQL, Data Migration, Snowflake, AWS, AWS EMR, ELT, SSIS, APIs, Data Transformation, MariaDBSQL Lead Developer
2012 - 2016IRI- Developed SQL-based data warehouses and data marts.
- Wrote SQL queries to provide data for SSRS reports.
- Used SSIS, Talend, and DataStage for ETL processes depending on the client's requirements.
- Created custom business reports using SQL Server Reporting Services (SSRS).
- Managed junior developers and ran stand-up development meetings.
Technologies: SQL, SQL Server Integration Services (SSIS), SQL Server Reporting Services (SSRS), PSQL, MySQL, Talend ETL, IBM InfoSphere (DataStage), Data Warehousing, Data Engineering, ETL, Data Modeling, SQL Stored Procedures, Databases, Data, Data Architecture, Business Intelligence (BI), Data Pipelines, Data Integration, Big Data, T-SQL, Data Migration, ELT, SSIS, Data TransformationSQL/ETL Developer and Consultant
2010 - 2012Mi9 Retail (formerly JustEnough Software Corporation)- Managed SQL replication between mobile devices and SQL Server.
- Created SQL data warehouses using the Kimball methodology for reporting purposes.
- Designed and developed ETL packages using SQL Server Integration Services (SSIS).
- Designed and developed reports in SQL Server Reporting Services (SSRS).
- Performed database tuning and code reviews for any code being deployed to production.
Technologies: SQL, SQL CE, SQL Server Integration Services (SSIS), SQL Server Reporting Services (SSRS), Microsoft SQL Server, Data Engineering, ETL, Data Modeling, SQL Stored Procedures, Databases, Data, Data Architecture, Business Intelligence (BI), Data Pipelines, Data Integration, Big Data, T-SQL, Data Migration, SSIS, Data Transformation