Azure Developer in Calgary, AB, Canada

My responsibilities included the following activities:
• Project assessment
• Work breakdown structure (WBS) on different projects
• Estimating project timeline and resources
• Architecting and designing the data ingestion of the 12 data sources, including SAP S/4HANA, Tieto EC, Bourque Logistics, Endur Openlink, OSI PI Historian, Power Apps Dataverse, Workday, Enablon, SABA, IBM Maximo, Primavera P6, and EcoSys to Data Lake using Azure Synapse
• Architecting and designing the data integration using Azure Service Bus and Logic Apps that provide the event-driven data
• Architecting and designing the data protection and security requirements for Azure Purview
• Enterprise data modeling using Azure IDW
• Engineering the data ingestion from different data sources to Azure Data Lake and Azure Synapse Dedicated Pool (data warehouse)
• Security and access control on the enterprise data management system (EDMS)
• Self-served analytics framework
• Guiding and supporting the Power BI team on data consumption best practices
• Building the metadata ingestion, catalog, and lineage and integrating Azure Purview to different Azure resources used in EDMS
• Chair of the Data Governance Council
• Holding training to the junior data engineers

My tasks included the following:
• Architecting and productionalizing the MDA using lambda architecture, which included designing and spinning up the Data Lake and Delta Lake, designing and implementing data integration and analytics, monitoring using Azure Log Analytics, designing key management with Azure Key Vault, creating and configuring VM, Vnet, and NIC, and designing and implementing MDA operating and security model
• Developing data lake governance and data engineering governance
• Creating DevOps pipeline and releases using Azure DevOps (CI/CD)
• Working collaboratively with Microsoft solution architect, data engineers, data scientist, AltaML data scientist, DataBricks solution architect, and third-party vendors
• Project assessments and converting business and functional requirements into a cost-effective and robust solution
• Guiding the junior data engineers in data preparation and transformation
• Documenting solution architecture and design (SAD), which includes conceptual, logical, and physical design

I architected batch streaming (IoT), implemented the design on pulling data from AspenTech IP2, and loaded 21 wind farms, BC Hydro, and Sarnia plants. In addition to this, I architected the live streaming (IoT), implemented the design, and completed the proof-of-concept (POC) of five wind farms and device event logs. I played a key role in engaging with the business stakeholders, data owners, and SOX compliance team while assessing each project for data classification and sensitivity, NERC security, and personal identifiable information (PII data).

My responsibilities included:
• Architecting MLOPs and integrating to DevOps
• Creating wind power prediction model using machine learning
• Developing ML batch and live-scoring
• Creating pitch curve data analysis for underperforming turbines
• Developing MLOps for ML deployment to production and retraining the models

I developed a data flow for structured, semi-structured, and unstructured datasets by pulling data from SAP BW, SAP HANA, SharePoint with Logic Apps, Oracle databases, SQL on-premises, Azure SQL DB with Azure Data Factory, and from different websites (secured and public).

As an architecture owner following the Agile methodology, I worked closely with the product owner, scrum master, and Microsoft team, including solution architect, data and AI consultants, and project manager. I wrote the PowerShell scripting, oversaw the implementation of Suncor’s Digital Hub in Microsoft Azure for six pilot projects, and designed and configured the following items:
• security, high availability, and recovery in Azure Cloud,
• identity access management, role-based access control, and access control list,
• key management using Azure Key Vault,
• workspace and user table security in Databricks,
• advanced analytics model management,
• data integration and live streaming from OSI PI Historian to Azure IoT Hub,
• batch data ingestion and processing of SAP BW, SAP ECC, flat files/images data from on-premises data sources to Azure Data Lake Gen2
• enterprise data warehouse and compliance with an enterprise reference model,
• data catalog, architecture, and data modeling, and
• Azure Log Analytics.

My responsibility was to deliver security architecture documentation for Collibra SaaS, JobServer, and Lineage, onboarding checklist and framework of the data sources ingestion in Collibra, and best practice security and access control. I provided security best practices, guidance and support, and review and assessment of the current security architecture of the data management cloud platform.

I worked closely with the enterprise architect to develop standard processes and procedures, define different personas responsibilities, configure JobServer, and integrate SQL servers with Collibra using the native connector. I architected and designed the custom integration of Azure services to Collibra using REST API, provided technical requirements for API integration with Azure, and oversaw the creation of workflow and data and business dictionary.

My responsibilities included the following activities:
• Designing the project control conversion program in SAP ABAP by loading, translating, and pushing vendor’s data from spreadsheets to SAP HANA
• Designing data marts using SAP HANA model that integrates SAP Project Systems, SAP FICO, Primavera, and the conversion program data
• Creating the segregation of duties and roles
• Ensuring the proper security in Power BI, SAP HANA, and SAP ABAP
coordinating with the SAP Security specialist and governance risk compliance (GRC) team to provide appropriate security for the SAP ABAP conversion program and SAP HANA models
• Technical analysis, documenting the current and future state design, and preparing the solution design, security design, and technical design documentation

I worked with the Microsoft team on geo migration of Dynamic CRM, changed the Canvass apps data source from SQL DB to Data Lake, ingested CRM data (Dataverse) to Data Lake, and configured Azure Synapse link.