Edison is currently unavailable
Edison Zhu
Verified Expert in Engineering
Azure BI Developer
Sydney, New South Wales, Australia
Toptal member since July 7, 2021
Bio
Edison is a seasoned BI developer and data engineer with 19 years of experience across diverse industries. Specializing in Azure and Google Cloud Platform (GCP), he excels in designing high-performance BI solutions using tools like Azure Data Factory, Python, Power BI, Tableau, Databricks, and GCP BigQuery. An expert in data architecture, dimensional modeling, and optimizing large-scale data warehouses, Edison delivers scalable, high-quality solutions backed by strong technical expertise.
Portfolio
Woolworths
Azure Synapse, Azure Analysis Services, Microsoft Power BI...
A Leading D365 Consulting Company
Dynamics CRM 365, Azure Data Factory (ADF), Azure SQL Databases...
Aerial Industries Pte. Ltd
Microsoft Access, JavaScript, Azure SQL, Angular, Angular Material, C#.NET...
Experience
- ETL - 18 years
- SQL Server BI - 16 years
- Azure Data Factory (ADF) - 9 years
- Azure SQL Databases - 9 years
- Microsoft Power BI - 9 years
- Databricks - 6 years
- Python 3 - 3 years
- Microsoft Fabric - 2 years
Preferred Environment
SQL Server BI, Python 3, Microsoft Power BI, Azure Data Factory (ADF), Azure Functions, Tableau Desktop Pro, Google BigQuery, Fabric, Azure Databricks, Google Cloud Platform (GCP)
The most amazing...
...enterprise projects I've delivered helped businesses make data-driven decisions and make business operations more efficient.
Work Experience
Azure BI Data Engineer | Fabric | Power BI | Tableau | GCP BigQuery Developer
2020 - PRESENT
Woolworths
- Designed and developed the online sales dashboard, store operation dashboard, point-to-point delivery dashboard, and product availability dashboards using Microsoft Power BI, Azure Synapse, Azure Analysis Services, and DAX.
- Delivered the Data Factory ETL pipelines to dynamically copy data into Azure Blob Storage and load the dimension and fact tables.
- Designed and developed the data warehouse, analysis service cube, ETL process, and dimensional modeling.
- Migrated the seven existing fulfillment web reports of Woolworths' online sales to SSRS reports using Azure Synapse and SSRS.
- Created Azure Functions to process XML messages with Python, Azure Service Bus, Azure Cosmos DB, and Azure Storage Queue.
- Developed point-to-point delivery and Online KPI Tableau dashboards using GCP BigQuery and Tableau.
- Built a data ingestion pipeline with Python to integrate Google Sheet's data into GCP BigQuery and Storage Buckets.
- Created the Azure Functions to create Azure Iot Hub and Stream Analytics automatically.
- Delivered Fabric data engineering projects using dataflow and pipeline. Integrated Dynamics 365 finance and operation data to Fabric Lakehouse. Built notebook to load dimension and fact tables. Integrated Salesforce data to Fabric Azure OpenAI use.
- Delivered a Dynamics 365 data integration project for a client in the logistics industry using ETL, Data Factory, Dynamics 365, Azure SQL databases BI modeling, and Power BI.
Technologies: Azure Synapse, Azure Analysis Services, Microsoft Power BI, Azure Data Factory (ADF), Tableau, Azure SQL Databases, Azure Logic Apps, Data Warehouse Design, Microsoft SQL Server, SQL Server Analysis Services (SSAS), Data Engineering, SQL, Google BigQuery, DAX, Apache Airflow, Azure SQL, Database Design, Azure, Relational Databases, Databases, Data Warehousing, APIs, Data Migration, API Connectors, ETL, XML, REST, ETL Tools, Business Intelligence (BI), Dashboards, Data Visualization, Pandas, Data Pipelines, Data Analytics, Data Lake Design, Data Lakes, Data, Cloud Platforms, Data Modeling, Data Analysis, Python 3, Azure Functions, SQL Server BI, Database Administration (DBA), SQL Server DBA, Azure Blobs, BigQuery, NoSQL, Webhooks, Power Query, ODBC, Microsoft Excel, OLTP, OLAP, Data Integration, Database Integration, Google Cloud Platform (GCP), Azure DevOps, ELT, BI Reporting, Excel 2010, SSRS Reports, Database Modeling, Minimum Viable Product (MVP), Consolidation, Stored Procedure, Cloud Migration, eCommerce, Looker, Jira, Transact-SQL (T-SQL), Azure SQL Data Warehouse, Dedicated SQL Pool (formerly SQL DW), Reporting, SAP, Data Cleansing, Data Quality Analysis, Database Migration, Google Data Studio, ETL Pipelines, Database Lifecycle Management (DLM), Cloud, Entity Relationships, Azure Databricks, SQL Stored Procedures, Performance Tuning, Data Management, Databricks, Data Feeds, Data Extraction, Delta Lake, .NET, C#, Architecture, Requirements Analysis, JSON, Fabric, Data Fabric, PySpark, Star Schema, Data Transformation, Microsoft Fabric, Integration, Automation, Query Optimization, Synapse, Azure Cloud Services, Microsoft Azure Cloud Server, CI/CD Pipelines, DevOps, Database Performance, Analytics, B2B, B2C, Customer Analytics, Excel 365, Looker Studio, Retrieval-augmented Generation (RAG), NumPy, Artificial Intelligence (AI), Large Language Models (LLMs), Finance, Financial Modeling, Google Cloud, Google Cloud Storage, Data Architecture, Security, Large-scale Data Migration, Data Validation, Cloud Key Management Service (KMS), Google Sheets, Data Reporting, Metadata, Data Processing Automation, Active Directory (AD), Azure Active Directory, Azure Data Lake Storage, REST APIs, Azure Blob Storage API, Klaviyo
Senior Data Engineer
2023 - 2024
A Leading D365 Consulting Company
- Built a data platform ingesting SMSF, pension, trust, investment, and insurance data into Azure SQL, transformed in Databricks (PySpark/Delta Lake), pushed to D365. Built customer unification logic, creating golden customer view across product lines.
- Led data migration from on-premises Dynamics to D365 cloud across policy, claims, customer, and broker domains. Mapped legacy entities to D365 schema, built validation, reconciliation, and rollback pipelines—zero data loss across migrated entities.
- Delivered public sector migration from on-premises Dynamics to D365 cloud. Mapped and migrated education-domain entities. Built repeatable auditable tooling for dry-run and production waves. Met gov education data governance and compliance standards.
Technologies: Dynamics CRM 365, Azure Data Factory (ADF), Azure SQL Databases, Azure Databricks
Database and Full-stack Developer
2023 - 2023
Aerial Industries Pte. Ltd
- Reverse-engineered the existing Microsoft Access, including the tables, queries, and forms. Migrated all the tables from Microsoft Access to the Azure SQL database.
- Developed the front end of the web app using Angular 16.
- Built the REST API using C#, .NET Core, and Azure SQL database.
Technologies: Microsoft Access, JavaScript, Azure SQL, Angular, Angular Material, C#.NET, .NET Core, .NET, C#, Requirements Analysis, Azure OpenAI Service, Azure AI Studio, Star Schema, Data Transformation, Integration, Automation, Azure Cloud Services, Microsoft Azure Cloud Server, CI/CD Pipelines, DevOps, TypeScript, Retrieval-augmented Generation (RAG), Artificial Intelligence (AI), Large Language Models (LLMs), Data Processing Automation, Active Directory (AD), REST APIs
Azure Data Engineer
2022 - 2022
Acumen Commercial Insights
- Delivered a data integration project in the retail FMCG industry.
- Built an ADF pipeline, Azure Function, and C# Web API to integrate data—including invoice, sales, customer, product, and payment—from SAP IDoc XML, SFTP, and API to the Azure SQL Database.
- Developed a Blob-triggered Azure Function for multiple clients to transform XML data—including XSD validation and XSLT transformation—generating an output file and sending it to the client by API.
- Designed the pipeline to ingest Excel and CSV files into the integration database, transform the Excel data to unpivot the data, and export the data to clients using ADF data flows.
Technologies: Data Engineering, Azure Data Factory (ADF), XML, SQL, REST, ETL Tools, APIs, Azure Functions, ASP.NET Web API, Database Architecture, Webhooks, Microsoft Excel, Azure Queue Storage, Data Integration, Database Integration, HTML, Azure DevOps, Excel 2010, Database Modeling, Minimum Viable Product (MVP), Consolidation, Stored Procedure, Azure App Service, GitHub, Jira, Transact-SQL (T-SQL), SAP, Data Cleansing, Data Quality Analysis, Database Migration, ETL Pipelines, Database Lifecycle Management (DLM), Cloud, Entity Relationships, SQL Stored Procedures, Performance Tuning, Data Extraction, .NET, C#, Architecture, Requirements Analysis, JSON, Data Transformation, Integration, Automation, Query Optimization, Synapse, Azure Cloud Services, Microsoft Azure Cloud Server, CI/CD Pipelines, DevOps, Excel 365, Finance, Data Processing Automation, Active Directory (AD), Azure Active Directory, Azure Data Lake Storage, REST APIs, Azure Blob Storage API, XML Schema
Azure Data Engineer and Data Architect
2018 - 2020
Antares Solutions
- Designed and developed the ETL, data warehouse, cube, and reports of a digital email campaign for a leading health supplement company. This automation saved the client a substantial amount of time.
- Delivered a real-time big data analytics architecture, which included a real-time big data IoT, machine learning, and advanced analytics platform for a leading entertainment company.
- Developed text classification for clients using Python, Keras, word embedding, NLP, Word2vec, GloVe, deep learning, and CNN.
- Produced a movie recommendation proof of concept using Azure Databricks, machine learning, collaborative filtering, real-time data ingestion, Azure Event Hubs, Spark MLlib, and Python.
- Created a data integration pipeline to ingest SAP IDoc XML data—including invoice, sales, customer, product, and payment—into the integration database using ADF, Azure SQL Database, and Azure Blob Storage.
- Built a Blob-triggered Azure Function for multiple clients to transform XML data—including XSD validation and XSLT transformation—generating an output file and sending it to the client by API.
Technologies: Azure Data Factory (ADF), Databricks, Azure Analysis Services, Microsoft Power BI, SSRS Reports, SQL Server Integration Services (SSIS), Python, Azure Event Hubs, Azure Stream Analytics, Azure Logic Apps, GloVe, Generative Pre-trained Transformers (GPT), Natural Language Processing (NLP), Data Warehouse Design, Microsoft SQL Server, Data Architecture, SQL Server Analysis Services (SSAS), Data Engineering, SQL, DAX, Azure SQL, Database Design, Azure, Relational Databases, Databases, Data Warehousing, Azure Machine Learning, APIs, Data Migration, API Connectors, ETL, XML, REST, ETL Tools, Business Intelligence (BI), Dashboards, Data Visualization, Pandas, Data Pipelines, Data Analytics, Marketing, PostgreSQL, Data Lake Design, Data Lakes, Data, Redshift, Cloud Platforms, Data Modeling, Data Analysis, Azure Functions, Azure SQL Databases, SQL Server BI, Database Administration (DBA), SQL Server DBA, Spark, Azure Blobs, Salesforce API, ASP.NET Web API, Database Architecture, BigQuery, NoSQL, Power Query, ODBC, JavaScript, Microsoft Excel, OLTP, OLAP, Snowflake, Data Integration, Database Integration, Web Scraping, Data Scraping, Scientific Data Analysis, Google Analytics, HTML, Azure DevOps, Apache Spark, ELT, BI Reporting, Excel 2010, Database Modeling, Minimum Viable Product (MVP), Consolidation, PL/SQL, Stored Procedure, Azure App Service, GitHub, Cloud Migration, eCommerce, Jira, Transact-SQL (T-SQL), Azure SQL Data Warehouse, Dedicated SQL Pool (formerly SQL DW), Reporting, Hadoop, Data Cleansing, Data Quality Analysis, Data Science, Database Migration, Big Data, ETL Pipelines, Database Lifecycle Management (DLM), Cloud, Entity Relationships, Amazon S3 (AWS S3), Azure Databricks, SQL Stored Procedures, Performance Tuning, Data Management, Data Feeds, Data Extraction, Delta Lake, Spark SQL, .NET, C#, Architecture, Requirements Analysis, JSON, Star Schema, Data Transformation, Crypto, Scraping, Integration, Automation, Query Optimization, Funnel Analysis, Synapse, Azure Cloud Services, Streaming Data, Microsoft Azure Cloud Server, CI/CD Pipelines, DevOps, Database Performance, Oracle, Analytics, Customer Analytics, Excel 365, TypeScript, NumPy, Security, Data Validation, Data Reporting, Metadata, Data Processing Automation, Active Directory (AD), Azure Active Directory, Azure Data Lake Storage, REST APIs, Azure Blob Storage API, GitHub Actions, XML Schema, Klaviyo
Senior Azure BI Developer
2017 - 2018
Woolworths Group
- Designed and built a dimensional modeling data warehouse, a tabular cube, and Power BI dashboard of online sales using Azure Data Warehouse, Azure Analysis Services cube, and Power BI.
- Performance-tuned large volume data with table partition, cube partition, polybase, and hash distribution on Azure Data Warehouse.
- Automated the cube process using Azure Automation account, runbooks, and webhooks.
Technologies: Azure Data Factory (ADF), Azure Synapse, Microsoft Power BI, Azure Analysis Services, Azure Logic Apps, Data Warehouse Design, Microsoft SQL Server, SQL Server Analysis Services (SSAS), Data Engineering, SQL, DAX, Azure SQL, Database Design, Azure, Relational Databases, Databases, Data Warehousing, Data Migration, ETL, ETL Tools, Business Intelligence (BI), Dashboards, Tableau, Data Visualization, Data Pipelines, Data Analytics, Data Lake Design, Data Lakes, Data, Cloud Platforms, Data Modeling, Data Analysis, Azure SQL Databases, SQL Server BI, SQL Server DBA, Azure Blobs, Database Architecture, Power Query, JavaScript, Microsoft Excel, OLTP, OLAP, Data Integration, Database Integration, Azure DevOps, BI Reporting, Excel 2010, SSRS Reports, Database Modeling, Minimum Viable Product (MVP), Consolidation, Stored Procedure, Cloud Migration, Transact-SQL (T-SQL), Azure SQL Data Warehouse, Dedicated SQL Pool (formerly SQL DW), Reporting, Data Cleansing, Data Quality Analysis, Database Migration, ETL Pipelines, Database Lifecycle Management (DLM), Cloud, Entity Relationships, Azure Databricks, SQL Stored Procedures, Performance Tuning, Data Management, Data Feeds, Data Extraction, C#, Architecture, Requirements Analysis, JSON, Star Schema, Data Transformation, Integration, Automation, Query Optimization, Synapse, Azure Cloud Services, Streaming Data, Microsoft Azure Cloud Server, CI/CD Pipelines, DevOps, Database Performance, Analytics, Excel 365, Data Validation, Data Reporting, Metadata, Data Processing Automation, Azure Active Directory, Azure Data Lake Storage, REST APIs, XML Schema
Senior SQL BI Consultant
2016 - 2017
Agile BI
- Delivered high-quality BI solutions and projects for clients on BI visualization and reports, data warehousing, SSAS tabular model cube, DAX/MDX, SQL Server, Microsoft BI, and Power BI.
- Modeled, designed, and developed independently a sales cube using a SSAS tabular model cube and DAX for a large wholesale/retail company.
- Delivered an advanced benchmark BI system and advanced SSRS reports for a leading accounting firm using SSRS, MDX, SSAS, an OLAP cube, and SQL Server.
Technologies: SQL Server Integration Services (SSIS), SSRS Reports, SQL Server BI, DAX, Microsoft Power BI, Data Warehouse Design, Microsoft SQL Server, SQL Server Analysis Services (SSAS), Data Engineering, SQL, Azure SQL, Database Design, Azure, Relational Databases, Databases, Data Warehousing, APIs, Data Migration, ETL, ETL Tools, Business Intelligence (BI), Dashboards, Data Visualization, Data Pipelines, Data Analytics, Microsoft Access, Data, Cloud Platforms, Data Modeling, Data Analysis, Python 3, Azure SQL Databases, Spark, Azure Blobs, Database Architecture, Power Query, JavaScript, Microsoft Excel, OLTP, OLAP, Data Integration, Database Integration, Data Scraping, HTML, Azure DevOps, ELT, BI Reporting, Excel 2010, Database Modeling, Consolidation, Stored Procedure, Azure App Service, GitHub, Stripe, Transact-SQL (T-SQL), Azure SQL Data Warehouse, Dedicated SQL Pool (formerly SQL DW), Financials, Reporting, Data Cleansing, Data Quality Analysis, Database Migration, Firebase, ETL Pipelines, Database Lifecycle Management (DLM), Cloud, Entity Relationships, SQL Stored Procedures, Performance Tuning, Data Management, Data Feeds, Data Extraction, C#, JSON, Star Schema, Data Transformation, Web Crawlers, Integration, Query Optimization, Synapse, Azure Cloud Services, Microsoft Azure Cloud Server, CI/CD Pipelines, DevOps, Database Performance, Excel 365, Data Validation, Data Reporting, Metadata, Data Processing Automation, REST APIs, GitHub Actions, XML Schema
Senior SQL BI Developer
2011 - 2016
Aussie Home Loan
- Designed and developed independently a new data warehouse, cube, ETL, and some reports for analyzing mortgage and non-mortgage sales data (including home loan, insurance other financial data).
- Designed and developed a GL cube for the finance department.
- Optimized and performance-tuned a large volume BI data warehouse.
Technologies: SQL Server Reporting Services (SSRS), SQL Server Integration Services (SSIS), SQL Server Analysis Services (SSAS), SQL Server BI, SQL, Database Architecture, Microsoft Excel, MySQL, OLTP, OLAP, Data Integration, BI Reporting, Excel 2010, SSRS Reports, Stored Procedure, Transact-SQL (T-SQL), Financials, Reporting, Data Cleansing, Data Quality Analysis, Database Migration, ETL Pipelines, Entity Relationships, SQL Stored Procedures, Performance Tuning, Data Management, Data Feeds, Data Extraction, Star Schema, Data Transformation, Database Performance, Data Processing Automation, Pay-per-Click (PPC)
SQL BI Developer
2010 - 2011
Hyundai
- Developed several reports using SSRS (Reporting Service 2008) for finance, marketing, and sales departments.
- Oversaw and was in charge of ETL development and developed an SSIS package to extract data from an ERP system and transformed and processed the data using SQL Server 2008.
- Worked with an analysis programmer and other developer to realize the system based on InfoPath, Microsoft Web Services, Workflow, and SQL Server 2008.
Technologies: SQL Server Integration Services (SSIS), SQL Server Reporting Services (SSRS), SQL Server BI, SQL, OLTP, OLAP, BI Reporting, SSRS Reports, Stored Procedure, Transact-SQL (T-SQL), Reporting, Data Cleansing, Data Quality Analysis, ETL Pipelines, Entity Relationships, SQL Stored Procedures, Data Extraction, Data Transformation
Microsoft BI Developer
2006 - 2010
Pactera
- Worked onsite at Microsoft and contributed to the development and support of five BI applications based on SQL Server, SSRS (reporting service), SSAS (cube), SSIS, ETL, data warehousing development, dimensional modeling design, and schema design.
- Developed reports using SSRS and implemented ETL through programming store procedures and SSIS package.
- Constructed the ETL with an SSIS package—slowly changing dimensions with store procedures, aggregation, a processing cube, and so on.
Technologies: C#.NET, SQL Server BI, SQL Server Integration Services (SSIS), SQL Server Reporting Services (SSRS), SQL, OLTP, OLAP, HTML, BI Reporting, SSRS Reports, Stored Procedure, Visual Basic for Applications (VBA), Excel VBA, Excel Macros, Macros, Transact-SQL (T-SQL), Reporting, Data Quality Analysis, ETL Pipelines, Entity Relationships, SQL Stored Procedures, C#
Experience
Australian Leading Supermarket | eCommerce DWH and Dashboard
Designed and developed an online sale and eCommerce store operations data warehouse and dashboard using Azure Data Warehouse, ADF, Azure Analysis Services cube, and Power BI technologies. The solution enabled businesses to navigate business operations better, helping businesses improve customer satisfaction, increase online sales, and save costs. I also developed the Tableau dashboard and BigQuery routines.
Blackmores | Digital Email Campaigns and Analytics
Designed and developed ETL, data warehouse, cube, and reports for digital email campaigns using ADF, logic apps, SharePoint, REST APIs, Azure Analytics Service, and Power BI. The automation saved the client lots of time instead of manually generating reports.
Intersport | Retail Inventory ETL and Dashboard
Developed Azure Function and Data Factory to extract inventory data from the REX system (SOAP API) to Azure SQL Database. I also created and designed the retail inventory dashboard and model using Power BI and supported their data warehouse and Power BI reports. This solution gave store owners and the head office instant insights into stock turnover, stock value, and other inventory metrics.
Tourism Australia | Data Warehouse and Dashboard
Developed the ETL pipelines to automate at least five data sources' extraction—including bookings, searches, flights, and visitor surveys—and orchestrated data processing from the source to the data warehouse. I also designed the Power BI data model and delivered several Power BI reports.
The solution provided instant insights to staff—from analysts to C-suite executives—and eliminated the immense amount of resources and time spent on manual reporting and data extraction.
The solution provided instant insights to staff—from analysts to C-suite executives—and eliminated the immense amount of resources and time spent on manual reporting and data extraction.
Charity Company | Text Analytics Dashboard
Developed a text analytics dashboard of employee surveys to show the answers' sentiment and quickly identify their different domains. I used Microsoft's cognitive Text Analytics API, Python, NLP, GloVe, machine learning, and Power BI.
The project aimed to quickly identify important answers from a large number of survey responses every year to uncover the issues in the business and identify opportunities to improve services.
The project aimed to quickly identify important answers from a large number of survey responses every year to uncover the issues in the business and identify opportunities to improve services.
Star Group | Real-time Big Data Architecture
Delivered a real-time big data architecture, which entailed developing a real-time big data IoT, machine learning, and advanced analytics platform.
This platform was built to support next-generation analytics, including processing structure and unstructured data, real-time data ingestion, analytics, and machine learning.
This platform was built to support next-generation analytics, including processing structure and unstructured data, real-time data ingestion, analytics, and machine learning.
Deloitte | Motor Industry Benchmark BI System
Designed and developed a motor industry benchmark BI system and several BI reports and dashboards for the eProfit system using Microsoft's reporting service, MDX, SSIS, and SQL Server.
Moreover, I tackled the challenging performance issue caused by previous design by reverse engineering and reinventing the "engine." The solution enabled extensive reports to be rolled out to the end users and has since been recognized and applauded by the CIO and partners.
Moreover, I tackled the challenging performance issue caused by previous design by reverse engineering and reinventing the "engine." The solution enabled extensive reports to be rolled out to the end users and has since been recognized and applauded by the CIO and partners.
Aussie Home Loan | Mortgage Sales DWH and Reports
Independently designed and developed the new data warehouse, cube, ETL, and some reports for the mortgage sales from scratch based on SQL Server, dimensional modeling, SSRS, SSAS, SSIS, MDX, and T-SQL. The reports were being used by sales managers and executives daily. In addition, I enhanced the cube to enable users to do self-servicing reports using Excel.
SaaS eCommerce Web Platform
Independently designed and developed a SaaS eCommerce web platform using Angular 8+, .NET Core, REST API, and Azure SQL Database. Users can register a store on this platform and the store has all the eCommerce features.
Revenue Management Consultancy Company | Data Integration
Delivered the data integration project, which involved developing an ADF pipeline, Azure Function, and C# Web API to integrate data—including invoice, sales, customer, product, and payment—from SAP IDoc XML, SFTP, CSV, and REST API to the Azure SQL Database. I also built pipelines to export the data for different clients in the retail FMCG industry.
Additionally, I created blob-triggered Azure Functions to transform XML data, including XSD validation and XSLT transformation, generating and sending output files to clients by API. I also designed the pipeline to ingest Excel and CSV files into the integration database, transforming and unpivoting the Excel data and exporting them to clients using ADF data flows.
Additionally, I created blob-triggered Azure Functions to transform XML data, including XSD validation and XSLT transformation, generating and sending output files to clients by API. I also designed the pipeline to ingest Excel and CSV files into the integration database, transforming and unpivoting the Excel data and exporting them to clients using ADF data flows.
Python IoT Project
Developed various Azure Functions as a Python IoT developer. Among these were Azure Functions that:
• Receive, validate, and process XML messages from IoT devices.
• Write and read data from Cosmos DB.
• Push messages to the Azure Storage Queue.
• Was triggered by Cosmos DB feed and Azure Storage Queue changes.
• Receive, validate, and process XML messages from IoT devices.
• Write and read data from Cosmos DB.
• Push messages to the Azure Storage Queue.
• Was triggered by Cosmos DB feed and Azure Storage Queue changes.
Dynamics 365 CRM Integration for Data Warehouse and Power BI
I delivered a Dynamics 365 data integration project for a client in the logistics industry using ETL, Data Factory, Dynamics 365, Azure SQL DB, BI modeling, and Power BI. I also developed the pipeline to ingest data from Dynamics 365 CRM Sales. Additionally, I created ETL stored procedures to populate dimension and fact tables. Finally, I developed the Power BI dashboard to display leads, opportunities, conversions, and more.
Data Engineering and Data Warehouse Using Fabric
I created the lakehouse and data warehouse in Fabric. I also built the Dataflow Gen2 to ingest data from an on-premise SQL server to the lakehouse in Fabric. Additionally, I developed stored procedures to load data into the data warehouse in Fabric. Finally, I created the safety dashboard using Power BI.
Dynamics 365 Finance and Operation Integration with Fabric Lakehouse
I built a notebook and pipeline to ingest Dynamics 365 Finance and Operations data to the Fabric Lakehouse's Bronze, Silver, and Gold layers. Additionally, I developed the notebook with PySpark to populate dimension and fact tables in the lakehouse.
Salesforce Integration with Fabric and OpenAI
I developed Dataflow Gen2 to ingest data from Salesforce to the Fabric Lakehouse. Additionally, I created a notebook to transform data from the Bronze Lakehouse to the Silver Lakehouse for structured data. I also developed a pipeline to ingest unstructured data from Azure Data Lake Storage Gen2 to the Fabric Lakehouse and provided the data for OpenAI.
Education
1999 - 2003
Bachelor's Degree in Computer Science
Shijiazhuang Tiedao University - Hebei, China
Certifications
APRIL 2018 - PRESENT
Neural Networks and Deep Learning
Coursera
JUNE 2017 - PRESENT
MCSE: Cloud Platform and Infrastructure — Certified 2017
Microsoft
Skills
Libraries/APIs
REST APIs, ODBC, Azure Blob Storage API, Pandas, Azure Cognitive Services, Azure API Management, Salesforce API, Stripe, Fabric, PySpark, NumPy, Cloud Key Management Service (KMS)
Tools
SQL Server BI, Microsoft Power BI, Microsoft Access, BigQuery, Microsoft Excel, Excel 2010, Synapse, Azure Logic Apps, Tableau, Azure App Service, GitHub, Jira, Google Sheets, Apache Airflow, Azure Machine Learning, Tableau Desktop Pro, Power Query, Google Analytics, Looker, Spark SQL, Power BI Desktop, Azure OpenAI Service
Languages
SQL, Snowflake, HTML, Stored Procedure, Transact-SQL (T-SQL), C#.NET, Python 3, Python, XML, Batch, JavaScript, Visual Basic for Applications (VBA), C#, TypeScript, XSLT, XSD, Excel VBA
Paradigms
Dimensional Modeling, Business Intelligence (BI), Database Design, ETL, OLAP, Automation, REST, Azure DevOps, Requirements Analysis, DevOps, B2C, B2B, Pay-per-Click (PPC)
Platforms
Databricks, Azure, Azure SQL Data Warehouse, Dedicated SQL Pool (formerly SQL DW), Microsoft Fabric, Azure Data Lake Storage, Azure Synapse, Azure Functions, Google Cloud Platform (GCP), Klaviyo, Azure Event Hubs, Apache Kafka, Firebase, Salesforce, Azure AI Studio, Oracle
Storage
SQL Server Integration Services (SSIS), SQL Server Reporting Services (SSRS), Azure SQL, Microsoft SQL Server, SQL Server Analysis Services (SSAS), Relational Databases, Databases, Data Pipelines, OLTP, Data Integration, Database Integration, Database Modeling, Database Migration, Database Lifecycle Management (DLM), SQL Stored Procedures, JSON, Azure Cloud Services, Database Performance, Azure Blobs, Azure SQL Databases, Data Lake Design, Data Lakes, Database Architecture, Azure Queue Storage, Google Cloud, Google Cloud Storage, Data Validation, Azure Active Directory, SQL Server 2016, PostgreSQL, Redshift, Azure Cosmos DB, Database Administration (DBA), SQL Server DBA, NoSQL, MySQL, PL/SQL, Amazon S3 (AWS S3), XML Schema
Frameworks
Angular, Spark, .NET, .NET Core, Angular Material, ASP.NET Web API, Apache Spark, Hadoop, Data Fabric, Data Lakehouse
Industry Expertise
Marketing
Other
Azure Data Factory (ADF), Data Modeling, Dashboards, Data Warehouse Design, Data Engineering, Google BigQuery, Data Warehousing, Data Migration, API Connectors, ETL Tools, Data Analytics, Data, Cloud Platforms, ELT, BI Reporting, SSRS Reports, Minimum Viable Product (MVP), Consolidation, Reporting, Data Cleansing, Data Quality Analysis, ETL Pipelines, Cloud, Entity Relationships, Performance Tuning, Data Extraction, Star Schema, Data Transformation, Integration, Query Optimization, Excel 365, Data Reporting, Data Processing Automation, Azure Analysis Services, Data Architecture, APIs, DAX, Data Visualization, Data Analysis, Webhooks, Cloud Migration, eCommerce, Financials, Big Data, Azure Databricks, Data Management, Data Feeds, Delta Lake, Architecture, Microsoft Azure Cloud Server, CI/CD Pipelines, Analytics, Customer Analytics, Retrieval-augmented Generation (RAG), Finance, Financial Modeling, Security, Metadata, Active Directory (AD), GitHub Actions, Azure Stream Analytics, GloVe, Natural Language Processing (NLP), Machine Learning, Azure Data Lake, Generative Pre-trained Transformers (GPT), SAP IDoc, Web Scraping, Data Scraping, Scientific Data Analysis, Excel Macros, Macros, SAP, Data Science, Google Data Studio, Dynamics CRM 365, Crypto, Web Crawlers, Scraping, Funnel Analysis, Streaming Data, Looker Studio, Artificial Intelligence (AI), Large Language Models (LLMs), Large-scale Data Migration
Collaboration That Works
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
1
Share your needs
Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2
Choose your talent
Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3
Start your risk-free talent trial
Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.
Top talent is in high demand.
Start hiring