Muhammad Bilal

Served as the lead ETL developer in a data modernization program for a Gulf Cooperation Council's leading telecom company to transform their legacy data warehouse into a modern data platform and enable the organization to access a single version of the truth.

The project deliverables included:
• Migration of existing data from a legacy data warehouse platform to an enterprise data warehouse, including data pipelines developed on the IBM InfoSphere DataStage platform.
• Understanding and documentation of the existing data architecture.
• Transformation of existing data architecture to modern data architecture, including new business requirements.
• Design documentation followed by business reviews and a sign-off.
• ETL development guided by the design document and followed by tech and business user acceptance testing.
• Deployment with handover training sessions to AMS and operations teams.
• Technical training on new data models for business.
• Data quality framework development and implementation to ensure data quality concerning the enterprise data warehouse.

Developed an advanced data analytics platform to identify fraudulent suspicious activities in the telecom network. While working on the project, I contributed to optimizing company processes, prevented fraud and daily losses totaling over 1 million Pakistani rupees, and identified fraudulent transactions saving the company millions of Pakistani rupees.

Contributed to a project for China Mobile, Pakistan's second-largest telecom operator, with 70 terabytes of legacy data warehouse in Oracle Database 11g. They had over 1 billion call detail records in data sessions, calls, text messages, recharges, unstructured supplementary service data, and value-added service from 25 million active subscribers, which were processed in daily batches.

I led the optimization of the ETL batch and saved two hours. We started with the implementation of logging for each job to find out which one was taking the longest.

To optimize query executions, we performed these actions:
• Partitioning and subpartitioning of tables.
• Creating proper local and global indexes.
• Gathering stats.
• Applying data retention.
• Rewriting queries.
• Developing materialized views for the business.

Radix is an online, cloud-based multifamily market research and analytics platform with built-in tools designed to help users make sense of ambiguous data and drive performance. My role was to migrate ETL logic from ADF pipelines to Azure Databricks Notebooks.

• Develop pipelines in Azure Databricks using PySpark notebooks to ingest data from multiple sources, including Azure Eventhub, AWS Containers, SFTP, and MongoDB.
• Save data as parquet files in the enterprise landing layer, i.e., Azure Data Lake Storage
• Transform data and sink it into a dimensional model implemented in Azure Synapse Analytics

I migrated the data from the Azure SQL data warehouse to Postgre DB. I developed PySpark notebooks on Azure Databaricks to migrate data from Azure SQL Server to Postgre DB. I also developed a reconciliation script to reconcile data between both databases.

I was part of China Mobile (Pakistan) when Zong acquired a 3G/4G license. I implemented a data model that was not rich enough to produce BI reports as per company requirements post-3G/4 G network rollout.

I led a team of data engineers to implement changes in the data model by introducing new facts and dimensions, followed by changes in ETL pipelines to sink data into a new model. We also develop new ETL pipelines to ingest 3G/4G network equipment data sources. BI reports were also created per company requirements to report 3G/4 G-related business KPIs.

This assignment was for Vodafone Qatar EDW Operations. The business used to report and lock several data-related issues related to data quality. The ops team spent 90% of their time debugging/RCA and fixing these issues by applying ad-hoc data fixes.

As the lead data engineer, I analyzed why so many issues are being reported daily.

We learned that 70% of issues are related to source data and can be avoided if data quality checks are implemented on the landing layer. So we implemented around 150 reviews on the landing layer to check the quality of source data with acceptance threshold to variation in trend.

Since implementation, the ops team and business team have been at peace.

Phillips Morris International maintaining the EDW application in Snowflake with ETL in WhereScape RED. Data vault models were developed in WhereScape 3D.

I was responsible for the architectural enhancement of all existing data products. The main supporting areas were the Sales and Digital Sustainability Program for any architectural enhancement and modifications.

My contribution:
• Data vault 2.0 design approach for integrated data platform of Data Ocean.
• WhereScape 3D for data vault designing.
• Standardized the implementation framework for data pipelines.
• The delivery of trusted data products to enable data-driven decision-making for the business.