Shahban Riaz

I built an enterprise data analytics platform for one of the leading telecom companies to curate, process, and store data from multiple corporate systems into an AWS data lake.

I was part of the team assigned to establish architectural patterns to process data from relational and non-relational sources and establish data governance frameworks, including data security, classification, ownership, discoverability, and consumption patterns.

During the later stages of the project, I worked on implementing the platform using AWS IAM, Glue, Athena, Lake Formation, ECS, and EMR, PySpark, Apache Airflow, and containerization technology.

I participated as the technical and design lead in a project that involved curating data from IoT devices in the electricity and gas network, geospatial networks, and relational systems. Then we modeled that data using Data Vault and stored it in Azure Data Lake for analysis. The information was stored in data lakes and warehouses using lakehouse architecture to support business intelligence and machine learning uses.

My responsibilities included creating detailed architectural and design documents for the data platform and reviewing design documents prepared by other team members. I also provided technical guidance to the team for building data pipelines and reviewed other engineers' work to ensure that best practices were followed.

I created a corporate data hub for a leading online employment marketplace that allows to curate data from various enterprise systems into an AWS-based data warehouse. This data warehouse was built using lakehouse architecture.

I developed and enhanced the pipeline scheduling, data curation, and processing framework using Apache Airflow and Spark; PostgreSQL; REST APIs; Delta Lake; AWS S3, Glue, Athena, and Lake Formation; CDC; and Type 2 SCD. I also built data marts for business consumption based on dimensional modeling.