
Salman Ahmed
Verified Expert in Engineering
Data Engineer and Developer
Karachi, Sindh, Pakistan
Toptal member since February 14, 2022
With 7 years of experience across GCP, BigQuery, Snowflake, and Databricks, Salman helps companies stop firefighting their data. He fixes spiraling cloud bills, slow Airflow jobs, and broken dashboards, then automates the manual work around them with modern pipelines and AI agents.
Portfolio
Experience
- Python - 4 years
- Data Engineering - 4 years
- Spark - 3 years
- ETL - 3 years
- BigQuery - 2 years
- Informatica ETL - 1 year
- Databricks - 1 year
- Google Cloud Platform (GCP) - 1 year
Work Experience
Big Data Engineer | Consultant
Freelance
- Wrote Spark, MapReduce, Hive, and Python scripts for distributed data processing.
- Extracted products from eCommerce store reviews and predicted their sentiments using Python, NLTK, spaCy, and Flair.
- Helped generate reports and articles related to big data, data engineering, and data science.
- Developed different Big Query ETLs, scheduled them using Airflow, and built some nice visualizations on Google Data Studio.
- Integrated several third-party sources with Big Query, including CRM, Asana, Bevy, etc., using stitch data, custom scripts, and APIs/webhooks.
- Optimized already running ETLs, which reduces the GCP cost and retrieval time by approximately 40-60%.
Data Engineer
Heirloom
- Integrated multiple 3rd-party sources into BigQuery for dashboarding and analysis.
- Developed multiple data pipelines to clean and transform data according to business needs.
- Set up alerting and monitoring for pipelines and infrastructure so that relevant stakeholders are notified whenever an issue arises.
- Designed workflows in Airflow to automate different processes.
- Speed up dashboards using techniques such as materialized views, partitions, and clustering.
- Upgraded and maintained Cloud SQL for MySQL instance for optimal performance.
- Developed MCP's for 3rd party platforms to help different teams (finance, ops) with their different use cases.
- Set up best practices across teams and optimized LLM's token usage.
- Developed data observability dashboards for infra and data pipelines.
Data Engineer
Turing
- Led a POD training and evaluated AI agents across multiple domains, building advanced reasoning agents on ReAct and plan-and-execute frameworks.
- Delivered domain-specific SFT and RLHF datasets for Google, generating multi-turn conversations to train LLM agents.
- Migrated data workflows from GitHub Actions to Airflow with dbt, and built data pipelines.
- Optimized existing ETLs and warehouse models, cutting GCP cost and query retrieval time by 30–40%.
- Optimized mode analytics jobs to reduce data scanned and eliminate overbilling.
- Built sources of truth (SOTs) that improved data accuracy and consistency across dashboards and metrics, removing data silos.
- Integrated third-party sources into the BigQuery warehouse and served warehouse data to products via APIs, automating manual internal processes.
Data Engineer
Sanpei Ventures
- Collaborated with a team to build pipelines that collect, clean, and analyze different eCommerce brands' data.
- Managed the Airflow server for optimal performance and set up a proper process to synchronize it with the GitHub repository.
- Optimized an already-running ETL with reduced network and CPU usage.
- Normalized and converted NoSQL (JSON) data to relational forms.
- Created a process to automate the regex-based pattern matching for products from their warehouse.
Data Engineer
Amigoals
- Wrote multiple scrapers and parsers to fetch and transform data into its best form.
- Set up data pipelines on Databricks to scrape different public sites and clean and ingest data into the warehouse.
- Wrote transformation pipelines on Spark and Spark SQL to clean and transform data for further analysis.
Big Data Consultant
Blutech Consulting
- Developed and automated fault-tolerant ETL pipelines in multiple banking streams to load data into a dimension layer that is then provided to the BI team and relevant departments for report generation and decision-making.
- Worked with clients to translate business problems into quantitative queries and collect/clean the necessary data.
- Optimized Informatica/SSIS workflows and improved the performance of ETL pipelines that process GBs of data daily.
- Developed non-skewed data marts on Teradata, where daily ETLs can be run easily.
- Built an app called Data Robot to search for accounts and entities in a data lake using fuzzy search, which helps multiple departments in their ongoing operations. The manual process that preceded this product used to take weeks.
- Automated regulatory returns through a data lake, using big data and reporting tools, which helps in making the process fast, dynamic, and scheduled instead of doing it manually in Excel.
- Built an app called ScoreCard to automate Spark reports for anomalous transactions, accounts, and agents. It helped the bank score their app agents to reward and improve their services.
- Developed processes to check for ETL anomalous behavior and alerting capabilities to ensure the data is loaded correctly and on time.
Project History
CourseThread
FoodCase
Movie Review Sentiment Analysis (Kernels Only)
https://www.kaggle.com/code/pantherpanther/simple-1d-cnn-with-glove-twitter-embeddings?scriptVersionId=10076590Gift Assistant
Assessments for Udemy Business Pro
Education
Bachelor's Degree in Computer Science
National University of Computer and Emerging Sciences (FAST) - Pakistan
Certifications
Databricks Certified Data Engineer Associate
Databricks
Professional Data Engineer
Google Cloud
Skills
Libraries/APIs
PySpark, Keras, Pandas, Matplotlib, Scikit-learn
Tools
Impala, BigQuery, Apache Airflow, GitHub, Informatica ETL, Stitch Data, Looker, Amazon Elastic MapReduce (EMR), Jupyter, Slack, Microsoft Power BI, Tableau, MySQL Workbench, Microsoft Excel, Confluence, Jenkins, Google Analytics, AWS Glue
Languages
Python, SQL, SQL DML, Transact-SQL (T-SQL), JavaScript, Snowflake
Paradigms
Database Development, ETL, Automation, Business Intelligence (BI)
Platforms
Jupyter Notebook, Databricks, Google Cloud Platform (GCP), AWS Lambda, Oracle, Windows, Linux, Docker, Kubernetes, Azure, Amazon Web Services (AWS), Azure SQL Data Warehouse, Amazon EC2, Apache Kafka, Dedicated SQL Pool (formerly SQL DW)
Storage
Apache Hive, Data Pipelines, JSON, Databases, Relational Databases, RDBMS, MySQL, Teradata, Google Cloud Storage, PostgreSQL, MariaDB, SQL Performance, Database Administration (DBA), SQL Server Analysis Services (SSAS), Data Lakes, Redshift, SQL Server Integration Services (SSIS), Microsoft SQL Server, SQL Server Reporting Services (SSRS), Alibaba Cloud, Azure SQL Databases, Azure SQL, SSAS Tabular, Amazon S3 (AWS S3), Google Cloud SQL
Frameworks
Spark, Hadoop, Apache Spark, Flask, Django, Selenium
Other
Data Engineering, ETL Tools, Data Queries, ELT, Schemas, Data Transformation, CSV, CSV File Processing, Big Data, Data Warehousing, Informatica, Data Analysis, Natural Language Processing (NLP), Analytics, Data Analytics, Data Modeling, Query Optimization, Data Warehouse Design, API Integration, Integration, Performance Tuning, Automated Data Flows, Data Architecture, Change Data Capture, Data Processing, Relational Data Mapping, Google BigQuery, Amazon RDS, Consulting, Scaling, Reporting, CRM APIs, Generative Pre-trained Transformers (GPT), Data Science, Visualization, Machine Learning, Scraping, APIs, Deep Learning, Data Visualization, Delta Lake, Business Intelligence (BI) Platforms, CI/CD Pipelines, Google Data Studio, Reports, Pub/Sub, Dashboards, Google Search Console, Artificial Intelligence (AI), Google Cloud Functions, Data Build Tool (dbt), Mode Analytics, LLM Reasoning, Supervised Fine-tuning (SFT), Reinforcement Learning from Human Feedback (RLHF), GitHub Actions, Data Quality
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring