Abdul is available for hire
Hire AbdulAbdul Rafey Tahir
Verified Expert in Engineering
Research Engineer and Developer
Lahore, Punjab, Pakistan
Toptal member since July 24, 2022
Bio
Abdul Rafey is a data scientist with 5+ years of industry experience. He has worked on challenging AI, big data, and LLM use cases in eCommerce, healthcare, finance, insurance, and safety and compliance domains. He is proficient in Python and relevant DS libraries like Pandas, NumPy, scikit-learn, big data frameworks like Apache Spark and Hadoop, and LLM integration and fine-tuning with OpenAI, LangChain, and Hugging Face.
Portfolio
Mazars USA LLP - Main
Machine Learning, Python, Artificial Intelligence (AI)...
QPharma, Inc.
Amazon Web Services (AWS), Big Data, Scala, Python, Graph Theory...
Ponte Energy Partners GmbH
Amazon SageMaker, AWS CloudFormation, Machine Learning Operations (MLOps)...
Experience
- SQL - 5 years
- Machine Learning - 5 years
- Python - 5 years
- Data Science - 5 years
- Amazon Web Services (AWS) - 3 years
- PySpark - 2 years
- Kubeflow - 2 years
Availability
Part-time
Preferred Environment
Data Science, Machine Learning, Big Data, Python, Amazon Web Services (AWS), Deep Learning, Large Language Models (LLMs), LangChain, OpenAI, Kubeflow
The most amazing...
...thing I've built is a real-time collision detection system using sensor data for Motive, Inc., a multi-billion dollar startup in Silicon Valley.
Work Experience
Data/AI Expert
2024 - 2024
Mazars USA LLP - Main
- Developed a data pipeline in Azure DevOps to ingest and categorize financial documents using a large language model and move them to relevant SharePoint folders to organize workflow for internal users like auditors and tax consultants.
- Implemented a RAG-based approach to categorize documents. I used the OpenAI text-embedding-3-small model to generate embeddings for a large dataset of financial documents. A vector store with embeddings and document categories was built in Cosmos DB.
- Built a FastAPI to process every new file uploaded to SharePoint by the users. The API would generate embeddings for the file, retrieve the top 7 results from the vector store, and use popular voting to return the document category.
Technologies: Machine Learning, Python, Artificial Intelligence (AI), Large Language Models (LLMs), Azure ML Studio, OpenAI, Supervised Machine Learning, Model Tuning, Serverless, LangChain, Generative Pre-trained Transformer 3 (GPT-3), Generative Pre-trained Transformer 4 (GPT-4)
Senior Data Scientist
2022 - 2024
QPharma, Inc.
- Developed an LLM chatbot using a LangChain SQL agent with dynamic few-shot prompting to answer user queries from a database of top healthcare professionals in their respective fields. It was used by medical and sales reps of pharma clients.
- Built a big data analytics pipeline to identify healthcare professional leaders at local and national levels based on referrals and prescription data. These are provided to pharma clients for maximum market penetration of new and existing brands.
- Architected social media scrapers for Twitter and YouTube to gauge the social media influence of HCPs. This data is preprocessed and fed to a new analytics pipeline that identifies key opinion leaders in specific areas of medicine for pharma clients.
Technologies: Amazon Web Services (AWS), Big Data, Scala, Python, Graph Theory, Artificial Intelligence (AI), PySpark, Machine Learning Automation, Amazon Machine Learning, Machine Learning Operations (MLOps), SQL, Git, Large Data Sets, Data Scientist, Data Gathering, Data Scraping, Web Scraping, AWS CloudFormation, Identity & Access Management (IAM), AWS CodeBuild, Amazon EC2, Amazon S3 (AWS S3), Sentiment Analysis, Agile Data Science, MySQL, NumPy, Pandas, ChatGPT, Large Language Models (LLMs), DevOps, Docker, CI/CD Pipelines, Data Versioning, ETL Tools, CSV, APIs, Data Modeling, Predictive Analytics, Data Matching, Amazon, Analytics, Optimization, Exploratory Data Analysis, Selenium, Chatbots, OpenAI GPT-3 API, TensorFlow, Python API, LangChain, Supervised Machine Learning, MongoDB, AWS Lambda, Model Tuning, Kubeflow, LlamaIndex, Generative Pre-trained Transformer 3 (GPT-3)
ML Engineer
2023 - 2023
Ponte Energy Partners GmbH
- Developed an AWS Sagemaker pipeline to support training, processing, batch transformation, and inference functionality for ML models that predict price variations on the company's renewable energy trading platform.
- Restructured a large portion of the codebase, set up debug configs for local model execution, and optimized CI/CD scripts and several functionalities for efficient data loading and processing, including the use of Manifest files and property files.
- Utilized a bunch of new tools like Typer for efficient parsing of CLI args and contextlib for building the dependency wheel as a background process while executing the pipeline.
Technologies: Amazon SageMaker, AWS CloudFormation, Machine Learning Operations (MLOps), Machine Learning, Identity & Access Management (IAM), AWS CodeBuild, AWS CodePipeline, Amazon SageMaker Pipelines, Data Versioning, ETL Tools, CSV, Containerization, Predictive Analytics, Amazon, High-frequency Trading (HFT), Optimization, AI Model Training, Bots, Stock Market, TensorFlow, Python API, Trading Bots, Supervised Machine Learning, Serverless, Amazon Bedrock, Kubeflow
Data Scientist
2023 - 2023
Neyl Skalli
- Developed a web scrapper for Transfermarkt.com to scrape data for soccer players. Successfully built and deployed it on AWS Glue to scrape data for 2,000+ teams (more than 60,000 players).
- Utilized scraped data to train an unsupervised machine learning model, specifically K-Medoid clustering, enabling effective grouping of players based on their statistics, rankings, and valuation.
- Played a key role in integrating the trained model into the client's platform, allowing users to receive the top 5 most similar players based on their search queries.
Technologies: Python, Web Scraping, Data Scraping, Unstructured Data Analysis, Data Scientist, Data Gathering, Amazon EC2, Amazon S3 (AWS S3), NumPy, Pandas, CSV, Analytics, AI Model Training, Selenium, Python API
Data Scientist
2020 - 2022
Motive
- Developed an unsafe driving detection algorithm for Motive, a US-based multibillion-dollar startup. It detects unsafe acceleration, brake, and corner events generated by customer fleets' truck drivers using sensor data used for driver coaching.
- Trained a real-time crash detection ML model with huge volumes of sensor data for Motive's safety product. The system saves event and video data, notifies authorities in minutes, and helps save lives, exonerate drivers, and reduce insurance liability.
- Built a smoothing algorithm in collaboration with the embedded team at Motive to improve the quality of raw sensor data from the electronic logging device in the customers' vehicles, improving the system's precision in catching hard events by 40%.
Technologies: Python, Big Data, Machine Learning, SQL, PySpark, Ruby, Snowflake, Redash, Amazon Web Services (AWS), Microsoft Excel, Spark SQL, Data Queries, Tableau, Data Visualization, Data Science, Statistics, Predictive Modeling, Predictive Learning, ETL, Signal Processing, Artificial Intelligence (AI), Data Reporting, Data Analytics, Data Engineering, Cloud, Jupyter Notebook, XGBoost, Data Cleaning, AI Design, Automation, Task Automation, AWS Fargate, Graphs, Classification, Data Pipelines, Jupyter, Amazon SageMaker, Machine Learning Automation, Machine Learning Operations (MLOps), Git, Docker, Large Data Sets, Unstructured Data Analysis, Data Scientist, Data Gathering, Amazon SageMaker Pipelines, AWS CodePipeline, Computer Vision, Amazon EC2, Amazon S3 (AWS S3), APIs, PostgreSQL, Neural Networks, Agile Data Science, MySQL, NumPy, Pandas, Generative Artificial Intelligence (GenAI), DevOps, CI/CD Pipelines, Data Versioning, CSV, FastAPI, Containerization, PyTorch, Data Modeling, Predictive Analytics, Trend Analysis, Amazon, Logistic Regression, A/B Testing, Analytics, Optimization, Exploratory Data Analysis, AI Model Training, H2O AutoML, TensorFlow, Python API, Databricks, Supervised Machine Learning, AWS Lambda, Model Tuning, Kubeflow
Data Scientist
2019 - 2020
Foot Locker
- Built a machine learning model to predict customer tier change (upgrade and downgrade) in the company's loyalty program in the next quarter based on data from the last three quarters to offer rewards as part of the customer retention policy.
- Performed RFM (recency, frequency, and monetary value) analysis for Foot Locker customers to segment more frequent and high-spending customers from others. The purpose was to lay the groundwork for a personalized recommendation system.
- Was part of the Churn prediction project at Foot Locker. Like the loyalty program, the company wanted to determine which customers would churn. Based on data from the previous three quarters, the criteria were set to no spending in one quarter.
Technologies: Databricks, Python, PySpark, Machine Learning, Data Analysis, Data Visualization, Forecasting, Data Science, Statistics, Predictive Modeling, Predictive Learning, ETL, Data Analytics, SQL Server 2016, Cloud, Jupyter Notebook, Amazon SageMaker, Data Cleaning, API Integration, Automation, Graphs, Classification, Text Classification, Financial Modeling, Jupyter, Git, Data Scientist, eCommerce, Sentiment Analysis, Agile Data Science, NumPy, Pandas, CSV, Time Series, ARIMA, Retail & Wholesale, ARIMAX, Data Modeling, Predictive Analytics, Regression Modeling, Marketing Mix Modeling, Trend Analysis, Digital Marketing, Bayesian Statistics, Logistic Regression, A/B Testing, Exploratory Data Analysis, AI Model Training, Trading Bots, Supervised Machine Learning, SQL
Data Scientist
2019 - 2019
CUNA Mutual Group
- Developed a machine learning model to predict which insurance advisors would not be able to sell a product in the following 12 months based on three years of historical data or sales. Trained and deployed the model on the Azure cloud.
- Built a model to forecast which credit unions the company did business with would be able to survive in the next two years after COVID-19 hit based on historical data going as far back as 1990.
- Developed an algorithm following a weighted average metric model to score the performance of insurance advisors based on their performance in the last four quarters to identify top, medium, and low-performing advisors.
Technologies: Python, Data Analysis, Machine Learning, Time Series Analysis, Microsoft Excel, Data Queries, Statistical Modeling, Azure, Data Science, Statistics, Predictive Modeling, Predictive Learning, ETL, Data Reporting, Data Analytics, SQL Server 2016, Statistical Analysis, R, Google Cloud Platform (GCP), Cloud, Risk Analysis, Jupyter Notebook, XGBoost, Agent-based Modeling, Data Cleaning, AI Design, API Integration, Automation, Task Automation, Graphs, Classification, Text Classification, Financial Modeling, Jupyter, Git, Unstructured Data Analysis, Data Scientist, Data Scraping, Web Scraping, PostgreSQL, Recurrent Neural Networks (RNNs), Neural Networks, Sentiment Analysis, Agile Data Science, MySQL, NumPy, Pandas, Stock Trading, CSV, Excel 365, Time Series, ARIMA, ARIMAX, Data Modeling, Predictive Analytics, Regression Modeling, Marketing Mix Modeling, Finance, Trend Analysis, Bayesian Statistics, Actuarial, Logistic Regression, A/B Testing, Analytics, Optimization, Exploratory Data Analysis, AI Model Training, TensorFlow, Supervised Machine Learning
Research Associate
2018 - 2018
National University of Computer and Emerging Sciences
- Involved in the full-year project that researched and developed road anomaly detection, i.e., potholes, manholes, speed breakers, cat-eyes, and rumble strips. Smartphone sensors were used for data collection through hours of drives across the city.
- Used the model trained in research to build a crowd-sourced application to map road anomalies across the cities for users to avoid using routes with high anomalies. The model was retrained over time to improve the prediction of road anomalies.
- Published a research paper in the 2018 IEEE Intelligent Vehicle Symposium (IV) Conference titled "Intelligent Crowd Sourced Road Anomaly Detection System."
Technologies: Research, Artificial Intelligence (AI), Machine Learning, Data Analysis, Computer Vision, TensorFlow, Sensor Fusion, Data Science, Predictive Modeling, Predictive Learning, Jupyter Notebook, Data Cleaning, Graphs, Classification, Jupyter, Optical Character Recognition (OCR), CSV, PyTorch, R, AI Model Training
Experience
Forecast Customer Loyalty Status for Foot Locker USA Loyalty Program
Foot Locker USA has an extensive loyalty program that rewards customers based on their spending. They have three customer classes based on heuristics:
• X1: VIP customers
• X2: average customers (with reasonable spending)
• X3: low-spending customers
The project involved building a machine learning model to predict which customers would and wouldn't change their category in the next quarter based on data from the past eight quarters.
Using pandas, I cleaned and prepared the dataset from 2019 Q1 to 2020 Q4 for feature engineering. The target variable class_change came from 2021 Q1 data. It was set to 1 for customers whose classes had changed during that time (either upgraded or downgraded) and 0 for those who hadn't. The test set target variable came from 2021 Q2. After generating quarterly features from data—including the number of website visits, orders placed, items checked out, items viewed, the amount spent, and other session data and shopping history—a random forest classifier was trained using scikit-learn. The model did fairly well on the test set, with a recall of 0.62 and a precision of 0.87. It was then deployed on Azure.
• X1: VIP customers
• X2: average customers (with reasonable spending)
• X3: low-spending customers
The project involved building a machine learning model to predict which customers would and wouldn't change their category in the next quarter based on data from the past eight quarters.
Using pandas, I cleaned and prepared the dataset from 2019 Q1 to 2020 Q4 for feature engineering. The target variable class_change came from 2021 Q1 data. It was set to 1 for customers whose classes had changed during that time (either upgraded or downgraded) and 0 for those who hadn't. The test set target variable came from 2021 Q2. After generating quarterly features from data—including the number of website visits, orders placed, items checked out, items viewed, the amount spent, and other session data and shopping history—a random forest classifier was trained using scikit-learn. The model did fairly well on the test set, with a recall of 0.62 and a precision of 0.87. It was then deployed on Azure.
YouTube Comment Classification for Content Creators
https://github.com/abdulrafeytahir/Youtube-Comment-ClassificationThe idea was to create an application for YouTube content creators to filter out requests for new content from their viewers in the comment section. This would help them create content that the audience demanded to watch.
There project involved:
• Data collection. I developed a scraper using Python and Selenium, crawled many channels, and collected the top 100 comments on every video. The clients had a team of annotators who annotated about 100,000 comments, of which there were roughly 10,000 requested comments.
• Model training. The dataset was quite imbalanced, so I downsampled negative class data points to 30,000. I then applied basic text preprocessing techniques using Pandas and NLTK, such as special character removal, lowercase alphabets, text tokenization, and stemming. Also, I generated simple frequency-based features using TF-IDF vectorization in scikit-learn and trained a support vector machine (SVM) model. It achieved 83% recall and 91% precision on the validation set.
• Inference and optimization. I used my scraper to scrape a few more websites and ran an inference for the scraped comments. Since the goal was high precision, we got the events classified as requests, annotated them again, and retrained the model.
There project involved:
• Data collection. I developed a scraper using Python and Selenium, crawled many channels, and collected the top 100 comments on every video. The clients had a team of annotators who annotated about 100,000 comments, of which there were roughly 10,000 requested comments.
• Model training. The dataset was quite imbalanced, so I downsampled negative class data points to 30,000. I then applied basic text preprocessing techniques using Pandas and NLTK, such as special character removal, lowercase alphabets, text tokenization, and stemming. Also, I generated simple frequency-based features using TF-IDF vectorization in scikit-learn and trained a support vector machine (SVM) model. It achieved 83% recall and 91% precision on the validation set.
• Inference and optimization. I used my scraper to scrape a few more websites and ran an inference for the scraped comments. Since the goal was high precision, we got the events classified as requests, annotated them again, and retrained the model.
LLM-based Meeting Minutes Generation
My team and I developed an LLM-based meeting minutes generation model, which was trained on 1,000 meeting videos scraped from YouTube. The first part involved audio/video-to-text transcription, for which we used Meta's Whisper model. Once the transcript was generated, we used Llama 2 to create a summary of the transcript and then used the summary to generate meeting minutes in the form of bullet points and a flow chart.
We used GCP for training and a Tesla A100 for running training jobs. For inference, we used P100 GPU, which worked pretty well for the scope of our application. We also built a Flask app that is containerized and deployed on GCP. The models are deployed as API endpoints and consumed by the app.
We used GCP for training and a Tesla A100 for running training jobs. For inference, we used P100 GPU, which worked pretty well for the scope of our application. We also built a Flask app that is containerized and deployed on GCP. The models are deployed as API endpoints and consumed by the app.
Recommendation System and Churn Reporting for an eCommerce System
Used "eCommerce Behaviour Data from Multi-category Store" with 285 million users. Performed RFMV analysis on the data to segregate more frequent and high-spending customers from others. I then used the LightFM library to train a deep learning recommendation model based on purchased data. Trained an SRGNN deep learning model from the RecBole library on session data. Combined the predictive capability of the two models for a better recommendation. In addition to the recommendation system, the focus was on churn reporting, especially for more frequent and high-spending customers. The goal was to retain these customers by suggesting personalized marketing strategies.
MedBot - An LLM-based Medical Chatbot
The project involved building a medical chatbot using state-of-the-art LLM models to answer user queries related to different diseases. The bot would ask relevant follow-up questions and then provide the final answer using only its own knowledge base and provide references as well. For this purpose, I implemented a RAG-based AI solution using open-source LLMs. I had a set of 20 medical PDFs that were parsed using a Python script. Each page was converted to a text embedding using the All-Mini-LM-L6-V2 model from Hugging Face. Then, a vector store was set up in Pinecone to store all embeddings. For each user query, the retrieval pipeline would generate embeddings and use MMR to get a diverse set of top-5 matches from the vector store to be passed into the prompt. Then, the TinyLlama model was used to generate a follow-up response or question. The model maintained chat history, and the retrieval pipeline used that as context to fetch relevant information. Ultimately, the model would provide answers rooted only in the knowledge source, along with references to documents and page numbers. The solution was then deployed as a Streamlit app.
Education
2020 - 2022
Master's Degree in Data Science
National University of Computer and Emerging Sciences - Lahore, Pakistan
2014 - 2018
Bachelor's Degree in Computer Science
National University of Computer and Emerging Sciences - Lahore, Pakistan
Certifications
MARCH 2019 - PRESENT
Neural Networks and Deep Learning
Coursera
Skills
Libraries/APIs
Pandas, NumPy, Scikit-learn, PySpark, Matplotlib, TensorFlow, Python API, XGBoost, LSTM, PyTorch, Node.js, Keras
Tools
Git, Microsoft Excel, Spark SQL, Amazon SageMaker, Jupyter, ARIMA, ARIMAX, H2O AutoML, AWS CloudFormation, ChatGPT, Redash, Tableau, AWS Fargate, AWS CodeBuild, Azure ML Studio
Languages
Python, SQL, R, Snowflake, Java, C++, Ruby, C, Scala
Frameworks
Selenium, LlamaIndex, Streamlit
Paradigms
ETL, Automation, Agent-based Modeling, DevOps, Business Intelligence (BI)
Platforms
Amazon Web Services (AWS), Databricks, Jupyter Notebook, Amazon EC2, Amazon, AWS Lambda, Kubeflow, Google Cloud Platform (GCP), Docker, Azure
Storage
SQL Server 2016, Amazon S3 (AWS S3), PostgreSQL, MySQL, MongoDB, Data Pipelines, Databases
Industry Expertise
Retail & Wholesale, High-frequency Trading (HFT)
Other
Data Science, Machine Learning, Data Analysis, Algorithms, Big Data, Natural Language Processing (NLP), Artificial Intelligence (AI), Data Scraping, Time Series Analysis, Data Queries, Web Scraping, Data Visualization, Computer Vision, Statistics, Predictive Modeling, Predictive Learning, Data Reporting, Data Analytics, Data Engineering, Statistical Analysis, Data Cleaning, AI Design, API Integration, Task Automation, Graphs, Classification, Financial Modeling, Machine Learning Operations (MLOps), Large Data Sets, Unstructured Data Analysis, Data Scientist, Data Gathering, APIs, Recurrent Neural Networks (RNNs), Neural Networks, eCommerce, Sentiment Analysis, Agile Data Science, Stock Trading, Large Language Models (LLMs), Data Versioning, CSV, Time Series, Data Modeling, Predictive Analytics, Regression Modeling, Marketing Mix Modeling, Finance, Trend Analysis, Data Matching, OpenAI GPT-4 API, Digital Marketing, Bayesian Statistics, Actuarial, Logistic Regression, A/B Testing, Analytics, Optimization, Exploratory Data Analysis, AI Model Training, Chatbots, OpenAI GPT-3 API, Bots, Stock Market, Trading Bots, LangChain, Supervised Machine Learning, Model Tuning, Serverless, Generative Pre-trained Transformer 3 (GPT-3), Generative Pre-trained Transformer 4 (GPT-4), Statistical Modeling, Cloud, Risk Analysis, Generative Pre-trained Transformers (GPT), Text Classification, Machine Learning Automation, Amazon Machine Learning, Amazon SageMaker Pipelines, Optical Character Recognition (OCR), Generative Artificial Intelligence (GenAI), CI/CD Pipelines, BERT, ETL Tools, Excel 365, FastAPI, Containerization, PDF Scraping, Amazon Bedrock, Data Structures, Deep Learning, Forecasting, Churn Analysis, Recommendation Systems, Sensor Fusion, Signal Processing, Research, Language Models, Graph Theory, Identity & Access Management (IAM), AWS CodePipeline, Hugging Face, Llama 2, Text Summarization, OpenAI, Retrieval-augmented Generation (RAG)
Collaboration That Works
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
1
Share your needs
Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2
Choose your talent
Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3
Start your risk-free talent trial
Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.
Top talent is in high demand.
Start hiring