Amir Rauf
Verified Expert in Engineering
Data Engineer and Developer
Round Rock, TX, United States
Toptal member since February 27, 2026
Amir is a senior data engineering and cloud architecture expert with 20+ years of experience building enterprise-scale systems. He specializes in Azure (Data Factory, Databricks, ADX), C3.ai, Data Build Tool (dbt), Sqitch, and AWS, designing scalable data platforms that power advanced analytics and AI. Amir combines deep technical expertise with business-focused execution to deliver reliable, high-impact solutions.
Portfolio
Experience
- SQL - 20 years
- Matplotlib - 5 years
- Python - 5 years
- Python 3 - 5 years
- Pandas - 5 years
- NumPy - 5 years
- Azure - 3 years
- C3.ai - 3 years
Preferred Environment
Azure Data Factory (ADF), Azure Databricks, ADX, Azure Synapse, C3.ai, Sqitch, Data Build Tool (dbt), Amazon Web Services (AWS), Azure
The most amazing...
...project I've led involved architecting a cloud-native data platform that unified enterprise data and enabled real-time AI-driven insights.
Work Experience
Senior Data Engineer
LyondellBasell
- Integrated, persisted, and normalized datasets within the C3.ai platform, facilitating robust data-driven insights.
- Built and deployed custom metrics and machine learning models in C3.ai, driving advanced analytics capabilities.
- Developed and maintained data pipelines in Azure Data Factory (ADF), enabling seamless data ingestion and transformation processes.
- Designed and implemented a version-controlled database deployment system using Sqitch, reducing deployment errors and improving rollback efficiency by 30%.
- Built modular, testable data models using dbt, enhancing analysis speed and reducing pipeline errors by 25%.
- Established a robust CI/CD environment in GitHub using Git and ARM templates, automating the deployment of Azure Data Factory (ADF) across multiple environments.
- Collaborated with data scientists, software engineers, and business SMEs to develop critical data pipelines and workflows for data-driven decision-making.
- Partnered with IT teams to establish architecture, enforce security and coding standards, and implement rigorous exception handling and testing protocols.
Senior Data Integration Architect
U.S. Citizenship and Immigration Services
- Utilized Python 3 for data manipulation, wrangling, and cleanup, ensuring datasets were optimized for analysis.
- Re-architected and upgraded legacy Jakarta EE applications to a microservices architecture using the Spring framework, enhancing scalability and maintainability.
- Applied Python libraries like NumPy, Pandas, and Scikit-learn for data analysis, manipulation, and machine learning tasks.
- Designed a CI/CD environment with Jenkins, automating build, testing, and deployment processes.
- Independently architected an Apache Kafka client application to integrate Apache Kafka with the TIBCO EMS server for real-time data processing.
- Designed TIBCO-based integration solutions using an SOA framework with an ESB model for seamless application communication.
- Participated in migrating applications from the DHS datacenter to AWS, ensuring smooth transition and minimal downtime.
- Designed and implemented cloud applications using AWS services like S3, EC2, IAM, RDS, SNS, and SQS.
Experience
NBFC Loan Default (Hackathon Project)
I was provided a sample containing data on 90,000+ clients who had taken a loan from the company in the past, along with information on whether the loan defaulted. The challenge was to create a model with the highest accuracy to predict whether a client would default on a loan payment. My model scored 91% accuracy.
Education
Postgraduate Diploma in Data Science and Business Analytics
University of Texas - Austin, TX, USA
Bachelor's Degree in Computer Science
Minnesota State University - Mankato, MN, USA
Certifications
Certified C3.ai Developer
C3.ai Academy
C3.ai Data Integrator
C3.ai Academy
Skills
Libraries/APIs
Pandas, NumPy, Matplotlib
Tools
Sqitch, StatsModels, Git
Languages
Python, Python 3, SQL, Java
Platforms
Azure, Azure Synapse, Amazon Web Services (AWS)
Frameworks
Spring 5
Other
Data Modeling, ETL Pipelines, C3.ai, Data Build Tool (dbt), TIBCO, Software Development, Data Science, Business Analytics, Web Services, Machine Learning, Ensemble Methods, Model Tuning, Azure Data Factory (ADF), Azure Databricks, ADX, Computer Science
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring