Rishabh Agarwal
Verified Expert in Engineering
Data Engineer and Developer
Rishabh is a goal-oriented professional with over three years of experience in data engineering and analysis. He is recognized as a valuable team member and a successful leader, being a fast learner who provides optimized solutions to technology and business problems. Rishabh has hands-on experience working with AWS, SQL, Python, and more. He is motivated by working on new technologies and challenges and is eager to take on growth-driven and challenging roles and projects.
Portfolio
Experience
Availability
Preferred Environment
Amazon Web Services (AWS), SQL, Python, Shell, Microsoft Power BI, ETL, Microsoft Excel, Technical Project Management, Redshift, Scripting
The most amazing...
...result I've achieved is providing successful training in SQL, Microsoft Excel, and Power BI, resulting in interns working independently within three months.
Work Experience
Business Technology Solutions Associate Consultant
ZS Associates
- Led four teams of 3–5 associates as a data engineering, analysis, and business intelligence (BI) lead in solution design and team and client management.
- Participated in developing an AWS Airflow-based ETL tool to replace the legacy Unix-based system and designed the data quality management module.
- Developed proofs of concept (POCs) for AWS services, such as AWS Glue, AWS Lambda, Amazon QuickSight, and AWS Secrets Manager.
- Directed a team of 10 associates across three Microsoft Power BI projects. Designed ETL processes and data models for reporting sales campaign analytics, health care professional (HCP) engagements, and drug development data.
- Leveraged Microsoft Power BI dataflows to manage the computation of heavy transformations for data from multiple sources, such as Amazon Redshift, SQL servers, on-premise servers, and SharePoint.
- Managed three significant Fortune 500 clients based in the USA and the EU, communicating project updates, new dashboards, and feature requests and addressing those questions.
- Eliminated manual collection and quality checks of the sales representatives' time entry data by creating a Microsoft Excel-based dynamic tool with automated validations and dashboarding.
Business Technology Associate
ZS Associates
- Worked as part of three teams of six or seven members as a data engineering, analysis, and operations associate, tasked with solution design, implementation, and system optimization.
- Executed and managed the operations, development, and change requests as a DevOps associate, working on sales incentive compensation (IC) for a pharmaceutical company.
- Automated and optimized the AWS-based IC system using shell scripts, reducing the turnaround time from five to three days and the manual effort by 80%, conserving the Amazon Redshift monthly cluster cost by 20% and enabling leaner operations.
- Supported the setup of a sales incentive compensation (IC) project for a pharmaceutical company. Architected a one-click IC and reporting pipeline using SQL and shell scripts.
- Coordinated with stakeholders and data vendors to ensure high-quality data inputs and mentored the development team to ensure alignment with the best practices and optimized code delivery.
- Developed scalable queries for complex ad-hoc analysis requests, leveraged in future analyses and new modules.
- Onboarded team members at multiple hierarchies working as a subject matter expert (SME) for two pharmaceutical sales IC projects.
Experience
AWS Airflow-based ETL tool
• Participated in designing the ETL tool and system tables using services such as AWS-managed Airflow, Redshift, RDS, and S3.
• Developed the data quality management module.
• Developed proofs of concept (POCs) for AWS Glue, AWS Lambda, Amazon QuickSight, and AWS Secrets Manager to test the integration of these services into the ETL tool.
• Mentored other developers with ETL concepts and the business context of the modules on which they worked.
Device Sales and Campaign Engagement | US-based Technology Giant
• Converted business requirements into technical requirements.
• Coordinated with dashboard users, owners, and the UX team to finalize the dashboard design.
• Coordinated with the client's internal teams to gain access to various data sources.
• Designed the data architecture to support the dashboard.
• Leveraged Power BI dataflows to manage the computation of heavy transformations for data from multiple sources: Redshift, SQL servers, on-premise servers, SharePoint, etc.
• Led the development team and delivered the dashboard with automated refreshes.
Python SQL ETL Tool
The ETL tool had the following features:
• Data extraction from the source database to the desired locations.
• Data loading from multiple locations into the target database.
• Configurable data quality checks and logging and removal of erroneous records.
• Configurable SQL commands executed in the desired order.
Education
Bachelor's Degree in Computer Science
SRM Institute of Science and Technology - Chengalpattu, Tamil Nadu, India
Skills
Libraries/APIs
NumPy, Pandas
Tools
Microsoft Excel, Microsoft Word, Query Plan, Shell, Microsoft Power BI, Microsoft PowerPoint, Power Query, AWS Glue, Apache Airflow, GitHub
Languages
SQL, Python, SQL DML, Snowflake
Storage
Redshift, Amazon S3 (AWS S3), Databases, Relational Databases, SQL Performance, Data Lakes, MySQL, PostgreSQL, Data Pipelines, Database Administration (DBA), Database Migration, Database Architecture, SQL Server Management Studio (SSMS), Microsoft SQL Server, SQL Server Integration Services (SSIS)
Paradigms
ETL, Business Intelligence (BI), Automation, Event-driven Architecture
Platforms
Amazon EC2, Amazon Web Services (AWS), Databricks, SharePoint
Frameworks
Spark
Other
Communication, Data Engineering, Technical Project Management, Data Analysis, Scripting, CSV File Processing, Database Schema Design, Data, Data Analytics, Data Transformation, Data Queries, Process Improvement, Leadership, IT Project Management, Analysis, Team Management, Data Architecture, Big Data, Query Optimization, AWS Cloud Architecture, Datasets, Data Cleansing, Data Profiling, Data Visualization, Reports, Pipelines, DAX, Analytics, Data Warehousing, Back-end, Data Modeling, Data Migration, Documentation, Technical Writing, Operations, Scaling, Reporting, Performance Tuning, Delta Lake, System Design, Amazon RDS, BI Reports, Dashboards, APIs
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring