Chirag Dhawan
Verified Expert in Engineering
Data Engineer and Developer
Dubai, United Arab Emirates
Toptal member since November 24, 2022
Chirag is a data engineer with 5+ years of experience in multiple industries like healthcare, eCommerce, and banking. He is a quick learner with a knack for problem-solving who can be the driving force in busy teams, performing various tasks that different situations might require. He is also well-versed in patient-level data, making him an asset to healthcare teams. Chirag believes in making an impact and delivering value to his clients and customers from day one.
Portfolio
Experience
- SQL - 4 years
- Python - 4 years
- Linux - 4 years
- Git - 4 years
- Redshift - 3 years
- Amazon Web Services (AWS) - 3 years
- Spark - 3 years
- Docker - 1 year
Availability
Preferred Environment
Linux, Redshift, Python, SQL, Spark, Amazon Web Services (AWS), Git, Docker
The most amazing...
...project I've worked on is setting up data pipelines using Spark at the core with the ability to process 4+ TBs of data in a few hours.
Work Experience
Data Engineer
Amazon.com
- Developed, maintained, and optimized data pipelines that serve data to analytics teams.
- Improved unit test coverage for existing pipelines using Pytest and unit test packages.
- Collaborated in gathering the requirements from the analysts and BI engineers, working with stakeholders in an Agile environment to fulfill their requests within the defined SLA.
Data Engineer
G42 Healthcare
- Identified areas for improvement and optimization in the in-house ETL application developed using Scala and Spark.
- Implemented data pipelines and worked on performance tuning of existing pipelines using the in-house ETL application.
- Analyzed and studied the source data, helping the team identify significant data quality issues at the source.
- Developed an application using Scala to measure the data quality metrics for a given data set.
Senior Analyst | Data Engineering
ZS Associates
- Worked on developing pipelines for processing high-volume patient-level data sets using technologies like Spark on EMR clusters and orchestration of processes using Apache Airflow.
- Handled a dashboard's back end, which involved creating data warehouse tables using Pandas, a Python library, by consuming flat files from Amazon S3.
- Collaborated in the front- and back-end development of a web-based dashboard. The tech stack of the application was AngularJS and C3.js for visualizations, along with Flask and Pandas on the back end.
- Developed a deep understanding of patient-level data sets and OHDSI tools, which helped me act as an SME for such data sets within the team.
- Defined business rules for transforming these data sets into a common data model.
Software Engineer
Newgen Software Technologies Limited
- Worked on developing banking systems that digitized processes such as account opening and processing inward Swift messages.
- Developed an archival data module for a bank that archived all documents after specific timeframes, moving them from client servers to a device called EMC Centera.
- Handled direct communication with the client to gather requirements.
- Assisted the client with user acceptance testing (UAT) and deployment on UAT and production servers.
- Frequently led on-call support and collaborated with client teams to help resolve production issues within the defined SLA.
Experience
Unified Data Model
Education
Bachelor's Degree in Information Technology
Jaypee Institute of Information Technology - Noida, India
Skills
Tools
Git, Amazon Athena, AWS Glue, PyCharm, DataGrip, Apache Airflow, Amazon Elastic MapReduce (EMR)
Languages
SQL, Python, Scala, Java
Paradigms
ETL
Storage
Relational Databases, Redshift, Databases, PostgreSQL
Frameworks
Spark, Jakarta Server Pages (JSP)
Platforms
AWS Lambda, Windows, Linux, Amazon Web Services (AWS), Docker
Other
Data Modeling, Software Development, Operating Systems, Computer Networking
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring