Rachit is currently unavailable
Rachit Mutreja
Verified Expert in Engineering
Software Developer
Kitchener, ON, Canada
Toptal member since June 27, 2024
Bio
Rachit is a senior back-end engineer with over a decade of experience building big data pipelines, microservices, and APIs. He is proficient in Java, Scala, and Python and is skilled in leveraging various AWS-managed and unmanaged services to build scalable and high-performance back-end systems. Rachit is excited to take on new challenges and new projects.
Portfolio
Nielsen
Java, Amazon Web Services (AWS), Spark, Microservices, RESTful Web Services...
Cognizant
Python 3, Hadoop, HDFS, Scrum, SQL, Yarn, PySpark, Agile
Optum
Scala, Unix, Unix Shell Scripting, Spark, Hadoop, HDFS, Yarn, Java...
Experience
- Java - 11 years
- Microservices - 10 years
- SQL - 10 years
- Spark - 7 years
- Amazon Web Services (AWS) - 4 years
- Python 3 - 4 years
- Functional Programming - 4 years
- Scala - 4 years
Work Experience
Senior Software Developer
2021 - PRESENT
Nielsen
- Built and optimized data pipelines processing around 10 billion ad impressions monthly, ensuring efficient storage in Amazon S3 and seamless retrieval.
- Designed and implemented distributed, event-driven asynchronous REST APIs using Java and Spring boot, processing around 20,000 ad impressions per second, ensuring p99 latency under 50 ms.
- Operationalized machine learning models within Apache Spark data pipelines to identify and flag around 7% of sophisticated invalid traffic in real time using micro-batching, enhancing reporting accuracy.
Technologies: Java, Amazon Web Services (AWS), Spark, Microservices, RESTful Web Services, Boto 3, Troposphere, Scala, Functional Programming, AWS Glue, Amazon Athena, Amazon EC2, AWS Lambda, Python 3, Amazon Elastic MapReduce (EMR), Amazon S3 (AWS S3)
Manager Projects
2020 - 2021
Cognizant
- Designed and implemented Spark jobs to create an organized data layer on top of a data lake based on Apache Hive containing common variables to be used across the organization.
- Created and delegated tasks to a team of four people based on their skill set, task priority, and risk.
- Ensured any technical or operational blockers faced by the team were addressed.
Technologies: Python 3, Hadoop, HDFS, Scrum, SQL, Yarn, PySpark, Agile
Senior Software Engineer
2016 - 2020
Optum
- Developed an on-premise data lake using the Hadoop ecosystem.
- Built an ingestion system using Sqoop and Spark to ingest data from various sources into the data lake.
- Automated the monitoring of the data ingestion process to reduce manual effort.
Technologies: Scala, Unix, Unix Shell Scripting, Spark, Hadoop, HDFS, Yarn, Java, Apache Airflow, Apache Sqoop
Senior Software Engineer
2015 - 2016
Impetus Technologies
- Collaborated closely with a team of data scientists working on merchant transaction data for American Express to identify anomalies.
- Designed and implemented scalable search infrastructure using Apache Solr to enable fast and advanced search capabilities like text-based and geospatial search.
- Created and implemented REST APIs using the Spring MVC framework to retrieve data indexed in Elasticsearch.
Technologies: Hadoop, HDFS, Java, Elasticsearch, SQL, ETL, Spring, MapR, Apache Hive, Apache Pig, Unix Shell Scripting
Senior Software Engineer
2013 - 2015
Ness Digital Engineering
- Developed a framework used by car dealers across the USA for building their websites and inventory management.
- Ensured that the websites published by dealers were compliant with Web Content Accessibility Guidelines (WCAG).
- Implemented a test-driven development (TDD) methodology within an Agile framework, utilizing the Rally Agile project management tool for efficient project planning and execution.
Technologies: Java, Spring, SQL, MySQL, Agile
Associate Consultant
2010 - 2013
Oracle
- Customized and implemented FLEXCUBE, a core banking solution by Oracle, for over 10 clients.
- Built complex PL/SQL procedures to migrate data from various legacy banking systems to a database schema defined for Oracle FLEXCUBE.
- Generated insightful reports for clients using Oracle BI Publisher.
Technologies: Oracle, SQL, PL/SQL, Java, Oracle BI Publisher, Oracle WebLogic Application Server
Project History
Data Pipeline for Processing Near Real-time Ad Measurement Data
I was a back-end engineer who developed a data pipeline that collects and processes ad measurement data in near real-time. The pipeline would collect millions of data points using RESTful services, process them using Spark on Elastic MapReduce (Amazon EMR), and catalog them in AWS Glue for further analysis.
Skills
Libraries/APIs
PySpark
Tools
Slack, Amazon Elastic MapReduce (EMR), IntelliJ IDEA, GitHub, Boto 3, AWS Glue, Amazon Athena, AWS CloudFormation, GitLab CI/CD, Apache Airflow, Apache Sqoop, Oracle BI Publisher
Languages
Java, Scala, Python 3, SQL
Frameworks
Spark, Hadoop, Yarn, Spring
Platforms
AWS Lambda, Amazon Web Services (AWS), Amazon EC2, Linux, Unix, MapR, Apache Pig, Oracle
Paradigms
Microservices, Functional Programming, Scrum, Agile, ETL
Storage
Amazon S3 (AWS S3), Amazon DynamoDB, HDFS, Elasticsearch, Apache Hive, MySQL, PL/SQL
Other
Troposphere, RESTful Web Services, SA, Unix Shell Scripting, Oracle WebLogic Application Server
Collaboration That Works
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
1
Share your needs
Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2
Choose your talent
Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3
Start your risk-free talent trial
Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.
Top talent is in high demand.
Start hiring