Rachit Mutreja
Verified Expert in Engineering
Software Developer
Kitchener, ON, Canada
Toptal member since June 27, 2024
Rachit is a senior back-end engineer with over a decade of experience building big data pipelines, microservices, and APIs. He is proficient in Java, Scala, and Python and is skilled in leveraging various AWS-managed and unmanaged services to build scalable and high-performance back-end systems. Rachit is excited to take on new challenges and new projects.
Portfolio
Experience
- Java - 11 years
- Microservices - 10 years
- SQL - 10 years
- Spark - 7 years
- Amazon Web Services (AWS) - 4 years
- Python 3 - 4 years
- Functional Programming - 4 years
- Scala - 4 years
Availability
Preferred Environment
Linux, IntelliJ IDEA, GitHub, Slack, Spark, Java, Scala, Python 3
The most amazing...
...thing I've built is a re-usable all-in-1 scheduling system which took care of cluster management, and running Spark jobs on EMR.
Work Experience
Senior Software Developer
Nielsen
- Built and optimized data pipelines processing around 10 billion ad impressions monthly, ensuring efficient storage in Amazon S3 and seamless retrieval.
- Designed and implemented distributed, event-driven asynchronous REST APIs using Java and Spring boot, processing around 20,000 ad impressions per second, ensuring p99 latency under 50 ms.
- Operationalized machine learning models within Apache Spark data pipelines to identify and flag around 7% of sophisticated invalid traffic in real time using micro-batching, enhancing reporting accuracy.
Manager Projects
Cognizant
- Designed and implemented Spark jobs to create an organized data layer on top of a data lake based on Apache Hive containing common variables to be used across the organization.
- Created and delegated tasks to a team of four people based on their skill set, task priority, and risk.
- Ensured any technical or operational blockers faced by the team were addressed.
Senior Software Engineer
Optum
- Developed an on-premise data lake using the Hadoop ecosystem.
- Built an ingestion system using Sqoop and Spark to ingest data from various sources into the data lake.
- Automated the monitoring of the data ingestion process to reduce manual effort.
Senior Software Engineer
Impetus Technologies
- Collaborated closely with a team of data scientists working on merchant transaction data for American Express to identify anomalies.
- Designed and implemented scalable search infrastructure using Apache Solr to enable fast and advanced search capabilities like text-based and geospatial search.
- Created and implemented REST APIs using the Spring MVC framework to retrieve data indexed in Elasticsearch.
Senior Software Engineer
Ness Digital Engineering
- Developed a framework used by car dealers across the USA for building their websites and inventory management.
- Ensured that the websites published by dealers were compliant with Web Content Accessibility Guidelines (WCAG).
- Implemented a test-driven development (TDD) methodology within an Agile framework, utilizing the Rally Agile project management tool for efficient project planning and execution.
Associate Consultant
Oracle
- Customized and implemented FLEXCUBE, a core banking solution by Oracle, for over 10 clients.
- Built complex PL/SQL procedures to migrate data from various legacy banking systems to a database schema defined for Oracle FLEXCUBE.
- Generated insightful reports for clients using Oracle BI Publisher.
Experience
Data Pipeline for Processing Near Real-time Ad Measurement Data
Skills
Libraries/APIs
PySpark
Tools
Slack, Amazon Elastic MapReduce (EMR), IntelliJ IDEA, GitHub, Boto 3, AWS Glue, Amazon Athena, AWS CloudFormation, GitLab CI/CD, Apache Airflow, Apache Sqoop, Oracle BI Publisher
Languages
Java, Scala, Python 3, SQL
Frameworks
Spark, Hadoop, Yarn, Spring
Platforms
AWS Lambda, Amazon Web Services (AWS), Amazon EC2, Linux, Unix, MapR, Apache Pig, Oracle
Paradigms
Microservices, Functional Programming, Scrum, Agile, ETL
Storage
Amazon S3 (AWS S3), Amazon DynamoDB, HDFS, Elasticsearch, Apache Hive, MySQL, PL/SQL
Other
Troposphere, RESTful Web Services, SA, Unix Shell Scripting, Oracle WebLogic Application Server
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring