Tapan Bhatt

A cloud-based platform that received real-time data from around 80,000 users based in Europe. I developed the Spark-based application, which processed data landing on S3 Dropbox in real time. Also, designed an HBase-backed bot detection engine that identified and blacklisted malicious IP addresses with 97% efficiency.

A rule-based engine for an insurance product. I served as the database developer and completely rewrote the entire calculation engine from scratch using T-SQL. The end result was mesmerizing—the new version performed 6x faster than the legacy component and was much easier to maintain because it took only one-sixth number of lines of code.

The new version could process 430,000 transactions in one hour compared to only 75,000 processed by the legacy version. I also added additional features in the new version, such as pre-payments, plan transfers, recurring payments, and deductions.

Customer.io (CIO) is a CRM platform used by marketers to connect with their users and increase their engagement through powerful marketing campaigns. I worked as an integration specialist and supplied real-time Kafka events by calling CIO endpoints. I also worked with the OPS team to design the workflow of CRM campaigns used to send welcome bonuses.

I served as an Apache Spark developer for one of the world's largest banks. Our team was building a Hadoop-based data lake where data from multiple sources was loaded and later used for advanced analysis.

I implemented a Spark-based framework that:
• Compared the ingested data with the source in an efficient manner.
• Stored a summary of comparison results for audit purposes, like the number of rows, minimum and maximum, the number of distinct values of each column, and the average value of each numerical column.
• Generated alerts if any mismatch was found during validation.

This resulted in increased confidence of stakeholders in the quality of data stored in the lake, less time spent on identifying root causes of potential issues, and faster delivery of new ingest requests.

I worked for a large business suite in the accounts payable module as an application engineer. The module was responsible for storing invoices in the system and paying suppliers on time. The module needed to be upgraded when a new major release became available from Oracle, and old versions went out of support.

The process responsible for upgrades used to be very slow, especially for large customers where millions of invoices needed to be upgraded. I identified key architectural issues in the upgrade process and implemented a new upgrade process using parallel threads that leveraged a multi-core CPU.

The upgrade downtime came down from 12 hours to just 2.5 hours.