Anish Chakravartty

The project was a part of a client's group insurance vertical takeover by a different insurance company. The ETL jobs feeding data into the data warehouse and ODS for the client depended on the proprietary ETL framework used to manage parameterization, restartability, and load status. This ETL framework was not part of the takeover items, and as such, it enabled the ETL process to function in the target landscape, where existed a requirement for creating a new framework that would allow the execution of the jobs without making any changes.

I created the architecture of the new framework and worked as the technical lead in implementing the new framework and migrating jobs. The completed project enabled DataStage jobs migration without any changes, which resulted in all-around applause and appreciation.

A Wix plugin enables Wix store owners to track abandoned carts and send recovery messages via different channels such as Facebook, email, and WhatsApp.

I contributed to this project as a full-stack developer creating the front end for the Wix plugin and subscription modal script in Vue 2. The back end was hosted as multiple Google Cloud functions using Node.js and Express.js, which followed the REST architecture. We used Cloud Firestore as the operational database.

The project consisted of two parts. The first part involved creating a WhatsApp bot that served customer inquiries, lead generation, sample collection, and report distribution. This bot was developed using Google Dialogflow for NLU and context detection. A Node.js back end hosted on GCP App Engine and a Cloud Firestore DB was designed and developed, integrating with Twilio for communication over WhatsApp.

The second part of the project consisted of designing an ODS in BigQuery that used data from the WhatsApp bot, client website, and physical store data. Cloud functions were used to consume data in the form of files, REST API calls, and BigQuery stored procedures wrapped in pub/sub cloud functions for loading data into the ODS.

This project was done to create a data warehouse on Google BigQuery for a renewable energy provider. Data from the SCADA systems of different renewable energy power plants (solar, wind, etc.) were fetched in near real-time and loaded via REST APIs to BigQuery.

After loading the data into the warehouse, I created various materialized views with different aggregations to optimize performance. These views were utilized in a dashboard built in Looker for easy visualization and analysis. Additionally, I developed a web app dashboard that utilized data from these materialized views for real-time insights.

Additionally, I constructed a batch ETL framework using Apache Beam deployed on Cloud Dataflow, generating files on Cloud Storage for data models deployed on Vertex AI. The output of these models was leveraged on Looker dashboards and the web app.

I created the architecture of the entire system and worked on its implementation. Also, I set up and managed the initial phases of a data engineering team for the clients.