Anish Chakravartty
Verified Expert in Engineering
ETL Developer
Anish is an IT professional with over 13 years of experience in various verticals and knowledge of several technologies. His experience includes working at several service provider companies and freelancing. As a professional with vast working experience with state-of-the-art and modern technologies in data engineering, cloud solutions, and data analytics, Anish excels in full-stack languages and tools, including JavaScript and MEVN stack.
Portfolio
Experience
Availability
Preferred Environment
PostgreSQL, BigQuery, Google Cloud, Node.js, Azure, Microsoft Power BI, Azure Databricks, Databricks
The most amazing...
...thing I've worked on is designing the ETL framework for IBM InfoSphere DataStage batch jobs, enabling over 3,000 ETL processes to load data across the project.
Work Experience
Senior Developer
Freelance
- Developed a Wix plugin dealing with abandoned carts as a full-stack developer.
- Acted as a back-end developer for a mobile app providing banking services.
- Contributed to a bot creation project for a pharmaceutical test clinic as the primary developer.
- Developed a plugin for the Front integration platform as a full-stack developer.
- Created multiple chatbots built on a varied number of platforms.
Assistant Consultant
Tata Consultancy Services
- Designed ETL framework that enables 3,000+ batch ETL jobs to load data across various subject areas.
- Acted as a senior developer and architect for migrating on-prem databases to the cloud.
- Designed and developed a data-fix process in data warehouse and ODS layers when source systems were merged during the cloud migration.
- Contributed to the SME projects using IBM InfoSphere DataStage and Oracle PL/SQL.
- Conducted technical interviews for relevant technologies during onboarding new people across the project.
- Received recognition and award as the contextual master of the entire business unit and numerous appreciations from senior client management for successfully implementing several projects.
Data Engineers (Toptal Teams)
Compliance Made Easy Inc. d/b/a CertifyOS
- Created an ETL framework using Apache Beam (Cloud Dataflow) to migrate data from Firestore to PostgreSQL.
- Employed Cloud Run to execute Dockerized Python jobs essential for the migration process.
- Used Cloud Workflow to orchestrate the entire ETL framework.
IoT Data Engineer
Regent Climate Connect Knowledge Solutions Pvt Ltd
- Created a warehouse on BigQuery to store SCADA data from various power plants, handling modeling, ETL, and DevOps tasks. Created ETL jobs on Cloud Dataflow to feed data to downstream systems.
- Assisted the Data Science team in deploying their data models on Vertex AI and created a dashboard on Looker on the model outputs, automating the entire process.
- Performed several enhancements and bug fixes on an existing Data Lake on Azure and the ETL framework on Databricks and Data Factory. Set up a fresh dashboard on Power BI based on the Data Lake.
- Helped set up and manage the data team for the company.
ETL Specialist
IBM
- Designed the ETL architecture to load data into a data warehouse for an insurance client.
- Created the architecture of data marts for several service lines.
- Acted as a senior developer for a data migration project to SAP.
- Performed efficient multiple data recoveries and various spontaneous actions for which I received appreciation emails from business users.
CRM Consultant
Cognizant
- Loaded data into Siebel CRM as an ETL subject matter expert.
- Tuned the data loading tasks with low performance, which saved a lot of time.
- Provided training to new joiners on DataStage development and application-based knowledge transfer.
Project Engineer
Wipro
- Acted as the SPOC for data warehousing applications for EMEA and APAC regions.
- Implemented performance enhancement and new geography accommodation for existing applications successfully. Reduced run time by more than half as needed for older tasks.
- Trained and guided new team members on DataStage job development and application-based knowledge transfer.
- Received the AIM Star of The Year and two awards for individual performance in different projects.
Experience
ETL Framework to Enable InfoSphere DataStage Batch Loads
I created the architecture of the new framework and worked as the technical lead in implementing the new framework and migrating jobs. The completed project enabled DataStage jobs migration without any changes, which resulted in all-around applause and appreciation.
Abandoned Cart Recovery Wix Plugin
I contributed to this project as a full-stack developer creating the front end for the Wix plugin and subscription modal script in Vue 2. The back end was hosted as multiple Google Cloud functions using Node.js and Express.js, which followed the REST architecture. We used Cloud Firestore as the operational database.
WhatsApp Bot and ODS Development for Pharmacy Test Clinic
The second part of the project consisted of designing an ODS in BigQuery that used data from the WhatsApp bot, client website, and physical store data. Cloud functions were used to consume data in the form of files, REST API calls, and BigQuery stored procedures wrapped in pub/sub cloud functions for loading data into the ODS.
Data Warehouse for a Renewable Energy Provider on GCP
After loading the data into the warehouse, I created various materialized views with different aggregations to optimize performance. These views were utilized in a dashboard built in Looker for easy visualization and analysis. Additionally, I developed a web app dashboard that utilized data from these materialized views for real-time insights.
Additionally, I constructed a batch ETL framework using Apache Beam deployed on Cloud Dataflow, generating files on Cloud Storage for data models deployed on Vertex AI. The output of these models was leveraged on Looker dashboards and the web app.
I created the architecture of the entire system and worked on its implementation. Also, I set up and managed the initial phases of a data engineering team for the clients.
Education
Bachelor's Degree in Electronics and Instrumentation Engineering
West Bengal University of Technology - Kolkata, India
Skills
Libraries/APIs
Node.js, PySpark, Vue, Pandas, Rasa NLU, Vue 2
Tools
BigQuery, IBM InfoSphere (DataStage), Dialogflow, Apache Beam, Microsoft Power BI, Looker, Cloud Dataflow, Docker Compose, Google Analytics, ManyChat
Frameworks
Express.js, Spark, Botpress.io
Languages
JavaScript, SQL, Python, C, C++, Embedded C
Paradigms
ETL, Business Intelligence (BI), Microservices
Platforms
Oracle, Google Cloud Platform (GCP), Databricks, Apache Kafka, Docker, Azure, Chatfuel, Kubernetes, Firebase, AWS IoT, Vertex AI
Storage
PL/SQL, Oracle PL/SQL, Databases, Data Pipelines, Data Lakes, PostgreSQL, Google Cloud, IBM Db2, SQL Server 2010, Cloud Firestore, MongoDB, Siebel EIM, Teradata, Database Migration, NoSQL
Other
PL/SQL Tuning, Google Cloud Functions, Data Engineering, Data Architecture, Data Warehousing, Data Modeling, Google Data Studio, API Integration, Pub/Sub, Google BigQuery, Azure Databricks, Data Analysis, Full-stack, Azure Data Lake, Azure Data Factory, Shell Scripting, Microprocessors, Internet of Things (IoT), Looker Studio
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring