Principle Engineer2019 - 2023Stack Trace Eng LLP
Technologies: Python 3, Python 2, APIs, Cloud, Java 8, Apache Airflow, Hadoop, REST
- Worked as a consultant on various projects from medium to high scale, across a wide variety of technologies.
- Collaborated on building products from scratch with a team, while leading the team responsible for managing APIs that handle a billion requests per day.
- Lead the efforts to scale existing products from 1x to 3x, which in turn helped in reducing scale-up costs at their peaks.
Lead Software Engineer2017 - 2019Scripbox
Technologies: Amazon Web Services (AWS), MongoDB, PostgreSQL 10.1, MySQLdb, SQL, Data Engineering, Docker, MySQL, PostgreSQL, Redshift, React, Django, Apache Spark, RabbitMQ, Celery, Apache Airflow, Python, ETL
- Developed an analysis tool from scratch using Python and Django with React.
- Implemented a workflow framework on top of Airflow.
- Developed entire data pipelines from scratch using Apache Airflow and Apache Spark.
- Integrated multiple upstream data sources into AWS Redshift.
- Implemented dashboards using Redash and Metabase for data visualization purposes.
- Managed the data engineering team and provided technical guidance to other teams as well.
- Mentored different teams on database-design-related issues.
Senior Software Engineer2016 - 2017JPMorgan Chase & Co.
Technologies: PostgreSQL, Flask, Cassandra, Amazon Athena, Python
- Implemented a graphical user interface (GUI) with Python and Enaml for securities trading management.
- Implemented REST services for downstream application usage.
- Integrated multiple upstream services to get securities-related data in the bank.
- Migrated a part of the legacy application to a new framework from scratch.
- Improved unit test coverage from 60% to 95%.
Engineering Lead2015 - 2016NearFox
- Designed and implemented a Nearfox product from scratch.
- Implemented the back end using Python and Django REST.
- Designed the admin panel for user management for the internal operations team.
- Built and maintained infrastructure fully deployed in AWS.
- Implemented CI/CD pipelines with Jenkins, automated most of the deployment process.
- Mentored and managed a team of four passionate engineers.
- Setup and deployed container infrastructure with Docker and Kubernetes.
Co-founder | CTO2014 - 2016Indiallo.com
Technologies: Kubernetes, Docker, PostgreSQL, Scrapy, Django, Python, Node.js, Ionic, Cordova, ETL, Web Scraping
- Developed a hybrid app using Cordova with Ionic, managed app developers.
- Implemented a back end with Python and Django REST.
- Crawled data heavily from multiple sources using Scrapy and custom crawlers.
- Implemented a large-scale data processing layer for data consistency.
- Implemented CI/CD pipelines with Kubernetes and Docker.
- Managed the tech part of the product without any other help.
Senior Software Engineer2014 - 2015Bank of America
Technologies: Netezza, Sybase, Cassandra, Quartz, Flask, Python
- Implemented a desktop application using Python and Tkinter.
- Developed a framework for a position reconciliation process.
- Implemented a worker strategy to handle heavy volume of data input from multiple sources.
- Migrated ETL pipelines from Informatica tool to a Python-based Quartz infrastructure.
- Built the back-end API that can be accessed by downstream applications for data updations.
Software Developer2011 - 2014Ness Digital Engineering
- Developed an internal tool for workflow management for Google Map Maker edits.
- Implemented complex algorithms to take a correct sampling of edits for routing.
- Created an internal dashboard for the higher management of map project insights.
- Built a people management system web application with Python and Django.
- Constructed a data store in Google Dremel.
- Implemented a tool for automated approval of edits over Google Map Maker.
- Designed a data layer with multiple databases like MySQL, PostgreSQL, and Dremel.
- Implemented a caching layer with Java for reducing the load on the mainstream servers.