Principle Engineer
2019 - 2023Stack Trace Eng LLP- Worked as a consultant on various projects from medium to high scale, across a wide variety of technologies.
- Collaborated on building products from scratch with a team, while leading the team responsible for managing APIs that handle a billion requests per day.
- Lead the efforts to scale existing products from 1x to 3x, which in turn helped in reducing scale-up costs at their peaks.
Technologies: Python 3, Python 2, APIs, Cloud, Java 8, Apache Airflow, Hadoop, RESTLead Software Engineer
2017 - 2019Scripbox- Developed an analysis tool from scratch using Python and Django with React.
- Implemented a workflow framework on top of Airflow.
- Developed entire data pipelines from scratch using Apache Airflow and Apache Spark.
- Integrated multiple upstream data sources into AWS Redshift.
- Implemented dashboards using Redash and Metabase for data visualization purposes.
- Managed the data engineering team and provided technical guidance to other teams as well.
- Mentored different teams on database-design-related issues.
Technologies: Amazon Web Services (AWS), MongoDB, PostgreSQL 10.1, MySQLdb, SQL, Data Engineering, Docker, MySQL, PostgreSQL, Redshift, React, Django, Apache Spark, RabbitMQ, Celery, Apache Airflow, Python, ETLSenior Software Engineer
2016 - 2017JPMorgan Chase & Co.- Implemented a graphical user interface (GUI) with Python and Enaml for securities trading management.
- Implemented REST services for downstream application usage.
- Integrated multiple upstream services to get securities-related data in the bank.
- Migrated a part of the legacy application to a new framework from scratch.
- Improved unit test coverage from 60% to 95%.
Technologies: PostgreSQL, Flask, Cassandra, Amazon Athena, PythonEngineering Lead
2015 - 2016NearFox- Designed and implemented a Nearfox product from scratch.
- Implemented the back end using Python and Django REST.
- Designed the admin panel for user management for the internal operations team.
- Designed and implemented most of the front end using JavaScript and Bootstrap.
- Built and maintained infrastructure fully deployed in AWS.
- Implemented CI/CD pipelines with Jenkins, automated most of the deployment process.
- Mentored and managed a team of four passionate engineers.
- Setup and deployed container infrastructure with Docker and Kubernetes.
Technologies: Flask-Marshmallow, Flask-RESTful, Flask, JavaScript, Kubernetes, Docker, Redis, Elasticsearch, PostgreSQL, Android, Django REST Framework, Django, PythonCo-founder | CTO
2014 - 2016Indiallo.com- Developed a hybrid app using Cordova with Ionic, managed app developers.
- Implemented a back end with Python and Django REST.
- Crawled data heavily from multiple sources using Scrapy and custom crawlers.
- Implemented a large-scale data processing layer for data consistency.
- Implemented CI/CD pipelines with Kubernetes and Docker.
- Managed the tech part of the product without any other help.
Technologies: Kubernetes, Docker, PostgreSQL, Scrapy, Django, Python, Node.js, Ionic, Cordova, ETL, Web ScrapingSenior Software Engineer
2014 - 2015Bank of America- Implemented a desktop application using Python and Tkinter.
- Developed a framework for a position reconciliation process.
- Implemented a worker strategy to handle heavy volume of data input from multiple sources.
- Migrated ETL pipelines from Informatica tool to a Python-based Quartz infrastructure.
- Built the back-end API that can be accessed by downstream applications for data updations.
Technologies: Netezza, Sybase, Cassandra, Quartz, Flask, PythonSoftware Developer
2011 - 2014Ness Digital Engineering- Developed an internal tool for workflow management for Google Map Maker edits.
- Implemented complex algorithms to take a correct sampling of edits for routing.
- Created an internal dashboard for the higher management of map project insights.
- Built a people management system web application with Python and Django.
- Constructed a data store in Google Dremel.
- Implemented a tool for automated approval of edits over Google Map Maker.
- Designed a data layer with multiple databases like MySQL, PostgreSQL, and Dremel.
- Implemented a caching layer with Java for reducing the load on the mainstream servers.
Technologies: SQLAlchemy, JavaScript, BigQuery, PL/SQL, PostgreSQL, MySQL, Dremel, Borg, Django, Java, Python