Senior Data Engineer
2021 - PRESENTWILBUR- Designed and built a scalable data vault in Delta Lakehouse for multiple source systems and their tenants, using AWS, Databricks, a database, Python, Powershell, and Unix. Provisioned environments in AWS using Terraform.
- Developed notebooks in Databricks (dynamic) to load an enterprise Delta Lakehouse. Configured the Databricks platform from scratch on AWS and orchestrated the design of a Delta Lakehouse (data vault model).
- Made high performance dashboards available to insurance clients with architecture, a total win-win for Wilbur as a third party.
Technologies: Databricks, Python, SQL, Unix, AWS, Azure, Datafactory, Terraform, Windows PowerShell, Pandas, SQL Server 2016, Azure SQL Data Warehouse (SQL DW)Senior Data Engineer
2020 - 2021Transport for NSW- Developed dashboards and data pipelines to project near real-time traffic travel times from point A to Point B. The Government used this data to make the lives of citizens easier.
- Built real-time dashboards in Tableau and dynamic data pipelines in Talend. Managed the data feed from different sources and integrated them in a data warehouse in Aurora and Redshift. Made Data Lakes in Amazon Athena that housed data for ML models.
- Showcased traffic patterns and helped government stakeholders to make decisions for building or maintaining infrastructure to help the citizens of the region.
Technologies: AWS, Talend ETL, Python, Amazon Aurora, Datalake, Data Warehouse Design, Redshift, AWS AthenaSenior Data Analyst
2017 - 2019Jones Lang LaSalle- Architected and developed data workflow for multiple source systems. Data was shared across the globe for organizational groups providing them with daily insights to make business decisions.
- Developed data pipelines in Databricks on Azure Cloud, dynamic notebooks were built that could handle any data loading strategy for a data warehouse and data vault. Developers and business users were made aware of data status on a day-to-day basis.
- Created dynamic and scalable architecture so that build time was significantly under control. 360-degree views of businesses have helped real estate agents to ace business deals.
- Developed data feeds on Informatica PowerCenter and Informatica Cloud to ingest into EDW.
Technologies: Databricks, Azure, Python, Informatica ETLSenior Consultant
2012 - 2017Deloitte- Migrated client's data warehouse and data pipelines to the cloud from on-premise.
- Architected and developed data pipelines to ingest data feeds into a data warehouse, using Informatica ETL and Talend ETL. Created a library of Unix functions to reduce the build time in projects and to promote function reusability.
- Transitioned to the cloud, saving up to two million dollars in licensing, administration, and maintenance. Scalable architecture was reused in multiple projects that resulted in bringing down the build time and overriding all manual tasks.
Technologies: AWS, Informatica ETL, Talend ETL, Python, Unix, Redshift
Nitin Keshav
Data Warehouse Design Developer in Sydney, New South Wales, Australia
Member since December 3, 2021
Nitin is a data engineering professional with over 12 years of work experience with expertise in data engineering, cloud computing (architecture and DevOps), enterprise data warehousing, and machine learning. Points of strength include Databricks (Pyspark), ETL (Talend, Informatica Cloud), programming (Python), SQL (Azure SQL, SQL Server, Teradata, Redshift, postgres), scripting (Unix, PowerShell), data modeling, and an impeccable business acumen.
Nitin is now available for hire
Portfolio
- WILBURDatabricks, Python, SQL, Unix, AWS, Azure, Datafactory, Terraform...
- Transport for NSWAWS, Talend ETL, Python, Amazon Aurora, Datalake, Data Warehouse Design...
- Jones Lang LaSalleDatabricks, Azure, Python, Informatica ETL
Experience
- SQL 12 years
- Data Warehouse Design 12 years
- Python 3 10 years
- AWS 7 years
- Informatica ETL 6 years
- Databricks 5 years
- Talend ETL 5 years
- Azure 4 years
Availability
Full-time
Preferred Environment
Databricks, Talend ETL, AWS, Azure, Python 3, Docker, Informatica ETL, Unix, Databases, Apache Airflow
The most amazing...
...project I’ve worked on was a scalable architecture for ingesting data into a data lake and data warehouse dynamically.
Employment
Experience
- Transport for NSW
I created a near real-time traffic and passenger app for New South Wales (NSW). NSW businesses and governments use this data set to upgrade or maintain infrastructure. I worked as a data engineer, ingesting data feeds from different sources into Data Lakes in AWS Athena and a data warehouse in Amazon Aurora and Redshift using Talend.
I also built dashboards in Tableau to showcase data feed status and data completeness for stakeholders. Additionally, dashboards included travel times from point to point on different motorways. The designed data model for Amazon Aurora DB (PostgreSQL) and optimized SQL queries to source Tableau dashboards. I created a data integration control framework in postgres functions to track run status of various jobs and perform SCD operations using a single process dynamically.
Skills
Languages
Python 3, SQL, PythonTools
Talend ETL, Informatica ETL, Azure Machine Learning, Tableau, AWS Athena, Terraform, Apache AirflowPlatforms
Databricks, Unix, AWS Lambda, Azure, DockerStorage
Databases, Redshift, Amazon Aurora, SQL Server 2016Other
AWS, Datalake, Data Warehouse Design, Datafactory, Programming, Azure SQL Data Warehouse (SQL DW)Frameworks
Windows PowerShellLibraries/APIs
Pandas
Education
- Bachelor's Degree in Electronics and Communications2005 - 2009PES University (School of Engineering) - Bangalore, India
Certifications
- AWS Certified Solutions ArchitectOCTOBER 2021 - PRESENTAmazon Web Services
- Databricks Apache Spark 3JUNE 2021 - PRESENTDatabricks
- Talend for Big DataMAY 2020 - PRESENTTalend
- Azure Data Scientist AssociateJANUARY 2020 - PRESENTMicrosoft
- InformaticaMAY 2013 - PRESENTInformatica
