Avinash Holla Pandeshwar, Developer in Bengaluru, Karnataka, India
Avinash is available for hire
Hire Avinash

Avinash Holla Pandeshwar

Verified Expert  in Engineering

BI Analyst, Data Engineer, and Developer

Location
Bengaluru, Karnataka, India
Toptal Member Since
November 9, 2021

Avinash has close to nine years of data engineering experience and a strong background in SQL, data modeling, ETL/ELT, and big data technologies such as Hive and Spark. He currently works as a technical lead in one of India's most popular social media apps. Avinash codes in Python and uses Apache Airflow to orchestrate data pipelines. He is a dedicated and self-driven professional who strives for excellence.

Portfolio

ShareChat
Apache Airflow, Spark, Google BigQuery, Python 3, Google Pub/Sub...
Noodle.ai
Python 3, RDBMS, ETL, Docker, Apache Airflow, Spark, HDFS, Programming...
UnitedHealthcare
SQL Server 2016, Teradata, Oracle, IBM Db2...

Experience

Availability

Part-time

Preferred Environment

PostgreSQL, SQL Server 2016, Python 3, Apache Airflow, Apache Hive, Spark, Google Cloud Platform (GCP), Amazon Web Services (AWS), Dimensional Modeling, Google BigQuery

The most amazing...

...solutions I've engineered helped in improving revenue due to increased stability and fewer downtimes.

Work Experience

Technical Lead

2022 - PRESENT
ShareChat
  • Led the ads data team at ShareChat's monetization department.
  • Orchestrated over 200 feature engineering pipelines using Airflow.
  • Scaled pipeline bottlenecks using Serverless Spark on GCP.
  • Created datasets on BigQuery housing ad relevance-related features spanning around 1TB/day.
  • Built a real-time pipeline using GCP Pub/Sub with BigQuery subscription to support storage of ad relevance service logs.
Technologies: Apache Airflow, Spark, Google BigQuery, Python 3, Google Pub/Sub, Google Cloud Storage, Google Cloud Platform (GCP), Google Cloud Dataproc, Jira, ELT, ETL, Big Data, Big Data Architecture, JSON

Principal Data Engineer

2019 - 2022
Noodle.ai
  • Built a data lake on AWS S3 for batch processing and feature engineering.
  • Used SQL Server and PostgreSQL to build multiple data-driven applications.
  • Created a DQA library using Spark to analyze batch data and provide quality insights.
  • Worked with Python and Airflow to create and orchestrate pipelines to sync data from client systems.
  • Re-implemented the existing batch ingestion and feature generation architecture of time series data to make it more reusable across clients.
Technologies: Python 3, RDBMS, ETL, Docker, Apache Airflow, Spark, HDFS, Programming, Databases, Project Lifecycle, Apache Hive, Ubuntu 16.04, PyCharm, PostgreSQL, SQL Server 2016, Jupyter Notebook, Bitbucket, Docker Hub, Python, Data Engineering, Relational Databases, Amazon Web Services (AWS), Amazon S3 (AWS S3), Data Analytics, SQL, Database Design, Database Schema Design, Microsoft SQL Server, Database Optimization, pgAdmin, ELT, Big Data, Big Data Architecture, JSON

Senior Data Engineer

2018 - 2019
UnitedHealthcare
  • Engineered appropriate solutions to avoid future roadblocks and loss of revenue due to government-imposed penalties to the company.
  • Built optimized frameworks for existing projects to improve debugging and monitoring capabilities using customized logging and notifications.
  • Analyzed project gaps and emphasized automation, leading to better team resource utilization.
Technologies: SQL Server 2016, Teradata, Oracle, IBM Db2, SQL Server Integration Services (SSIS), Data Engineering, Data Warehousing, Relational Databases, Data Analytics, SQL, Database Design, Database Schema Design, Microsoft SQL Server, Database Optimization, ETL, ELT

Business Intelligence Analyst

2014 - 2018
NTT Data
  • Built ETL pipelines that source data from multiple sources and transform them into a central golden copy at the application data warehouse.
  • Generated reports to provide information at multiple hierarchal levels.
  • Gained the client's trust and worked with highly confidential data.
Technologies: SQL Server 2012, Oracle, SQL Server Integration Services (SSIS), SQL Server Reporting Services (SSRS), Data Engineering, Relational Databases, Data Visualization, Dashboards, Reporting, Data-driven Dashboards, Data Analytics, SQL, Database Design, Database Schema Design, Microsoft SQL Server, .NET Core, C#, Database Optimization, ETL

Production Flow AI

Production Flow AI is an enterprise AI product that integrates into the manufacturing lifecycle of certain products and provides machine learning-driven insights on how wastage can be reduced, leading to cost savings. Client data is consumed and transformed into formats understood by the ML models, which use it to interpret optimal baselines. The outputs involve predictive indicators such as the optimal percentage of raw materials needed and the duration of processing at specific steps.

Data engineering is involved in designing the database schema used by the app, building pipelines to ingest and supply data to various components, batch tracking, and building procedures that APIs interact with.

Languages

SQL, Python 3, Python, C#, Solidity

Paradigms

ETL, Database Design, Dimensional Modeling

Storage

Databases, PostgreSQL, SQL Server 2016, Relational Databases, Microsoft SQL Server, SQL Server Integration Services (SSIS), SQL Server Reporting Services (SSRS), Google Cloud Storage, JSON, RDBMS, HDFS, Apache Hive, Teradata, IBM Db2, SQL Server 2012, Amazon S3 (AWS S3), NoSQL

Other

Data Engineering, Data Analytics, Programming, Database Schema Design, Database Optimization, Google BigQuery, ELT, Big Data, Big Data Architecture, Project Lifecycle, Windows 10, Ubuntu 16.04, Data Warehousing, Data Visualization, Dashboards, Reporting, Data-driven Dashboards, Google Pub/Sub

Platforms

Google Cloud Platform (GCP), Docker, Jupyter Notebook, Oracle, Amazon Web Services (AWS), AWS IoT, Blockchain

Frameworks

Spark, Hadoop, .NET Core

Tools

Apache Airflow, PyCharm, Docker Hub, GitHub, Bitbucket, pgAdmin, Google Cloud Dataproc, Jira

2008 - 2013

Bachelor's Degree in Computer Science

RNS Institute of Technology - Bangalore, India

JUNE 2022 - PRESENT

Advanced Certification in Software Engineering for Cloud, IoT and Blockchain

IIT Madras and Greatlearning

SEPTEMBER 2020 - PRESENT

Big Data Expert

Edureka

Collaboration That Works

How to Work with Toptal

Toptal matches you directly with global industry experts from our network in hours—not weeks or months.

1

Share your needs

Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2

Choose your talent

Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3

Start your risk-free talent trial

Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.

Top talent is in high demand.

Start hiring