Avinash Holla Pandeshwar
Verified Expert in Engineering
BI Analyst, Data Engineer, and Developer
Bengaluru, Karnataka, India
Toptal member since November 9, 2021
Avinash has close to nine years of data engineering experience and a strong background in SQL, data modeling, ETL/ELT, and big data technologies such as Hive and Spark. He currently works as a technical lead in one of India's most popular social media apps. Avinash codes in Python and uses Apache Airflow to orchestrate data pipelines. He is a dedicated and self-driven professional who strives for excellence.
Portfolio
Experience
Availability
Preferred Environment
PostgreSQL, SQL Server 2016, Python 3, Apache Airflow, Apache Hive, Spark, Google Cloud Platform (GCP), Amazon Web Services (AWS), Dimensional Modeling, Google BigQuery
The most amazing...
...solutions I've engineered helped in improving revenue due to increased stability and fewer downtimes.
Work Experience
Technical Lead
ShareChat
- Led the ads data team at ShareChat's monetization department.
- Orchestrated over 200 feature engineering pipelines using Airflow.
- Scaled pipeline bottlenecks using Serverless Spark on GCP.
- Created datasets on BigQuery housing ad relevance-related features spanning around 1TB/day.
- Built a real-time pipeline using GCP Pub/Sub with BigQuery subscription to support storage of ad relevance service logs.
Principal Data Engineer
Noodle.ai
- Built a data lake on AWS S3 for batch processing and feature engineering.
- Used SQL Server and PostgreSQL to build multiple data-driven applications.
- Created a DQA library using Spark to analyze batch data and provide quality insights.
- Worked with Python and Airflow to create and orchestrate pipelines to sync data from client systems.
- Re-implemented the existing batch ingestion and feature generation architecture of time series data to make it more reusable across clients.
Senior Data Engineer
UnitedHealthcare
- Engineered appropriate solutions to avoid future roadblocks and loss of revenue due to government-imposed penalties to the company.
- Built optimized frameworks for existing projects to improve debugging and monitoring capabilities using customized logging and notifications.
- Analyzed project gaps and emphasized automation, leading to better team resource utilization.
Business Intelligence Analyst
NTT Data
- Built ETL pipelines that source data from multiple sources and transform them into a central golden copy at the application data warehouse.
- Generated reports to provide information at multiple hierarchal levels.
- Gained the client's trust and worked with highly confidential data.
Experience
Production Flow AI
Data engineering is involved in designing the database schema used by the app, building pipelines to ingest and supply data to various components, batch tracking, and building procedures that APIs interact with.
Education
Bachelor's Degree in Computer Science
RNS Institute of Technology - Bangalore, India
Certifications
Advanced Certification in Software Engineering for Cloud, IoT and Blockchain
IIT Madras and Greatlearning
Big Data Expert
Edureka
Skills
Tools
Apache Airflow, PyCharm, Docker Hub, GitHub, Bitbucket, pgAdmin, Google Cloud Dataproc, Jira
Languages
SQL, Python 3, Python, C#, Solidity
Paradigms
ETL, Database Design, Dimensional Modeling
Storage
Databases, PostgreSQL, SQL Server 2016, Relational Databases, Microsoft SQL Server, SQL Server Integration Services (SSIS), SQL Server Reporting Services (SSRS), Google Cloud Storage, JSON, RDBMS, HDFS, Apache Hive, Teradata, IBM Db2, SQL Server 2012, Amazon S3 (AWS S3), NoSQL
Platforms
Google Cloud Platform (GCP), Docker, Ubuntu 16.04, Jupyter Notebook, Oracle, Amazon Web Services (AWS), AWS IoT, Blockchain
Frameworks
Spark, Hadoop, .NET Core
Other
Data Engineering, Data Analytics, Programming, Database Schema Design, Database Optimization, Google BigQuery, ELT, Big Data, Big Data Architecture, Project Lifecycle, Windows 10, Data Warehousing, Data Visualization, Dashboards, Reporting, Data-driven Dashboards, Google Pub/Sub
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring