Avinash Holla Pandeshwar
Verified Expert in Engineering
BI Analyst, Data Engineer, and Developer
Avinash has close to nine years of data engineering experience and a strong background in SQL, data modeling, ETL/ELT, and big data technologies such as Hive and Spark. He currently works as a technical lead in one of India's most popular social media apps. Avinash codes in Python and uses Apache Airflow to orchestrate data pipelines. He is a dedicated and self-driven professional who strives for excellence.
PostgreSQL, SQL Server 2016, Python 3, Apache Airflow, Apache Hive, Spark, Google Cloud Platform (GCP), Amazon Web Services (AWS), Dimensional Modeling, Google BigQuery
The most amazing...
...solutions I've engineered helped in improving revenue due to increased stability and fewer downtimes.
- Led the ads data team at ShareChat's monetization department.
- Orchestrated over 200 feature engineering pipelines using Airflow.
- Scaled pipeline bottlenecks using Serverless Spark on GCP.
- Created datasets on BigQuery housing ad relevance-related features spanning around 1TB/day.
- Built a real-time pipeline using GCP Pub/Sub with BigQuery subscription to support storage of ad relevance service logs.
Principal Data Engineer
- Built a data lake on AWS S3 for batch processing and feature engineering.
- Used SQL Server and PostgreSQL to build multiple data-driven applications.
- Created a DQA library using Spark to analyze batch data and provide quality insights.
- Worked with Python and Airflow to create and orchestrate pipelines to sync data from client systems.
- Re-implemented the existing batch ingestion and feature generation architecture of time series data to make it more reusable across clients.
Senior Data Engineer
- Engineered appropriate solutions to avoid future roadblocks and loss of revenue due to government-imposed penalties to the company.
- Built optimized frameworks for existing projects to improve debugging and monitoring capabilities using customized logging and notifications.
- Analyzed project gaps and emphasized automation, leading to better team resource utilization.
Business Intelligence Analyst
- Built ETL pipelines that source data from multiple sources and transform them into a central golden copy at the application data warehouse.
- Generated reports to provide information at multiple hierarchal levels.
- Gained the client's trust and worked with highly confidential data.
Production Flow AI
Data engineering is involved in designing the database schema used by the app, building pipelines to ingest and supply data to various components, batch tracking, and building procedures that APIs interact with.
SQL, Python 3, Python, C#, Solidity
ETL, Database Design, Dimensional Modeling
Databases, PostgreSQL, SQL Server 2016, Relational Databases, Microsoft SQL Server, SQL Server Integration Services (SSIS), SQL Server Reporting Services (SSRS), Google Cloud Storage, JSON, RDBMS, HDFS, Apache Hive, Teradata, IBM Db2, SQL Server 2012, Amazon S3 (AWS S3), NoSQL
Data Engineering, Data Analytics, Programming, Database Schema Design, Database Optimization, Google BigQuery, ELT, Big Data, Big Data Architecture, Project Lifecycle, Windows 10, Ubuntu 16.04, Data Warehousing, Data Visualization, Dashboards, Reporting, Data-driven Dashboards, Google Pub/Sub
Google Cloud Platform (GCP), Docker, Jupyter Notebook, Oracle, Amazon Web Services (AWS), AWS IoT, Blockchain
Spark, Hadoop, .NET Core
Apache Airflow, PyCharm, Docker Hub, GitHub, Bitbucket, pgAdmin, Google Cloud Dataproc, Jira
Bachelor's Degree in Computer Science
RNS Institute of Technology - Bangalore, India
Advanced Certification in Software Engineering for Cloud, IoT and Blockchain
IIT Madras and Greatlearning
Big Data Expert
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.Start hiring