Harish Chander Ramesh
Verified Expert in Engineering
Data Engineer and Developer
Harish is a data engineer who has been consuming, engineering, analyzing, exploring, testing, and visualizing data for personal and professional purposes for the last ten years. His passion for data has led him to work with multiple Fortune 50 organizations, including Amazon and Verizon. Harish loves challenges and believes he can learn and deliver best when out of his comfort zone.
Portfolio
Experience
Availability
Preferred Environment
Google Cloud Platform (GCP), Tableau, Microsoft Power BI, SQL, ETL, Business Intelligence (BI), Data Visualization, Amazon Web Services (AWS), Google BigQuery, Azure SQL Databases, Data Engineering, AWS Data Pipeline Service, Data Management, Collibra, Informatica Cloud, Informatica ETL, Informatica, Oracle, JavaScript, Data Architecture, Excel 365, CSV File Processing, Excel VBA, Data Extraction, MySQL
The most amazing...
...data platform I've built from scratch is for a video conferencing app, which managed to have no downtime despite the 600% usage increase during the pandemic.
Work Experience
Data Engineer Manager
MH Alshaya
- Developed the first-ever Data warehouse from scratch, incorporating product analytics at scale, using various GCP services.
- Developed the Golden Customer Record in real-time, extending the Loyalty program of 119 brands over 19 countries.
- Developed and maintained a data quality framework with the help of the entire business team in-house, using Great Expectations at scale. This was also used in fraud analytics across 50+ brands in near real-time.
- Led a team of six data engineers, the first set of data engineers in the organization, and started up a data-driven culture within the team.
Lead Data Engineer
Verizon Media
- Developed the first streaming analytics platform to handle media stats from videoconferencing solutions using Apache Spark and Storm on AWS-managed services.
- Built a data pipeline that autoscaled itself, not experiencing the impacts of the COVID-19 pandemic despite the 600% increase in the daily usage volume due to remote work implementation among clients’ teams.
- Tested and implemented Apache Hudi at its early stages of development, also providing ACID transactions the ability on historical data.
- Led a team of seven data engineers, three seniors, two juniors, and one intern. Created opportunities to interact with large clients worldwide on technical solution consultation and solution architecting.
- Migrated a live legacy database of PostgreSQL to Snowflake with DBT on the process with a size of 2.2 PB in five days. Designed, implemented, and validated the migration on the fly with the help of an error reporting framework with 0.3% of errors.
Data Engineer
Amazon
- Contributed to the world's largest eCommerce platform covering 16 marketplaces across the globe in different timezones. I was a part of the retail business team that handled the worldwide retail business data management and pipelines.
- Managed to handle high-pressure environments and meet tight deadlines. Worked alongside the best minds in the country and the world, initiating a data engineer forum within the organization for cross-polination of ideas among us.
- Built real-time pipelines to stream data from different platforms to the Amazon data warehouse with a service-level agreement (SLA) of a 2-minute time delay using Spark, Flink, and Tableau.
- Created a 360-degree dashboard with perspectives on Amazon's customers across different Amazon services. The dashboard was made public on a forum and gained massive popularity for the ease of data understanding by consumers.
Data Engineer
NTT Data
- Developed, tested, and deployed end-to-end real-time and Batch ETL pipelines for a healthcare provider.
- Documented every line of code and changes to the existing product from a business standpoint.
- Learned new technologies with an open-minded approach and grew as an agnostic developer.
- Developed two major data warehouse-related projects to save 23% of data storage cost and 26.5% of maintenance cost.
Experience
Competitive Price Monitoring System for eCommerce Business
Real-time Pipelines for Fraud Alerting
Driver's incentives Framework
Skills
Languages
SQL, Python, Snowflake, JavaScript, Excel VBA
Frameworks
Apache Spark, Spark, Storm, Hadoop, Django
Tools
Apache Airflow, Tableau, Microsoft Power BI, Abinitio, Kafka Streams, BigQuery, Collibra, Informatica ETL, Excel 2016, AWS Glue, ELK (Elastic Stack), Microsoft Access, pgAdmin, Amazon QuickSight, Amazon Elastic Container Service (Amazon ECS), Amazon CloudFront CDN, AWS CloudFormation, Google Analytics, Apache Storm, Logstash, Grafana, Terraform, Looker
Paradigms
ETL, Business Intelligence (BI), ETL Implementation & Design, Database Development, Data Science, Microservices
Platforms
Google Cloud Platform (GCP), Amazon EC2, Amazon Web Services (AWS), Firebase, AWS Lambda, Apache Flink, Azure, Oracle, Docker, Apache Kafka, Cloud Native
Storage
Teradata, Redshift, Databases, Amazon S3 (AWS S3), Data Pipelines, Data Lake Design, PostgreSQL, Azure SQL Databases, AWS Data Pipeline Service, MongoDB, Microsoft SQL Server, Database Architecture, Database Performance, Datadog, Data Lakes, Google Cloud, Oracle Cloud, MySQL, MemSQL, Elasticsearch
Other
Software, Dashboards, Data Visualization, Amazon RDS, Big Data, Data Warehouse Design, Data Warehousing, Data Engineering, Google BigQuery, Data Analysis, Cloud Platforms, Data Management, Informatica Cloud, Informatica, Data Architecture, Excel 365, Office 365, CSV File Processing, Data Migration, Data Extraction, ELT, Technical Architecture, ETL Tools, Big Data Architecture, Data Modeling, Analytics, Data Analytics, Data Governance, Parquet, Database Schema Design, Fivetran, Airbyte, Azure Synapse, TIBCO, Ads, Data Quality, Finance, Mobile Analytics, Monitoring, Data Build Tool (dbt), User Interface (UI), Great Expectations Cloud
Libraries/APIs
PySpark, Spark Streaming
Education
Bachelor of Engineering Degree in Electronics
Anna University - Chennai, India
Certifications
Google Cloud Certified - Professional Data Engineer
Google Cloud