Gopal Krishna Ranjan
Verified Expert in Engineering
Data Engineer and Developer
Greater Noida, India
Toptal member since October 1, 2018
Gopal is a Microsoft-certified Azure data engineer experienced in Spark, Scala, Python, Azure, HDFS, MapReduce, Hive, and AWS. He is skilled in data warehouse implementation and database design, including ETL/ELT with Azure Data Factory and Apache Airflow and CI/CD pipelines with Docker and Jenkins. Gopal is also an expert in data analysis using statistical techniques, writing complex SQL queries, and object-oriented programming such as Scala, Python, C#, and Java.
Portfolio
Experience
- Data Engineering - 12 years
- Python - 7 years
- Big Data - 6 years
- Apache Spark - 6 years
- Azure Databricks - 6 years
- Azure - 3 years
- Scala - 3 years
- PySpark - 2 years
Preferred Environment
Big Data, Apache Hive, Apache Spark, Hadoop, Azure, Python, Apache Airflow, Data Engineering, Java, Scala
The most amazing...
...thing I've built is a generalized appraisal system using dynamic XML mapping. HR staff only need to define the rules and start the system.
Work Experience
Associate Team Lead
Nagarro
- Automated data pipelines in Azure using Azure Data Factory, Python, Scala, and Airflow. Extensively worked on Hadoop, HDFS, Apache Spark, PySpark, Scala, Java, and other big data tools to solve complex data engineering problems.
- Performed data analysis and data wrangling extensively with the Azure Databricks and Azure Data Lake platforms. Used Spark SQL, PySpark, and Scala for data analysis.
- Created complex Transact-SQL queries and participated heavily in the project's performance tuning and query optimization using Microsoft SQL Server.
- Extracted data from different sources and performed data wrangling using NumPy and Pandas packages in Python.
- Applied exploratory data analysis techniques to get the statistical measures of the data using Matplotlib and Seaborn in Python.
- Identified the data distribution of the sample data set and simulated it with the help of probability distribution methods to get statistical inferences from the data set.
- Simulated the hypotheses using bootstrapping and A/B testing.
- Implemented HiveQL on Azure and the AWS cloud platform.
- Developed SQL Server Analysis Services (SSAS) cubes and modified existing cubes.
- Wrote SQL Server Reporting Services (SSRS) reports using MDX queries and implemented data engineering solutions using Scala, Java, and Python.
Senior Software Engineer
CoForge (formerly NIIT Technologies)
- Designed normalized databases for highly transactional systems.
- Identified and optimized the performance of existing queries.
- Improved the performance of an extensive 250-gigabyte ERP database.
- Created and optimized complex T-SQL queries using SQL Server Management Studio (SSMS).
- Wrote scripts to automate database maintenance activities.
- Developed highly interactive and rich UI reports using SSRS.
- Created data warehouse and ETL/ELT processes using the Microsoft platform.
Software Engineer
NSD TECH
- Created complex database objects and queries in the SQL Server.
- Implemented requirements using Visual Basic and Transact-SQL.
- Identified and optimized long-running queries using indexing and other optimization techniques.
- Developed web pages using ASP.NET, HTML, CSS, JavaScript, and C#.
Junior Software Engineer
Bagmati Pulp & Paper
- Conducted requirement gathering and analysis to implement the software.
- Designed the user interfaces in Visual Studio 2008.
- Contributed to database design, table creation, procedures, and function creation.
Experience
SQLRelease.com | Blog
Target Analytics | Project
IT Managed Services | Project
Internationalization of BI Reports | Project
Enterprise Resource Planning | Project
Fleximon | Project
Inventory Management System | Project
Education
Master of Science Degree in Computer Applications (Computer Science)
Sikkim Manipal University - Siliguri, India
Bachelor of Arts Degree in Economics
Veer Bahadur Singh Purvanchal University - Ballia, India
Certifications
Microsoft Certified: Azure Data Engineer Associate
Microsoft
Skills
Libraries/APIs
PySpark
Tools
Apache Airflow, Shell, Spark SQL
Languages
Python, Transact-SQL (T-SQL), C#, Java, Scala, SQL
Paradigms
Database Design, ETL, Dimensional Modeling
Platforms
Azure, Amazon Web Services (AWS), Apache Kafka
Frameworks
Apache Spark, Hadoop
Storage
Apache Hive, Databases, Database Modeling, HDFS, Azure Cloud Services
Other
Data Modeling, Big Data, Machine Learning, Data Engineering, Azure Databricks, Data Warehousing, ELT, Azure Data Lake, Azure Data Lake Analytics, Data Analysis, Analytics, Data Warehouse Design, Data Architecture, Azure Data Factory (ADF), ETL Tools
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring