Gopal Krishna Ranjan, Developer in Greater Noida, India
Gopal is available for hire
Hire Gopal

Gopal Krishna Ranjan

Verified Expert  in Engineering

Bio

Gopal is a Microsoft-certified Azure data engineer experienced in Spark, Scala, Python, Azure, HDFS, MapReduce, Hive, and AWS. He is skilled in data warehouse implementation and database design, including ETL/ELT with Azure Data Factory and Apache Airflow and CI/CD pipelines with Docker and Jenkins. Gopal is also an expert in data analysis using statistical techniques, writing complex SQL queries, and object-oriented programming such as Scala, Python, C#, and Java.

Portfolio

Nagarro
PySpark, Apache Hive, Big Data, Machine Learning, Python, Apache Spark...
CoForge (formerly NIIT Technologies)
C#, Data Warehousing, Data Warehouse Design, ETL, ELT
NSD TECH
C#, T-SQL (Transact-SQL)

Experience

  • Data Engineering - 12 years
  • Python - 7 years
  • Big Data - 6 years
  • Apache Spark - 6 years
  • Azure Databricks - 6 years
  • Azure - 3 years
  • Scala - 3 years
  • PySpark - 2 years

Availability

Part-time

Preferred Environment

Big Data, Apache Hive, Apache Spark, Hadoop, Azure, Python, Apache Airflow, Data Engineering, Java, Scala

The most amazing...

...thing I've built is a generalized appraisal system using dynamic XML mapping. HR staff only need to define the rules and start the system.

Work Experience

Associate Team Lead

2014 - PRESENT
Nagarro
  • Automated data pipelines in Azure using Azure Data Factory, Python, Scala, and Airflow. Extensively worked on Hadoop, HDFS, Apache Spark, PySpark, Scala, Java, and other big data tools to solve complex data engineering problems.
  • Performed data analysis and data wrangling extensively with the Azure Databricks and Azure Data Lake platforms. Used Spark SQL, PySpark, and Scala for data analysis.
  • Created complex Transact-SQL queries and participated heavily in the project's performance tuning and query optimization using Microsoft SQL Server.
  • Extracted data from different sources and performed data wrangling using NumPy and Pandas packages in Python.
  • Applied exploratory data analysis techniques to get the statistical measures of the data using Matplotlib and Seaborn in Python.
  • Identified the data distribution of the sample data set and simulated it with the help of probability distribution methods to get statistical inferences from the data set.
  • Simulated the hypotheses using bootstrapping and A/B testing.
  • Implemented HiveQL on Azure and the AWS cloud platform.
  • Developed SQL Server Analysis Services (SSAS) cubes and modified existing cubes.
  • Wrote SQL Server Reporting Services (SSRS) reports using MDX queries and implemented data engineering solutions using Scala, Java, and Python.
Technologies: PySpark, Apache Hive, Big Data, Machine Learning, Python, Apache Spark, Apache Airflow, Scala, Shell, Azure, Java, Hadoop, Azure Databricks, HDFS, Spark SQL, Azure Data Factory (ADF), Azure Data Lake, ETL Tools, Azure Cloud Services

Senior Software Engineer

2012 - 2014
CoForge (formerly NIIT Technologies)
  • Designed normalized databases for highly transactional systems.
  • Identified and optimized the performance of existing queries.
  • Improved the performance of an extensive 250-gigabyte ERP database.
  • Created and optimized complex T-SQL queries using SQL Server Management Studio (SSMS).
  • Wrote scripts to automate database maintenance activities.
  • Developed highly interactive and rich UI reports using SSRS.
  • Created data warehouse and ETL/ELT processes using the Microsoft platform.
Technologies: C#, Data Warehousing, Data Warehouse Design, ETL, ELT

Software Engineer

2011 - 2012
NSD TECH
  • Created complex database objects and queries in the SQL Server.
  • Implemented requirements using Visual Basic and Transact-SQL.
  • Identified and optimized long-running queries using indexing and other optimization techniques.
  • Developed web pages using ASP.NET, HTML, CSS, JavaScript, and C#.
Technologies: C#, T-SQL (Transact-SQL)

Junior Software Engineer

2009 - 2010
Bagmati Pulp & Paper
  • Conducted requirement gathering and analysis to implement the software.
  • Designed the user interfaces in Visual Studio 2008.
  • Contributed to database design, table creation, procedures, and function creation.
Technologies: C#

Experience

SQLRelease.com | Blog

http://www.sqlrelease.com/
I write technical blogs for SQLRelease.com. I also have created a video tutorial on a set-based approach in an SQL Server.

Target Analytics | Project

We have a large database so we apply exploratory data analysis techniques and then create predictive models using inferential statistics. We identify the machine learning problems, collect the data from various sources, apply data wrangling using Python/T-SQL, and deploy the models to the Azure ML.

IT Managed Services | Project

I extracted data from various data sources and RESTful APIs and loaded the data into DWH by applying data modeling. Then I created OLAP cubes for analysis and reporting purposes and finally wrote efficient T-SQL code for analysis purposes and data wrangling.

Internationalization of BI Reports | Project

I modified the existing DWH to support global reporting features which include the addition and changes in database and data warehouse design, cubes, ETL packages, C#, SSRS reports, and a MDX query.

Enterprise Resource Planning | Project

A system being used by human resources (HR) and administrators. The project involved the development of various modules including travel management, attendance management, a rule-based CTC restructuring tool, carbon footprint reporting, and more. It involved using SQL Server 2008R2, 2012, SSRS, SSIS, C#, and ASP.NET.

Fleximon | Project

The client deals in areas of equity, commodity, and currency. The project involved the creation of various modules using SQL Server and C# and using VB.NET to fetch the data from an SQL Server, then writing it into Excel files, and finally, sending them to the defined subscribers. For the project, I also developed a web portal using ASP.NET and C#. It involved using SQL Server 2005 and 2008, ASP.NET, VB.NET, and C#.

Inventory Management System | Project

For this project, the client was involved in the paper industry and they wanted me to develop an inventory management system that was also a repository for all the information related to goods of the company at multiple locations. The project involved using Visual Studio, C#, and SQL Server 2005.

Education

2007 - 2010

Master of Science Degree in Computer Applications (Computer Science)

Sikkim Manipal University - Siliguri, India

2001 - 2003

Bachelor of Arts Degree in Economics

Veer Bahadur Singh Purvanchal University - Ballia, India

Certifications

APRIL 2023 - APRIL 2024

Microsoft Certified: Azure Data Engineer Associate

Microsoft

Skills

Libraries/APIs

PySpark

Tools

Apache Airflow, Shell, Spark SQL

Languages

Python, T-SQL (Transact-SQL), C#, Java, Scala, SQL

Paradigms

Database Design, ETL, Dimensional Modeling

Platforms

Azure, Amazon Web Services (AWS), Apache Kafka

Frameworks

Apache Spark, Hadoop

Storage

Apache Hive, Databases, Database Modeling, HDFS, Azure Cloud Services

Other

Data Modeling, Big Data, Machine Learning, Data Engineering, Azure Databricks, Data Warehousing, ELT, Azure Data Lake, Azure Data Lake Analytics, Data Analysis, Analytics, Data Warehouse Design, Data Architecture, Azure Data Factory (ADF), ETL Tools

Collaboration That Works

How to Work with Toptal

Toptal matches you directly with global industry experts from our network in hours—not weeks or months.

1

Share your needs

Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2

Choose your talent

Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3

Start your risk-free talent trial

Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.

Top talent is in high demand.

Start hiring