Wagdy El-Sherbiny, Developer in Salmiya, Hawalli Governorate, Kuwait
Wagdy is available for hire
Hire Wagdy

Wagdy El-Sherbiny

Verified Expert  in Engineering

Data and Database Developer

Location
Salmiya, Hawalli Governorate, Kuwait
Toptal Member Since
April 18, 2022

Wagdy is a senior data and database engineer with over eight years of experience with data-related and real-time mega projects. Highly proficient in SQL, databases, and data warehouses, including designing and implementing, he has managed software projects and worked for significant clients, such as ADNOC, Emarat, KNPC, and KOC. Wagdy is an expert in data pipelines, data analysis, and Python and focuses on understanding clients' needs and designing and developing state-of-art solutions.

Portfolio

Susquehanna Technology Management, Inc. (STMI) - Main
Python, SQL, PL/SQL, Linux, C#.NET, Oracle, C#, Microsoft SQL Server, Shell...
SIG - Main
SQL, Python, PL/SQL, Oracle, Linux, C#.NET, C#, Data Warehousing...
Diyar United Company
Azure SQL, C#, C#.NET, Databases, Data Engineering, Data Analysis, Agile...

Experience

Availability

Full-time

Preferred Environment

Data Engineering, Databases, Data Warehouse Design, Python, Data Analysis, NoSQL, SQL, PostgreSQL, MongoDB, ETL Tools

The most amazing...

...thing I've developed is the self-service payment and operation station used by ADNOC in the UAE, the world's 12th largest oil company by production.

Work Experience

Python and PL/SQL Developer

2023 - 2024
Susquehanna Technology Management, Inc. (STMI) - Main
  • Implemented a Python program to replace a legacy system, drastically reducing processing time for calculating trading volumes per symbol and exchange messages. Reduced processing duration from 7 to 10 hours to approximately 20 to 30 minutes.
  • Engineered a Python script to extract data from URLs, parse JSON information, and seamlessly store it within an SQL server database for efficient data management.
  • Created a shell script designed to retrieve data from the FIDO trading system. Implemented filtering to extract essential information, saving it into a CSV file. Utilized external tables to facilitate the loading of data into Oracle tables.
  • Migrated all legacy Python scripts initially written in Python 2 to the latest Python 3.7 version, ensuring compatibility and optimization for current development standards.
  • Managed bug tracking and resolution processes utilizing Jira, ensuring seamless production support and maintenance.
  • Crafted a robust database logging and debugging utility for streamlined troubleshooting. Also, designed a Python module for easy integration into scripts, bolstering debugging capabilities.
  • Revamped and reimplemented outdated systems prone to failure when processing large files with efficient and swift Python programs.
Technologies: Python, SQL, PL/SQL, Linux, C#.NET, Oracle, C#, Microsoft SQL Server, Shell, Bash, Workflow Automation, Data Pipelines, ETL, Data Engineering, Big Data, PySpark

Python and PL/SQL Developer

2022 - 2023
SIG - Main
  • Developed a regression testing program that validates the outcomes of one of the leading trading solutions at SIG. That application was handy for top management to validate the data outcome.
  • Built a real-time data pipeline that monitors three trading symbols' order trades and extracts data as of the ACT messages protocol. The script extracts and parses data and then loads it into the Oracle database.
  • Participated in PySpark script development, converting legacy reports into new ones that optimized execution time and processed the vast amount of daily data, 500+ million transactions.
  • Developed an ETL, where parsing XML files loads data into an Oracle database table.
  • Created many reports using SSRS and a modern visualization dashboard using Qlik Sense.
  • Converted a legacy Perl program into Python. The scripts aim to process trading orders and calculate the five-minute sliding average for each symbol.
Technologies: SQL, Python, PL/SQL, Oracle, Linux, C#.NET, C#, Data Warehousing, Data Engineering, Database Modeling, Data Visualization, Data Reporting, Optimization, Data Pipelines, PySpark, Big Data, Qlik Sense

Senior Data Engineer

2014 - 2022
Diyar United Company
  • Collaborated with the project management team to analyze and consolidate request requirements to improve the decision-planning process and meet project deadlines.
  • Led medium projects from scope definition to execution and implementation.
  • Worked with cross-functional teams to structure problems, identify appropriate data sources, extract data, and develop integrated information delivery solutions.
  • Designed, implemented, and maintained various database management systems (DBMSs), such as Oracle, Microsoft SQL Server, Microsoft Azure SQL, and PostgreSQL.
  • Analyzed and compiled an itemized list of components needed for data migration or integration with various systems, such as Oracle Enterprise Resource Planning (ERP), SAP, and Comarch Loyalty Management (CLM).
  • Participated in designing and implementing a data warehouse, data marts for analysis, data mining projects, and implementing changes to existing BI systems.
  • Used the SSIS tool to gather data from various resources, followed the data-wrangling cycle to store and deliver cleaned data for data analysis, and developed a customized pipeline for data ingestion and loading from an external system.
  • Built a fast and reliable invoicing system that efficiently processes millions of transactions and integrates with the Oracle ERP or SAP system. Reduced the invoice generation time from six hours to 20 minutes.
  • Identified the key reports and dashboards to satisfy the client's requirements and developed different kinds of reports like parameterized reports, ad hoc reports, and drill down and drill through reports using SSRS and Power BI.
  • Troubleshooted data issues, analyzed and corrected error and performance logs, created backup routines, and prepared system deployments and upgrades. Trained various business unit teams on effectively using processes, tools, and resources.
Technologies: Azure SQL, C#, C#.NET, Databases, Data Engineering, Data Analysis, Agile, SQL Server Reporting Services (SSRS), SQL Server Integration Services (SSIS), Microsoft Power BI, Qlik Sense, Oracle, PL/SQL, T-SQL (Transact-SQL), Microsoft SQL Server, PostgreSQL, Python, Data Pipelines

Full-stack Developer

2012 - 2014
Management Development Center (MDC)
  • Participated in developing the Rhazes solution that includes all the medical, clinical, and administrative modules needed to manage and operate a hospital in a smooth and user-friendly way.
  • Developed SCHOOLIZER software used to efficiently manage all the systems and processes related to running any educational institute.
  • Troubleshot and fixed the production issues related to an e-learning system called MUST.
Technologies: Active Server Pages (ASP), ASP.NET, Windows Services, Web Services, Crystal Reports, SQL, Microsoft SQL Server, HTML, CSS, C#

Teaching Assistant

2012 - 2014
Misr University for Science and Technology
  • Managed, planned, and prepared the learning materials and courses related to computer science for different classes and levels.
  • Used different educational methods to promote a positive learning environment and organized group work projects and brainstorming sessions.
  • Set requirements for the different assessments and kept records of the students' development.
  • Kept up to date with changes and developments in the field of computer science.
Technologies: C#, C++, Data Structures, Operating Systems, Education, Communication, University Teaching, Algorithms, Applied Mathematics, Statistics, Databases

ADNOC Smart Wallet

https://smart.adnocdistribution.ae/
ADNOC Distribution is the UAE's leading operator of retail fuel service stations. ADNOC Wallet makes ADNOC service station payments quick and easy. It's fully secure and keeps track of everyday spending at ADNOC service stations. It includes refueling, shopping, lube change, and car washes. In addition, customers can register multiple cars or beneficiaries to an account.

I was the software engineer who participated in and developed all objects related to the data area, including databases such as triggers, stored procedures, jobs, data warehouse, pipelines, and reporting. I created a set of systems, including the following:

• A card management system (CMS).
• A comprehensive centralized software solution providing our clients with a user-friendly interface that integrates tightly with other back-end and external systems, such as customer management, beneficiary management, token management, transaction management, system settings, BI reports and dashboards, differential pricing, notification management, ERP integration, loyalty system, e-vouchers, coupons, data migration, and card printing system that personalizes smart cards, vehicle depot system, and invoicing.

Emarat Petroleum Company Station Management Service

https://www.emarat.ae/businesses/fleet-payment-solutions
Emarat is a multi-channel power and energy company with a network of service stations and fuel depots across Dubai and the Northern Emirates. In this project, I worked on the following:

• Implementing and customizing a turnkey solution to meet the company's station management and retail services needs.
• Participating in the system requirement gathering and analysis phases.
• Analyzing and compiling components needed for data migration and integration with various systems, such as Oracle ERP, NCR, and NAMOS.
• Developing a data migration workflow to move data into a new system using SSIS.
• Building a differential pricing system and NCR discounts.
• Developing various reports and dashboards using SSRS and Power BI.

Kuwait E-passport

I collaborated with a team to design and implement a set of reports and dashboards for executives and operation users of Kuwait’s Ministry of the Interior e-passport system. I performed requirements analysis, made system designs, oversaw implementation, and drafted documentation. Also, I created numerous processes using Microsoft BI tools to develop and implement a data warehouse and designed an SSIS package that extracts citizen data from an SQL Server database using lookup transformation and the Foreach Loop container. Finally, I built dashboards to demonstrate the overall performance of the passport printing center.

KNPC Petrol Filling Stations

The Kuwait National Petroleum Company (KNPC) is Kuwait's national oil refining company. As a software engineer, I participated in building a turnkey solution for retail service stations from scratch to meet KNPC's business needs and operations. I participated in all of the systems development lifecycle (SDLC) to build a centralized solution that manages customers' profiles, consumption, transactions, invoicing, loyalty points, e-vouchers, pre-printed vouchers, and data integrations with Oracle ERP and NCR systems, in addition to building modules related to managing users permissions and system setting and data backup routines.

Kuwait Digital Oil Field Project

I worked on the first Digital Oil Field (DOF) project in Kuwait, using Petroleum Experts (PETEX) technology to enhance the oil well production and operations by providing frameworks and dashboards for decision-makers. I served as the first line of support for the PETEX solution, assisting petroleum engineers during daily operations, overseeing troubleshooting data issues, and helping users interpret and understand report data. I used the following software solutions: Integrated Visualisation Management (IVM), Integrated Production Modeling (IPM), IFM, PROSPER, and Model Catalogue.

Kuwait National Guard's Fleet Management

This web application manages all the Kuwait National Guard's vehicles fleet operations. As a software engineer, I oversaw gathering requirements, analyzing the client's needs, and providing detailed design documents to the team. I worked closely with the development team to design and implement the database's objects and the Windows Communication Foundation (WCF) service to provide the front-end developers with the required info for a web app. Also, I managed the production deployment activities and troubleshooting issues and conducted training and user acceptance testing (UAT) sessions for end-users.

The Ford GoBike System

https://github.com/wagdySamy/DataAnalysis
The Ford GoBike System dataset contains information about individual rides made in a bike-sharing system covering the greater San Francisco Bay area. There are approximately 183,400 records with attributes in the dataset that describe rides, including start station, end station, and duration.

WeRateDogs Analysis

I illustrated the steps and procedures used for wrangling a real-world dataset known as WeRateDogs. WeRateDogs is a humorous Twitter account that rates dogs with commentary. I collected data from different resources, such as ZIP and CSV files, and queried the Twitter API to fetch data.

Languages

SQL, Python, C#, C++, C#.NET, T-SQL (Transact-SQL), Scala, Active Server Pages (ASP), HTML, CSS, Bash

Storage

Databases, SQL Server Reporting Services (SSRS), PL/SQL, PL/SQL Developer, Oracle PL/SQL, SQL Server Integration Services (SSIS), PostgreSQL, Data Pipelines, NoSQL, MongoDB, Azure SQL, Microsoft SQL Server, HBase, Apache Hive, Database Administration (DBA), Database Modeling, SQL Server Analysis Services (SSAS)

Other

Data Warehouse Design, PL/SQL Tuning, Data Engineering, Data Analysis, ETL Tools, ELT, Data Wrangling, Big Data, Azure Data Factory, Software Engineering, Data Structures, Operating Systems, Algorithms, Windows Communication Foundation (WCF), Artificial Intelligence (AI), Neural Networks, Streaming, Machine Learning, Data Visualization, PETEX, DAX, Data, IIS, Analysis, Teamwork, Team Management, Data & Backup Management, Data Modeling, Production, Deployment, Software Troubleshooting, Troubleshooting, Education, Communication, University Teaching, Applied Mathematics, Statistics, Windows Services, Web Services, Linear Regression, Multidimensional Expressions (MDX), Star Schema, Big Data Architecture, Data Warehousing, Data Reporting, Optimization, Workflow Automation

Libraries/APIs

Pandas, Matplotlib, NumPy, Twitter API, PySpark

Tools

Microsoft Power BI, Qlik Sense, Flink, Flume, Apache ZooKeeper, SQL Server BI, Oracle ERP, Crystal Reports, Shell

Paradigms

ETL Implementation & Design, Agile, Business Intelligence (BI), MapReduce, ETL, User Acceptance Testing (UAT)

Frameworks

Hadoop, Spark, ASP.NET

Platforms

Oracle, Apache Kafka, Linux

2008 - 2012

Bachelor's Degree in Computer Science

Misr University for Science and Technology - Giza, Egypt

MARCH 2021 - PRESENT

Spark and Python for Big Data with PySpark

Udemy

NOVEMBER 2020 - PRESENT

HCIA-Big Data Course

Huawei Academy

JULY 2020 - PRESENT

AWS Machine Learning

Udacity

MAY 2020 - PRESENT

Data Analysis Professional Nanodegree Program

Udacity

MAY 2020 - PRESENT

Data Engineer with Python

DataCamp

FEBRUARY 2020 - PRESENT

MongoDB Basics

MongoDB University

JANUARY 2020 - PRESENT

Data Analysis Advanced Nanodegree Program

Udacity

AUGUST 2019 - PRESENT

Career Track Data Analyst with SQL Server

DataCamp

MARCH 2019 - PRESENT

Big Data Introduction Course

Coursera

JUNE 2017 - JUNE 2020

MCPS: Microsoft Certified Professional

Microsoft

SEPTEMBER 2014 - PRESENT

Computer Science Diploma

Mansoura University

Collaboration That Works

How to Work with Toptal

Toptal matches you directly with global industry experts from our network in hours—not weeks or months.

1

Share your needs

Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2

Choose your talent

Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3

Start your risk-free talent trial

Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.

Top talent is in high demand.

Start hiring