Wagdy El-Sherbiny
Verified Expert in Engineering
Data and Database Developer
Wagdy is a senior data and database engineer with over eight years of experience with data-related and real-time mega projects. Highly proficient in SQL, databases, and data warehouses, including designing and implementing, he has managed software projects and worked for significant clients, such as ADNOC, Emarat, KNPC, and KOC. Wagdy is an expert in data pipelines, data analysis, and Python and focuses on understanding clients' needs and designing and developing state-of-art solutions.
Portfolio
Experience
Availability
Preferred Environment
Data Engineering, Databases, Data Warehouse Design, Python, Data Analysis, NoSQL, SQL, PostgreSQL, MongoDB, ETL Tools
The most amazing...
...thing I've developed is the self-service payment and operation station used by ADNOC in the UAE, the world's 12th largest oil company by production.
Work Experience
Python and PL/SQL Developer
Susquehanna Technology Management, Inc. (STMI) - Main
- Implemented a Python program to replace a legacy system, drastically reducing processing time for calculating trading volumes per symbol and exchange messages. Reduced processing duration from 7 to 10 hours to approximately 20 to 30 minutes.
- Engineered a Python script to extract data from URLs, parse JSON information, and seamlessly store it within an SQL server database for efficient data management.
- Created a shell script designed to retrieve data from the FIDO trading system. Implemented filtering to extract essential information, saving it into a CSV file. Utilized external tables to facilitate the loading of data into Oracle tables.
- Migrated all legacy Python scripts initially written in Python 2 to the latest Python 3.7 version, ensuring compatibility and optimization for current development standards.
- Managed bug tracking and resolution processes utilizing Jira, ensuring seamless production support and maintenance.
- Crafted a robust database logging and debugging utility for streamlined troubleshooting. Also, designed a Python module for easy integration into scripts, bolstering debugging capabilities.
- Revamped and reimplemented outdated systems prone to failure when processing large files with efficient and swift Python programs.
Python and PL/SQL Developer
SIG - Main
- Developed a regression testing program that validates the outcomes of one of the leading trading solutions at SIG. That application was handy for top management to validate the data outcome.
- Built a real-time data pipeline that monitors three trading symbols' order trades and extracts data as of the ACT messages protocol. The script extracts and parses data and then loads it into the Oracle database.
- Participated in PySpark script development, converting legacy reports into new ones that optimized execution time and processed the vast amount of daily data, 500+ million transactions.
- Developed an ETL, where parsing XML files loads data into an Oracle database table.
- Created many reports using SSRS and a modern visualization dashboard using Qlik Sense.
- Converted a legacy Perl program into Python. The scripts aim to process trading orders and calculate the five-minute sliding average for each symbol.
Senior Data Engineer
Diyar United Company
- Collaborated with the project management team to analyze and consolidate request requirements to improve the decision-planning process and meet project deadlines.
- Led medium projects from scope definition to execution and implementation.
- Worked with cross-functional teams to structure problems, identify appropriate data sources, extract data, and develop integrated information delivery solutions.
- Designed, implemented, and maintained various database management systems (DBMSs), such as Oracle, Microsoft SQL Server, Microsoft Azure SQL, and PostgreSQL.
- Analyzed and compiled an itemized list of components needed for data migration or integration with various systems, such as Oracle Enterprise Resource Planning (ERP), SAP, and Comarch Loyalty Management (CLM).
- Participated in designing and implementing a data warehouse, data marts for analysis, data mining projects, and implementing changes to existing BI systems.
- Used the SSIS tool to gather data from various resources, followed the data-wrangling cycle to store and deliver cleaned data for data analysis, and developed a customized pipeline for data ingestion and loading from an external system.
- Built a fast and reliable invoicing system that efficiently processes millions of transactions and integrates with the Oracle ERP or SAP system. Reduced the invoice generation time from six hours to 20 minutes.
- Identified the key reports and dashboards to satisfy the client's requirements and developed different kinds of reports like parameterized reports, ad hoc reports, and drill down and drill through reports using SSRS and Power BI.
- Troubleshooted data issues, analyzed and corrected error and performance logs, created backup routines, and prepared system deployments and upgrades. Trained various business unit teams on effectively using processes, tools, and resources.
Full-stack Developer
Management Development Center (MDC)
- Participated in developing the Rhazes solution that includes all the medical, clinical, and administrative modules needed to manage and operate a hospital in a smooth and user-friendly way.
- Developed SCHOOLIZER software used to efficiently manage all the systems and processes related to running any educational institute.
- Troubleshot and fixed the production issues related to an e-learning system called MUST.
Teaching Assistant
Misr University for Science and Technology
- Managed, planned, and prepared the learning materials and courses related to computer science for different classes and levels.
- Used different educational methods to promote a positive learning environment and organized group work projects and brainstorming sessions.
- Set requirements for the different assessments and kept records of the students' development.
- Kept up to date with changes and developments in the field of computer science.
Experience
ADNOC Smart Wallet
https://smart.adnocdistribution.ae/I was the software engineer who participated in and developed all objects related to the data area, including databases such as triggers, stored procedures, jobs, data warehouse, pipelines, and reporting. I created a set of systems, including the following:
• A card management system (CMS).
• A comprehensive centralized software solution providing our clients with a user-friendly interface that integrates tightly with other back-end and external systems, such as customer management, beneficiary management, token management, transaction management, system settings, BI reports and dashboards, differential pricing, notification management, ERP integration, loyalty system, e-vouchers, coupons, data migration, and card printing system that personalizes smart cards, vehicle depot system, and invoicing.
Emarat Petroleum Company Station Management Service
https://www.emarat.ae/businesses/fleet-payment-solutions• Implementing and customizing a turnkey solution to meet the company's station management and retail services needs.
• Participating in the system requirement gathering and analysis phases.
• Analyzing and compiling components needed for data migration and integration with various systems, such as Oracle ERP, NCR, and NAMOS.
• Developing a data migration workflow to move data into a new system using SSIS.
• Building a differential pricing system and NCR discounts.
• Developing various reports and dashboards using SSRS and Power BI.
Kuwait E-passport
KNPC Petrol Filling Stations
Kuwait Digital Oil Field Project
Kuwait National Guard's Fleet Management
The Ford GoBike System
https://github.com/wagdySamy/DataAnalysisWeRateDogs Analysis
Skills
Languages
SQL, Python, C#, C++, C#.NET, T-SQL (Transact-SQL), Scala, Active Server Pages (ASP), HTML, CSS, Bash
Storage
Databases, SQL Server Reporting Services (SSRS), PL/SQL, PL/SQL Developer, Oracle PL/SQL, SQL Server Integration Services (SSIS), PostgreSQL, Data Pipelines, NoSQL, MongoDB, Azure SQL, Microsoft SQL Server, HBase, Apache Hive, Database Administration (DBA), Database Modeling, SQL Server Analysis Services (SSAS)
Other
Data Warehouse Design, PL/SQL Tuning, Data Engineering, Data Analysis, ETL Tools, ELT, Data Wrangling, Big Data, Azure Data Factory, Software Engineering, Data Structures, Operating Systems, Algorithms, Windows Communication Foundation (WCF), Artificial Intelligence (AI), Neural Networks, Streaming, Machine Learning, Data Visualization, PETEX, DAX, Data, IIS, Analysis, Teamwork, Team Management, Data & Backup Management, Data Modeling, Production, Deployment, Software Troubleshooting, Troubleshooting, Education, Communication, University Teaching, Applied Mathematics, Statistics, Windows Services, Web Services, Linear Regression, Multidimensional Expressions (MDX), Star Schema, Big Data Architecture, Data Warehousing, Data Reporting, Optimization, Workflow Automation
Libraries/APIs
Pandas, Matplotlib, NumPy, Twitter API, PySpark
Tools
Microsoft Power BI, Qlik Sense, Flink, Flume, Apache ZooKeeper, SQL Server BI, Oracle ERP, Crystal Reports, Shell
Paradigms
ETL Implementation & Design, Agile, Business Intelligence (BI), MapReduce, ETL, User Acceptance Testing (UAT)
Frameworks
Hadoop, Spark, ASP.NET
Platforms
Oracle, Apache Kafka, Linux
Education
Bachelor's Degree in Computer Science
Misr University for Science and Technology - Giza, Egypt
Certifications
Spark and Python for Big Data with PySpark
Udemy
HCIA-Big Data Course
Huawei Academy
AWS Machine Learning
Udacity
Data Analysis Professional Nanodegree Program
Udacity
Data Engineer with Python
DataCamp
MongoDB Basics
MongoDB University
Data Analysis Advanced Nanodegree Program
Udacity
Career Track Data Analyst with SQL Server
DataCamp
Big Data Introduction Course
Coursera
MCPS: Microsoft Certified Professional
Microsoft
Computer Science Diploma
Mansoura University
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring