Mohamed Ahmed
Verified Expert in Engineering
Big Data Architect and Back-end Developer
Berlin, Germany
Toptal member since November 4, 2022
Mohamed is a big data platform architect with 15 years of experience in the IT industry. He excels with distributed systems, data engineering, machine learning, and DevOps. Mohamed builds robust batch and real-time data platforms for stakeholders and moves companies to state-of-the-art data platforms. He designed GDPR-compliant data lakes and optimized an ETL pipeline, which saved the client hundreds of thousands of dollars annually. Mohamed is pragmatic and has an agile mindset.
Portfolio
Experience
- SQL - 11 years
- Data Engineering - 7 years
- Apache Spark - 7 years
- Scala - 7 years
- Apache Kafka - 5 years
- Python - 4 years
- Google Cloud Platform (GCP) - 3 years
- BigQuery - 3 years
Availability
Preferred Environment
Linux, Slack, Jira, GitHub, Google Cloud Platform (GCP), Apache Kafka, Data Engineering, Amazon Web Services (AWS), Big Data, Big Data Architecture
The most amazing...
...project I've delivered required tuning a microservice to serve over 100,000 requests per second in 20 milliseconds per request.
Work Experience
Big Data Architect
Mobile.de
- Designed, led, and guided the company data platform migration from on-premises infrastructure to public cloud (GCP) and saved 30% of stakeholders' daily business hours.
- Built and designed a data lake that complies with GDPR requirements with minimal effect on downstream users.
- Planned and monitored the GCP budget across dozens of projects, which resulted in an accurate spend cost.
Staff Data Engineer
Careem Networks FZ
- Developed an ETL framework that automates data processing pipelines and runs hundreds of ETL daily.
- Optimized an ETL pipeline, which saved hundreds of thousands of dollars yearly.
- Established a data academy to help full-stack engineers grow their data engineering knowledge.
Senior Software Engineer and Big Data
Searchmetrics gmbh
- Designed and developed a challengeable data pipeline for billions of messages and records.
- Devised the ETL framework to work with many sources and sinks.
- Presented new technologies and discussed them with my team.
Senior Software Engineer and Big Data
Agoda
- Developed and tuned recommendation microservices to accept millions of requests per second with a success rate of 99.99 in 20 milliseconds for the whole request trip.
- Designed and developed a reactive DAG framework to build any logical flow over Akka actors and futures.
- Assisted the data scientist team with ETL pipelines to apply ML offline training.
Back-end Specialist
CIT global
- Developed a logging service that tracks all app actions on MongoDB with AspectJ.
- Developed an e-payment workflow using Mule ESB that controls payment steps.
- Created a wallet payment microservice that transfers payments across bank accounts.
Senior Java Developer
E-Finance
- Developed back-end and front-end payment services using multiple frameworks; ADF, Struts, and ICEfaces.
- Created business reports using the Jasper Reporting tool.
- Built and automated administration pages created from a DB ER diagram.
Service Information Developer
HP Inc
- Built the endpoint of the sales (EPOS) client app validator using Servlet and JSP, which can validate big XML files and return invalid tags.
- Wrote a user tutorial that guided users to new features and increased the customer acceptance rate.
- Contributed to the internal development community that helped new users get familiar with internal tools.
Java Developer
Networks Valley
- Created a custom payroll desktop app that handled complicated payroll logic and generated company payroll reports.
- Devised an innovative home service that monitored smart homes and sent mobile notifications to homeowners.
- Built a PCL interface app that controlled devices in an electricity plant.
Experience
On-premises to Public Cloud (GCP) Migration
I collected and discussed pain points with stakeholders, created general architecture ADR, reviewed the new design with my team and stakeholders, and collected feedback. Next, I estimated the budget and discussed it with the head of technology, removed obstacles to implementation, and modified open-source frameworks to fit our needs; for example, I added a new feature to the Atlas data catalog framework to support delta-lake. I reviewed the road map with my team and broke it down into epics and parallel stories, jumped in to help when blocks arose, and discussed best practices with different teams in the company from a data point of view.
Real-time Analytics Service
I designed a real-time solution that fulfilled stakeholders' requirements, tuned the reader service to achieve <100 milliseconds latency in the 99.99 success rate percentile, and introduced network solutions as the project ran in a hybrid cloud environment.
Building Marketplace Data Platform
Keyword Ranking
I created a changeable data pipeline for billions of messages and records and designed the ETL framework to work with many sources and sinks. I presented new technologies and discussed them with the team. I tuned the jobs to fit our cluster, reviewed the code, and took ownership of the project.
Reactive Framework (Jarvis)
Hotel Recommendation
Bidding Channel ROI Manager
Education
Bachelor's Degree in Electrical Engineering
Fayoum University - Egypt
Certifications
Algorithms on Graphs
Coursera
Deep Learning Specialisation
Coursera
Data Structures
Coursera
Algorithmic Toolbox
Coursera
OCE Java EE 6 EJB 3.x (1Z0-895)
Oracle
OCE Java EE 6 Web Service (1Z0-897)
Oracle
OCE Java Persistence API 2.0 - EE 6 (1Z0-898)
Oracle
Sun Certified Web Component Developer SCWCD 5 (310-083)
Sun Microsystems
Sun Java5 Certified SCJP 5 (310-055)
Sun Microsystems
Skills
Libraries/APIs
PySpark, JPA 2, Spark Streaming, JAX-WS, JAX-RPC, Pandas, Cloud Key Management Service (KMS), Quartz, Java API for XML Processing (JAXP)
Tools
Apache Airflow, Apache Maven, Kafka Streams, BigQuery, Slack, Jira, GitHub, Apache Zeppelin, Servlet, Amazon Elastic MapReduce (EMR), Oozie, RabbitMQ, Apache Beam, Amazon EKS, AWS Glue, Amazon Virtual Private Cloud (VPC)
Languages
Java, Scala, SQL, HTML, Python, Snowflake, C, C#, VHDL, Pascal, Stored Procedure, Go
Frameworks
Apache Spark, Spark, Hadoop, Akka, Presto, Spring Boot, Hibernate, Oracle ADF, Apache Struts, Django
Paradigms
ETL, Data-driven Design, DevOps, Microservices
Platforms
Google Cloud Platform (GCP), Apache Kafka, Amazon Web Services (AWS), Docker, Linux, Jupyter Notebook, Apache Flink, Oracle Database, AWS Lambda, Amazon EC2, Kubernetes, Cloud Run, Databricks
Storage
Data Pipelines, NoSQL, Google Cloud, Data Lake Design, Google Cloud Storage, Google Cloud SQL, MySQL, PostgreSQL, Apache Hive, MongoDB, Oracle PL/SQL, Microsoft SQL Server, HDFS, Amazon S3 (AWS S3), Redshift, Data Integration, Amazon DynamoDB, Database Architecture, Data Validation, Database Modeling
Other
Data Processing, General Data Protection Regulation (GDPR), Data Engineering, ELT, Data Architecture, Big Data, Big Data Architecture, Architecture, Solution Architecture, Google BigQuery, Cloud Migration, Data Migration, Data Analysis, Large-scale Data Migration, ETL Tools, Ajax, Web Services, Algorithms, Data Structures, Graph Algorithms, Delta Lake, Google Cloud Functions, Catalog Data Entry Services, Apache Cassandra, Apache Livy, EJB3, Atlas, Pub/Sub, Data Modeling, User-defined Functions (UDF), Data Governance, Data Strategy, APIs, EMR, RESTful Microservices, AWS Certified Solution Architect, Back-end Development, Machine Learning Operations (MLOps), Security, Back-end, Microprocessors, Microcontroller Programming, Java Card OpenPlatform (JCOP), Deep Learning, Apache Flume, Technical Program Management, Data Build Tool (dbt), NVivo, Infrastructure as Code (IaC), Amazon Kinesis, Machine Learning
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring