Mohamed Ahmed
Verified Expert in Engineering
Big Data Architect and Back-end Developer
Berlin, Germany
Toptal member since November 4, 2022
Mohamed is a senior data platform architect and staff data engineer with 19+ years of experience designing and scaling cloud data platforms for product companies. He specializes in lakehouse/medallion architectures, distributed processing, and governed data products, treating the platform as a product with clear contracts and great DX. He works fully remote with distributed teams, bridging strategy and execution as a hands-on architect and mentor.
Portfolio
Experience
- SQL - 11 years
- Data Engineering - 7 years
- Scala - 7 years
- Apache Spark - 7 years
- BigQuery - 5 years
- Apache Kafka - 5 years
- Python - 4 years
- Google Cloud Platform (GCP) - 3 years
Work Experience
Big Data Architect
mobile.de
- Architected 500+ TB multi-cloud data platform (GCP → AWS) across 25 product teams and 44 projects; designed the data lake, query engine, BI layer, and cloud cost governance with FinOps frameworks.
- Designed ML infrastructure for distributed training and inference on GPU clusters; built GDPR-compliant data privacy governance and cloud cost FinOps frameworks.
- Pioneered domain-driven data mesh architecture, built real-time/batch data infrastructure, and chaired cross-company architecture councils; established self-service platform blueprints adopted enterprise-wide.
Staff Data Engineer
Careem Networks FZ
- Developed an ETL framework that automates data processing pipelines and runs hundreds of ETL jobs daily.
- Optimized an ETL pipeline, which saved hundreds of thousands of dollars yearly.
- Established a data academy to help full-stack engineers grow their data engineering knowledge.
Senior Software Engineer and Big Data
Searchmetrics gmbh
- Designed and developed a challengeable data pipeline for billions of messages and records.
- Devised the ETL framework to work with many sources and sinks.
- Presented new technologies and discussed them with my team.
Senior Software Engineer and Big Data
Agoda
- Developed and tuned recommendation microservices to accept millions of requests per second with a success rate of 99.99 in 20 milliseconds for the whole request trip.
- Designed and developed a reactive DAG framework to build any logical flow over Akka actors and futures.
- Assisted the data scientist team with ETL pipelines to apply ML offline training.
Back-end Specialist
CIT global
- Developed a logging service that tracks all app actions on MongoDB with AspectJ.
- Developed an e-payment workflow using Mule ESB that controls payment steps.
- Created a wallet payment microservice that transfers payments across bank accounts.
Senior Java Developer
E-Finance
- Developed back-end and front-end payment services using multiple frameworks; ADF, Struts, and ICEfaces.
- Created business reports using the Jasper Reporting tool.
- Built and automated administration pages created from a DB ER diagram.
Service Information Developer
HP Inc
- Built the endpoint of the sales (EPOS) client app validator using Servlet and JSP, which can validate big XML files and return invalid tags.
- Wrote a user tutorial that guided users to new features and increased the customer acceptance rate.
- Contributed to the internal development community that helped new users get familiar with internal tools.
Java Developer
Networks Valley
- Created a custom payroll desktop app that handled complicated payroll logic and generated company payroll reports.
- Devised an innovative home service that monitored smart homes and sent mobile notifications to homeowners.
- Built a PCL interface app that controlled devices in an electricity plant.
Project History
On-premises to Public Cloud (GCP) Migration
I collected and discussed pain points with stakeholders, created general architecture ADR, reviewed the new design with my team and stakeholders, and collected feedback. Next, I estimated the budget and discussed it with the head of technology, removed obstacles to implementation, and modified open-source frameworks to fit our needs; for example, I added a new feature to the Atlas data catalog framework to support delta-lake. I reviewed the road map with my team and broke it down into epics and parallel stories, jumped in to help when blocks arose, and discussed best practices with different teams in the company from a data point of view.
Real-time Analytics Service
I designed a real-time solution that fulfilled stakeholders' requirements, tuned the reader service to achieve <100 milliseconds latency in the 99.99 success rate percentile, and introduced network solutions as the project ran in a hybrid cloud environment.
Building Marketplace Data Platform
Keyword Ranking
I created a changeable data pipeline for billions of messages and records and designed the ETL framework to work with many sources and sinks. I presented new technologies and discussed them with the team. I tuned the jobs to fit our cluster, reviewed the code, and took ownership of the project.
Reactive Framework (Jarvis)
Hotel Recommendation
Bidding Channel ROI Manager
Education
Bachelor's Degree in Electrical Engineering
Fayoum University - Egypt
Certifications
Algorithms on Graphs
Coursera
Deep Learning Specialisation
Coursera
Data Structures
Coursera
Algorithmic Toolbox
Coursera
OCE Java EE 6 EJB 3.x (1Z0-895)
Oracle
OCE Java EE 6 Web Service (1Z0-897)
Oracle
OCE Java Persistence API 2.0 - EE 6 (1Z0-898)
Oracle
Sun Certified Web Component Developer SCWCD 5 (310-083)
Sun Microsystems
Sun Java5 Certified SCJP 5 (310-055)
Sun Microsystems
Skills
Libraries/APIs
PySpark, Spark Streaming, Pandas, Cloud Key Management Service (KMS)
Tools
BigQuery, Apache Airflow, Apache Maven, Composer, Kafka Streams, Jira, GitHub, Apache Zeppelin, Amazon Elastic MapReduce (EMR), Oozie, Terraform, Apache Iceberg, Cloudera, Claude, Amazon Athena, RabbitMQ, AWS Glue, Amazon Virtual Private Cloud (VPC), dbt Cloud
Languages
Java, Scala, Python, SQL, Snowflake, Rust, Stored Procedure, Go
Frameworks
Apache Spark, Spark, Data Lakehouse, Hadoop, Akka, Presto, Spring Boot, Hibernate
Paradigms
ETL, Application Architecture, Automation, Real-time Systems, Role-based Access Control (RBAC), Data-driven Design, DevOps, Microservices
Platforms
Google Cloud Platform (GCP), Apache Kafka, Amazon Web Services (AWS), Databricks, Docker, Kubernetes, Linux, Jupyter Notebook, Apache Flink, Oracle Database, AWS Lambda, Amazon EC2, Cloud Run, Airbyte
Storage
Data Pipelines, NoSQL, Google Cloud, Data Lake Design, Google Cloud Storage, Data Lakes, Databases, Google Cloud SQL, MySQL, PostgreSQL, Apache Hive, MongoDB, Oracle PL/SQL, Microsoft SQL Server, Amazon S3 (AWS S3), Data Integration, Amazon DynamoDB, Database Architecture, Data Validation, Distributed Databases, Database Modeling
Other
Delta Lake, Data Processing, General Data Protection Regulation (GDPR), Data Engineering, ELT, Data Architecture, Big Data, Big Data Architecture, Architecture, Solution Architecture, Data Build Tool (dbt), Google BigQuery, Cloud Migration, Data Migration, Data Analysis, Large-scale Data Migration, ETL Tools, Distributed Systems, Software Architecture, Performance Optimization, ETL Pipelines, Data Orchestration, Dataproc, Medallion Architecture, Scalability, Data Transformation, Orchestration, Enterprise, DataOps, Systems Design, Technical Documentation, Cloud Architecture, Technical Leadership, Technology Management, Analytics, Data Analytics, Web Services, Algorithms, Data Structures, Graph Algorithms, Google Cloud Functions, Apache Cassandra, Apache Livy, Pub/Sub, Data Modeling, User-defined Functions (UDF), Data Governance, Data Strategy, APIs, EMR, RESTful Microservices, AWS Certified Solution Architect, Back-end Development, Machine Learning Operations (MLOps), Security, Back-end, Real-time Data, CI/CD Pipelines, Database Schema Design, Data Warehousing, Enterprise Resource Planning (ERP), Amazon Redshift, Streaming, Full-stack Development, Artificial Intelligence (AI), LLM Integration, Web Scraping, AI Engineering, Apache Flume, Technical Program Management, Infrastructure as Code (IaC), Amazon Kinesis, Machine Learning, CTO, Agentic AI, AI Agents, Large Language Models (LLMs), Geospatial Data, Human-in-the-loop (HITL)
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring