Big Data Architect2020 - PRESENTMobile.de
Technologies: Apache Spark, Scala, Apache Kafka, BigQuery, Google Cloud Platform (GCP), Python, Delta Lake, Data Processing, Kafka Streams, GDPR, Apache Airflow, Google Cloud SQL, Google Cloud Functions, Kubernetes, Catalog Data Entry Services, Apache Cassandra, PostgreSQL, Cloud Run, Apache Flume, Linux, Spark Streaming, MongoDB, Data Engineering, Data Pipelines, ETL, ELT, SQL, Jira, GitHub, Redshift, Pub/Sub, Data Modeling, Data Architecture, Spark, User-defined Functions (UDF), Data Governance, Data Strategy, Big Data
- Designed, led, and guided the company data platform migration from on-premises infrastructure to public cloud (GCP) and saved 30% of stakeholders' daily business hours.
- Built and designed a data lake that complies with GDPR requirements with minimal effect on downstream users.
- Planned and monitored the GCP budget across dozens of projects, which resulted in an accurate spend cost.
Staff Data Engineer2018 - 2020Careem Networks FZ
Technologies: Apache Spark, Apache Airflow, Scala, Python, Apache Livy, Jupyter Notebook, Apache Zeppelin, Amazon Elastic MapReduce (EMR), Apache Hive, Presto DB, Kafka Streams, Data Processing, Linux, Data Engineering, Data Pipelines, ETL, ELT, Amazon S3 (AWS S3), Data Structures, SQL, Deep Learning, Jira, GitHub, AWS EMR, Data Modeling, Data Architecture, Spark, User-defined Functions (UDF), Big Data, Pandas, Amazon Web Services (AWS), AWS Lambda
- Developed an ETL framework that automates data processing pipelines and runs hundreds of ETL daily.
- Optimized an ETL pipeline, which saved hundreds of thousands of dollars yearly.
- Established a data academy to help full-stack engineers grow their data engineering knowledge.
Senior Software Engineer and Big Data2017 - 2018Searchmetrics gmbh
Technologies: Amazon Web Services (AWS), RabbitMQ, Apache Spark, Apache Zeppelin, Java, Apache Kafka, Spark Streaming, Apache Flink, MySQL, Hadoop, Apache Sqoop, Apache Hive, Scala, Spring Boot, Data Processing, Linux, Data Engineering, Data Pipelines, ETL, ELT, Amazon S3 (AWS S3), Data Structures, Amazon Elastic MapReduce (EMR), SQL, Jira, GitHub, AWS EMR, Data Modeling, Spark, User-defined Functions (UDF), Big Data
- Designed and developed a challengeable data pipeline for billions of messages and records.
- Devised the ETL framework to work with many sources and sinks.
- Presented new technologies and discussed them with my team.
Senior Software Engineer and Big Data2015 - 2017Agoda
Technologies: Apache Spark, Hadoop, Scala, Akka, spray, Data Processing, Apache Cassandra, PostgreSQL, Linux, Data Engineering, Data Pipelines, ETL, ELT, Data Structures, SQL, Jira, Django, HDFS, Data Modeling, Spark, User-defined Functions (UDF), Big Data
- Developed and tuned recommendation microservices to accept millions of requests per second with a success rate of 99.99 in 20 milliseconds for the whole request trip.
- Designed and developed a reactive DAG framework to build any logical flow over Akka actors and futures.
- Assisted the data scientist team with ETL pipelines to apply ML offline training.
Back-end Specialist2014 - 2015CIT global
Technologies: Java, Mule ESB, ActiveMQ, Spring, MongoDB, JAX-WS, EJB3, Hibernate, Oracle Database, AspectJ, SQL, Apache Cassandra, Stored Procedure
- Developed a logging service that tracks all app actions on MongoDB with AspectJ.
- Developed an e-payment workflow using Mule ESB that controls payment steps.
- Created a wallet payment microservice that transfers payments across bank accounts.
Senior Java Developer2010 - 2014E-Finance
- Developed back-end and front-end payment services using multiple frameworks; ADF, Struts, and ICEfaces.
- Created business reports using the Jasper Reporting tool.
- Built and automated administration pages created from a DB ER diagram.
Service Information Developer2009 - 2010HP Inc
Technologies: Servlet, JSP, JAX-RPC, Java API for XML Processing (JAXP), Hibernate, Ant, Ajax, Web Services, JAX-WS
- Built the endpoint of the sales (EPOS) client app validator using Servlet and JSP, which can validate big XML files and return invalid tags.
- Wrote a user tutorial that guided users to new features and increased the customer acceptance rate.
- Contributed to the internal development community that helped new users get familiar with internal tools.
Java Developer2007 - 2009Networks Valley
- Created a custom payroll desktop app that handled complicated payroll logic and generated company payroll reports.
- Devised an innovative home service that monitored smart homes and sent mobile notifications to homeowners.
- Built a PCL interface app that controlled devices in an electricity plant.