Big Data Architect and Back-end Developer in Berlin, Germany

We experienced various limitations in the private cloud, so we decided to move to the public cloud (GCP). I took the initiative to design, lead, and build migration across the company and be an excellent example for new data platform infrastructure.

I collected and discussed pain points with stakeholders, created general architecture ADR, reviewed the new design with my team and stakeholders, and collected feedback. Next, I estimated the budget and discussed it with the head of technology, removed obstacles to implementation, and modified open-source frameworks to fit our needs; for example, I added a new feature to the Atlas data catalog framework to support delta-lake. I reviewed the road map with my team and broke it down into epics and parallel stories, jumped in to help when blocks arose, and discussed best practices with different teams in the company from a data point of view.

This real-time analytics tool extracts user-tracking metrics from event streams depending on configurable input using Kafka and Spark-streaming frameworks.

I designed a real-time solution that fulfilled stakeholders' requirements, tuned the reader service to achieve <100 milliseconds latency in the 99.99 success rate percentile, and introduced network solutions as the project ran in a hybrid cloud environment.

I worked across teams in three countries to define common pain points and introduce new solutions. This included creating the POC/RFC for new solutions, discussing solutions with teams, collaborating with teams and project managers to set the execution plan, mentoring data academy members, and building standard tools that accelerate development time.

I designed and developed innovative stream and batch data processing projects to improve the ranking of our clients with millions of keywords for many search engines in many countries with many languages.

I created a changeable data pipeline for billions of messages and records and designed the ETL framework to work with many sources and sinks. I presented new technologies and discussed them with the team. I tuned the jobs to fit our cluster, reviewed the code, and took ownership of the project.

Jarvis is a reactive DAG framework to build logical flow over Akka actors and futures. I reviewed the requirements with all teams involved to simplify and remove repeated functions. I designed and built a DAG solution to streamline and service our business logic units in reactive mode. I presented the solution and helped teams use it.

I designed and developed an innovative project to rank hotels based on user preferences. I assisted data scientists in collecting the data to apply the offline training and designed and built a solution replicating the ALS model to five data centers. I created a distributed and local cache solution to hold the ML model in memory. This achieved a four-millisecond response time in the worst-case scenarios. I developed a load balancer between servers in the same data center, applied our DAG framework (Jarvis) to build our ranking service, and tuned the microservice to accept millions of requests with a success rate of 99.99. I then audited the customer interaction with the microservice to use it for model evaluation and configured the deployment scripts for production and stage servers.

This is a back-end project to manage the bidding channel jobs for companies such as Google and TripAdvisor. I reviewed the design with the software architect, and built the dynamic implementation for channels, accounts, sync data, and the Oozie and HDFS clients. I built the migration scripts and configured the deployment environment for production.