Rafael Castro, Developer in London, United Kingdom
Rafael is available for hire
Hire Rafael

Rafael Castro

Verified Expert  in Engineering

Bio

Rafael is a seasoned engineer with over a decade of expertise. Notable for his role at Allianz, he led the development of an organization-wide streaming data platform and contributed to major projects. At Schibsted, Rafael played a key role in building end-to-end recommendation systems. His impactful contributions extend to Skyscanner, BNP Paribas, and Sky. With multiple certifications, he is dedicated to staying updated on tech trends and committed to delivering outstanding results.

Portfolio

Allianz
KSQL, Apache Kafka, Kafka Streams, Databricks, Azure, Python 3, Scala...
H&M Company
Python, Azure, Machine Learning, Machine Learning Operations (MLOps), SQL...
Schibsted
Apache Spark, Python, Solution Architecture, Team Mentoring, Big Data...

Experience

  • SQL - 10 years
  • Python 3 - 8 years
  • Data Engineering - 7 years
  • Spark - 6 years
  • Spark ML - 5 years
  • Scala - 5 years
  • Machine Learning - 4 years
  • Databricks - 3 years

Availability

Part-time

Preferred Environment

Python 3, Scala, Java, Spark, SQL, Apache Kafka, Databricks, Azure, Amazon Web Services (AWS)

The most amazing...

...thing I've done is the work on the search and recommender systems at Schibsted, which are used by millions of marketplace customers globally.

Work Experience

Senior Data Solutions Architect and Engineer

2020 - 2023
Allianz
  • Contributed to the shaping of a streaming data platform on Azure, defined best practice patterns, assisted in hiring new engineers, established synergies, and collaborated with business stakeholders to define feature and service specifications.
  • Led two major projects, overseeing the onboarding and development of specific features for clients like AZ Germany and AZ Italy. Designed, architected, and delivered key services like data quality (DQ) checking and 3rd-party integrations.
  • Implemented an enhanced data architecture featuring a data lake and multiple data pipelines, which allowed senior executives to make well-informed financial decisions and minimize operational costs.
Technologies: KSQL, Apache Kafka, Kafka Streams, Databricks, Azure, Python 3, Scala, Argo Workflows, Kubernetes, Azure Data Factory (ADF), Data Warehousing, Data Integration, Snowflake

Senior Machine Learning Engineer

2019 - 2020
H&M Company
  • Contributed to the ongoing iteration and played a key role in shaping the future direction of the store assortment forecasting project within the AI division at H&M.
  • Automated established workflows and engineered data/machine learning (ML) systems utilizing predominantly Python and Kubernetes within the Microsoft Azure environment.
  • Developed a system that predicts the optimal quantity of units for each article, assisting H&M's internal buyers in future season orders. This initiative improved precision over human estimates, leading to noteworthy cost savings for the company.
Technologies: Python, Azure, Machine Learning, Machine Learning Operations (MLOps), SQL, Kubernetes, Microservices, Docker, Data Warehousing, Data Science, Data Integration

Senior Data and Machine Learning Engineer

2018 - 2019
Schibsted
  • Joined the personalization team, focusing on the architecture and productionization of end-to-end recommendation systems. Contributed to data ingestion, scaling Spark-based recommender algorithms, and developing serving APIs using functional Scala.
  • Designed and implemented streaming Kafka pipelines to support recommenders. Migrated our technology stack to Kubernetes while exploring the integration of deep learning techniques for generating real-time recommendations in the online environment.
  • Implemented real-time information integration for marketplace clients like Leboncoin, enhancing the relevance of recommendations and significantly boosting the clickthrough rate.
  • Architected and implemented an MVP for a trending ads recommendation product, employing a serverless architecture on AWS, presenting a cost-effective solution to serve as a client's initial foray into recommender systems.
Technologies: SQL, Python, Docker, Data Analysis, Microservices, Solution Architecture, Big Data, Team Mentoring, Apache Spark, Scala, Spark, Spark ML, Apache Kafka, Amazon Web Services (AWS), Kubernetes, Data Engineering, Machine Learning, Machine Learning Operations (MLOps), Python 3, Software Architecture, APIs, Data Science

Senior Data and Machine Learning Engineer

2017 - 2018
Schibsted
  • Collaborated within the user modeling team on building and maintaining large-scale, GDPR-compliant ML pipelines for predicting customer attributes for millions of users across various global markets.
  • Enhanced the efficiency of a global notifications component by incorporating personalized ML logic to deliver notifications at optimal times, thereby improving the system's overall effectiveness.
  • Conducted experiments with deep learning techniques, including convolutional neural networks (CNNs) and recurrent neural networks (RNNs), on projects such as a real estate valuation model and demographic attributes prediction.
Technologies: Spark, Scala, Python, Amazon Web Services (AWS), Spark ML, Luigi, Machine Learning, Data Engineering, Machine Learning Operations (MLOps), SQL, Apache Kafka, Docker, Microservices, Data Science, Data Integration

Senior Software Engineer

2015 - 2016
Skyscanner
  • Participated in the API-first design of a microservices-based architecture. Implemented an ad-hoc DynamoDB auto-scaling strategy, enhancing the project's cost-effectiveness. Designed and provisioned a tailored ELK stack on AWS for project logging.
  • Revolutionized hotel catalog processing by suggesting and implementing an alternative architecture leveraging PySpark to reduce computing time for efficiently pre-processing large CSV files.
  • Developed and implemented an image tagging service prototype to identify non-relevant hotel photos using a CNN model executed with Caffe, achieving a high recall value for accurate classification.
Technologies: Amazon Web Services (AWS), Python 3, REST APIs, Big Data, Deep Learning, Spark, Microservices, Convolutional Neural Networks (CNNs), Elasticsearch, Data Warehousing, Data Integration

Senior Software Engineer

2013 - 2015
BNP Paribas United Kingdom
  • Developed a new monitoring big data and ML solution, a multi-tier, multi-region system that monitors the bank's activities, proactively predicting and detecting system failures.
  • Contributed to the presentation layer using technologies such as Django, D3.js, and others.
  • Played a pivotal role in back-end development utilizing technologies such as Java, Apache Storm, Redis, Splunk, and more.
Technologies: Java, Python 3, Apache Storm, Big Data, Streaming, Real-time Data, Django, Redis, Microservices

Senior Software Engineer

2011 - 2013
Sky UK
  • Collaborated with the CTT and systems integration teams to formulate a strategy for presenting a unified view of test status and quality across diverse streams of set-top box (STB) software development and testing.
  • Worked in a small team and contributed to designing and implementing web tools widely adopted by various peers, increasing interaction among all stakeholders involved in product delivery and testing.
  • Spearheaded a data mining initiative to automate test results analysis previously conducted manually by a separate team. Designed and initiated the implementation of a data mining solution for streamlined and automated processes.
Technologies: Java, Python 3, Groovy, Web Development, Machine Learning

Software Developer

2007 - 2010
Other Media
  • Served as the lead developer for the Royal London Society for the blind website project, overseeing the entire software lifecycle.
  • Contributed as a team member to the development of the new version of OtherObjects, an in-house content management system (CMS) built on the Spring Framework. Designed and implemented multiple modules within this Java Spring-based CMS.
  • Developed, supported, and integrated client websites, including circleanglia.com as a lead developer, ecb.co.uk, royalacademy.org.uk for which I built separate modules and provided support, and numerous others.
Technologies: Java, Hibernate, PostgreSQL

Experience

Holiday Accommodation Search App

Developed a RAG application for stay searches using natural language. I delegated scraper code to team members, enhanced data with large language models (LLMs), and indexed information in a vectorized database. Also, I am currently finalizing the API and preparing for integration into a web app.

Education

2013 - 2015

Master's Degree in Computer Science

UNED - Madrid, Spain

Certifications

SEPTEMBER 2023 - SEPTEMBER 2025

Databricks Certified Associate Developer for Apache Spark 3.0

Databricks

NOVEMBER 2022 - NOVEMBER 2024

Microsoft Certified: Azure Data Engineer Associate

Microsoft

DECEMBER 2019 - DECEMBER 2021

AWS Certified Solutions Architect

Amazon Web Services

SEPTEMBER 2019 - SEPTEMBER 2021

AWS Certified Machine Learning Engineer

Amazon Web Services

SEPTEMBER 2013 - SEPTEMBER 2015

Certified Developer for Apache Hadoop CDH4

Cloudera

OCTOBER 2008 - OCTOBER 2010

Sun Certified Java Programmer

Oracle

Skills

Libraries/APIs

Spark ML, Luigi, REST APIs

Tools

Kafka Streams, Apache Storm

Languages

Python 3, SQL, Python, Scala, Java, Groovy, Snowflake

Frameworks

Spark, Hibernate, Hadoop, Django, Apache Spark

Paradigms

Microservices

Platforms

Apache Kafka, Azure, Amazon Web Services (AWS), Kubernetes, Databricks, Docker

Storage

Data Integration, PostgreSQL, Elasticsearch, Redis

Other

Data Engineering, Artificial Intelligence (AI), Azure Data Factory (ADF), APIs, Machine Learning, Data Science, KSQL, Argo Workflows, Large Language Models (LLMs), Natural Language Processing (NLP), Machine Learning Operations (MLOps), Software Architecture, Big Data, Deep Learning, Convolutional Neural Networks (CNNs), Streaming, Real-time Data, Web Development, Data Analysis, Team Mentoring, Solution Architecture, Data Warehousing

Collaboration That Works

How to Work with Toptal

Toptal matches you directly with global industry experts from our network in hours—not weeks or months.

1

Share your needs

Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2

Choose your talent

Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3

Start your risk-free talent trial

Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.

Top talent is in high demand.

Start hiring