Data Engineering Manager | Tech Lead2019 - PRESENTHeetch
Technologies: Scala, Java, Apache Kafka, Kafka Streams, Dataiku, Apache Airflow, Kubernetes, Marathon, Redshift, Redshift Spectrum
- Designed, wrote, and maintained real-time streaming pipelines, enriching and aggregating raw data coming from multiple different sources.
- Implemented and designed a service to be able to predict and prevent users' bad behaviors in real-time.
- Got rid of third-party data synchronization and transformation pipelines by writing our own custom pipelines.
- Managed a team of four data engineers and did carrier reviews and handling rituals.
- Communicated with stakeholders and data analysts and data scientists to monitor the success of new initiatives.
- Advocated on schema management across the whole company.
- Provided tools to generate schemas, used them to validate the mobile messages; handled enrichment, storage, and exposition.
Senior Software Engineer2017 - 2019Audela
Technologies: Apache Kafka, Spark, Apache Avro, PostgreSQL
- Built real-time accurate network state view of physical, logical, and service topologies for telco operators.
- Oversaw the full CI/CD infrastructure automation, deploying to Kubernetes.
- Leveraged graph databases for storage and exposition.
Data Engineer2016 - 2017Societe Generale
Technologies: Spark, Spark SQL, Apache Ignite, Cloudera, HDFS
- Designed and built a data lake that would store daily stock exchange orders.
- Built pipelines allowing real-time data enrichment with Apache Ignite.
- Met with stakeholders and translated their business needs into technical specifications.
BI Engineer2012 - 2016Carrefour
Technologies: Scala, Java, SQL, Shell, Oracle, SSIS Custom Components, SQL Server 2000
- Managed high-volume databases (SQLServer and Oracle).
- Worked on the development and evolution of BI infrastructures (SSIS).
- Did data analysis and development of reporting exposed on a web portal (SSRS).