Joao Gabriel Oliveira
Verified Expert in Engineering
Data Engineer and Developer
Lisbon, Portugal
Toptal member since December 23, 2022
Joao is an experienced, challenge-driven, and detail-oriented software architect specializing in data engineering and machine learning. He is highly skilled in designing and implementing end-to-end data solutions that combine efficient processing, complex querying capabilities, and insights extraction. With solid foundations in computer science and math and excellent communication and teamwork skills, Joao can deliver value to the most challenging data-driven projects.
Portfolio
Experience
- Data Modeling - 10 years
- Java - 10 years
- Python - 10 years
- Data Engineering - 10 years
- Elasticsearch - 10 years
- Apache Spark - 5 years
- Data Lakes - 5 years
- Amazon SageMaker - 4 years
Availability
Preferred Environment
Python, Amazon Web Services (AWS), Visual Studio Code (VS Code), Slack, Jira, GitHub
The most amazing...
...contribution I've made was modeling and coordinating the development of a social media data platform with over 10 million profiles and 1.5 billion posts.
Work Experience
Senior Data Architect
Strata Company
- Redesigned an internal legacy BI system, drastically improving reliability and maintainability while still not disrupting established usage for key stakeholders.
- Designed and implemented a full-fledged Lakehouse architecture system from scratch, serving analytical and user-triggered batch operations requirements.
- Worked closely with product and higher leadership to guide the overall data strategy for the company.
Data Manager
Izea
- Increased the average team velocity up to 50% through process improvements, better planning, and knowledge-sharing initiatives.
- Supported multiple teams by delivering the technological direction of the company's social data platform.
- Led and managed a team of five members, including back-end engineers, data engineers, and data scientists.
Senior Data and Machine Learning Engineer
Izea
- Modeled and coordinated the deployment of a social media data platform with over 10 million profiles and 1.5 billion posts, integrating batch and stream processing technologies with an intricate index schema design.
- Spearheaded the execution of machine learning models as a core platform element, employing a combination of textual feature extraction and various regression and classification algorithms to forecast audience demographics.
- Served as the main point of contact between the product and data teams in specifying data product requirements.
- Supported the launch of three new products by leading the implementation process.
Senior Software Architect
TapInfluence
- Coordinated the system shifting to microservices architecture with a keen focus on analytics and search components.
- Steered significant efforts for implementing relevance score improvements on the product search functionality.
- Delivered exceptional assistance to the VP of engineering on critical technological and architectural decisions.
Senior Software Architect | Partner
Amtera Semantic Technologies
- Implemented diverse product concepts and components using state-of-the-art research on semantic search and natural language processing.
- Established robust and effective in-house solutions for various important clients in the telecom, oil and gas, and IT security industries.
- Collaborated with other partners to implement the overall company strategy.
Experience
Core Social Media Data Platform for Izea
I was one of the key architects and also played a central role in implementing the platform.
NLP-based Machine Learning Models for Izea
I was the principal architect and developer of the project.
Education
Bachelor's Degree in Computer Science
Federal University of Rio de Janeiro | UFRJ - Rio de Janeiro, Brazil
Certifications
Machine Learning with Python: From Linear Models to Deep Learning
MITx Online
MongoDB for Developers
MongoDB University
Skills
Libraries/APIs
Spark Streaming, PySpark, Node.js, PyTorch
Tools
GitHub, Amazon Elastic MapReduce (EMR), Amazon SageMaker, Slack, Jira, Amazon Elastic Container Service (ECS), Terraform, Amazon CloudFront CDN
Languages
Python, SQL, Java, Scala, JavaScript
Frameworks
Apache Spark, Spark, Spring
Paradigms
ETL, ETL Implementation & Design, Database Design, DevOps, Business Intelligence (BI)
Platforms
Amazon Web Services (AWS), Docker, Visual Studio Code (VS Code)
Storage
Elasticsearch, Databases, NoSQL, Data Pipelines, Amazon S3 (AWS S3), Database Architecture, Data Lakes, MongoDB, PostgreSQL
Other
Data Modeling, Algorithms, Programming, Time Complexity Analysis, Space Complexity Analysis, Linked Data, Semantics, Natural Language Processing (NLP), Machine Learning, Data Engineering, Data Architecture, Big Data Architecture, Data Warehousing, Data, APIs, Data Transformation, Data Science, Architecture, Generative Pre-trained Transformers (GPT), Artificial Intelligence (AI), Amazon Kinesis, AWS Cloud Architecture, Compilers, Number Theory, Deep Learning, CI/CD Pipelines, Data Visualization, Reporting
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring