Brenda Alexsandra Januário
Verified Expert in Engineering
Data Engineer and Developer
Brenda is an experienced data engineer focused on quality pipelines so that organizations make decisions with the best and most accurate information. Her love for innovative technologies and know-how in using them with data is what drives her professional career while providing customer-centric solutions. Brenda enjoys being a part of the field and having the privilege of working with professionals worldwide.
Databricks, PySpark, SQL, Amazon Web Services (AWS), Git, Python 3
The most amazing...
...thing I've done is work alone on the data engineering team after a series of layoffs, but still maintained the quality and quantity of deliverables.
- Coordinated migrating PostgreSQL Amazon RDS data and AWS Lambda or AWS Batch data pipelines to Databricks. Implemented the medallion architecture in the data lakehouse. Documented the architecture guidelines used and the motivation for choosing it.
- Developed batch and streaming data pipelines for big data. Extracted data from public and paid APIs and web scraping; transformed and loaded data into PySpark. Documented the methodologies used to interpolate missing values and outliers.
- Implemented testing and data quality using dbt and Great Expectations. Documented the tests. Applied methodologies to optimize the table.
- Coordinated the separation of the data lakehouse into multiple environments. Created the IaC using Terraform CDKTF in Python, created the Docker image with the necessary resources, and registered it in the repository. Implemented CI/CD using GitLab.
- Led the implementation of data observability and data quality, planning, architecture design, and development. Automated data metrics creation through data profiling with PySpark and Bigeye. Created the CI/CD in GitLab for deployment.
- Managed and maintained on-premise data pipelines for Itaú's real estate credit SQL database. Modeled, oversaw, and documented the tables.
- Created data pipelines created using python or SSIS on-premises. Created analytical dashboards to identify opportunities for Itaú's Real Estate Credit area using Tableau.
- Developed data pipelines for Itaú Real Estate Credit API requests using PySpark in AWS Glue and created views in AWS Athena to build dashboards in AWS QuickSight.
- Created the REST API for tracking real-estate financing proposals for the Itaú website in Kotlin with gRPC framework integrated with Amazon DynamoDB NoSQL database.
A Pipeline of Agriculture Data
The data was tested at every stage using the data quality open-source Great Expectations, and Sentry was enabled to notify in case of failure.
A Data Lakehouse and Visualizations for B2B eCommercehttps://github.com/brendajanuario/pipeline-bigdata-pyspark
I developed ETL pipelines and created data tests and the infrastructure as code (IaC) using Terraform to build the data lakehouse. I generated more than 700 million rows, and the final structured data were used to create visualizations in tabular and dashboards format, to work with BigData.
PySpark, REST APIs
ETL, Scrum, DevOps
Databricks, Amazon Web Services (AWS), Docker, Kubernetes, Apache Kafka
Data Pipelines, Microsoft SQL Server, NoSQL, SQL Server Integration Services (SSIS)
Azure Databricks, Web Scraping, Big Data, Data Quality, Data Modeling, Stream Processing, Data Engineering, Data Warehousing, Dashboard Development, Data Build Tool (dbt), Data Quality Analysis, Test Data, Streaming, Infrastructure as Code (IaC), Cloud, Events
GitLab, GitLab CI/CD, Tableau, AWS Glue, Amazon QuickSight, Sybase PowerDesigner, Terraform, Sentry, Git, Amazon Athena
gRPC, Django, Spark
Bachelor's Degree in Information Systems
Universidade Estadual De Campinas (Unicamp) - Limeira, Brazil
Cloud Data Engineer - Bootcamp
XP Education (formerly IGTI)
AWS Certified Cloud Practitioner
Amazon Web Services
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.Start hiring