Data Engineer and Developer in Aguascalientes, Mexico

I developed a back-end solution using the Microsoft tech stack for a scanning-and-profiling tool that benchmarks Microsoft security competitors. This work included a weekly process of automatic scans which were processed in AWS and transferred to Azure Data Lake using Azure Data Factory. After data was cleaned and transformed, it was then ingested into an SQL Server database with a star schema data warehouse model.

We then used stored procedures with complex logic to do a week-over-week comparison, add new domains/accounts, and flag alerts that trigger an Azure Logic Apps API to submit email notifications. Finally, the data was then sent to an Azure database for a custom web application to consume and then produce dashboards to let sellers and account executives input additional information for profiling.

I was part of the design and development of an Azure modern data warehouse solution for an international chocolate manufacturer company.

The platform allows a scalable, maintainable method to start the migration of independent subsidiaries' data needs from different on-premise solutions and external tools.

I also contributed to the data architecture proposal based on the Azure Data Lake Gen2 storage with a raw, clean, and reporting data layers model. I used Azure Data Factory for the ingestion, transformation, and orchestration of data between layers using the dataflow component. We then built a final star schema relational database model for live reporting on Azure SQL databases to allow connections from Power BI dashboards and ad-hoc views for data analyst requirements.

In addition, I developed the pipelines for third-party external market share sources coming from CSV exported files. Besides the ETL logic and process, created the validation and auditing procedure with custom logs that are stored in an Azure SQL database and automatic mail notifications triggered by ADF on Azure Logic Apps.

The tool is a custom .NET MVC C# web application designed to be a 1-stop-shop portal for managing the US monthly business review process around the financial scorecard KPI's of an international technology company. I was the project manager and technical lead of a 4-person team that designed, developed, and maintained the solution.

My main activities were to design the back-end architecture to handle the monthly scorecard data coming from multiple data sources and handled by an ETL process using a hybrid approach on Integration Services and Azure Data Factory pipelines.

I created the SQL server algorithm to handle the forecast inputs submitted through the web application. The program was capable of gathering forecasts at three different levels of granularity and be able to roll up the numbers at all levels and update different measures depending on the specific logic of each KPI.

I was involved in the design and creation of Power BI reports which were later embedded into the web portal to give the drill-down ability for specific metrics and sales-unit details.