Verified Expert in Engineering
Machine Learning Developer
Carlos is an exceptional data generalist who brings vast experience in the design, implementation, and validation of data-intensive systems to all of his projects, along with deep expertise in machine learning and real-time stream processing. He has worked in the eCommerce and media industries, working for large corporations and startups. Carlos is a versatile engineer and looks forward to his next challenge.
Apache Spark, Amazon Web Services (AWS), Python, Scala, Machine Learning, TensorFlow, PyMC, Apache Kafka
The most amazing...
...thing I've built is a low latency custom recommendation system for an eCommerce startup doing flash sales.
Freelance Data Scientist and Engineer
- Designed and built various machine learning models for risk assessment, customer churn, and rate optimization for a retail bank using Python, Pandas, NumPy, SciPy, PyMC, TensorFlow, and scikit-learn. Deployed with custom FastAPI back end.
- Designed and built a custom system for products offering recommendations from transactions and demographic data (retail bank) using Python, Pandas, NumPy, and C++. Deployed with custom FastAPI back end.
- Designed and built a continuous analytics backbone and a data warehouse in a hybrid onsite/AWS environment with Kafka, Scala, Python, and Redshift.
- Designed and built a real-time data fusion pipeline to create a complete picture of customer transactions from different systems.
- Assembled a low-latency custom recommender system for eCommerce flash sales using Python and C++ for an eCommerce startup. Deployed with custom C++/Boost.Asio back end.
- Developed a custom marketing message timing optimizer for eCommerce using Python and R for an eCommerce startup.
- Built a bespoke transaction risk analysis system for eCommerce using Python and R.
- Refactored, optimized, fixed, streamlined, documented, and further developed a complex and largely undocumented Airflow delivery workflow.
- Ported to Scala, optimized, tested, and documented Spark UDF/UDAF/UDTs written in Java and included functions dealing with text and URL matching, information extraction from text and URLs, and supporting data structures. Wrote Python bindings.
- Built a prototype knowledge management system for a manufacturing/engineering company. Performed technical due diligence for the development with a partner of CAD software to be distributed as part of this system.
Director of Data Science
- Implemented real-time analytics on operations, modeled interventions on customer experience to address returns and cancellations, built a policy optimizer through retrospective simulation with historical data, and enabled it as a microservice.
- Expanded the policy optimizer to improve order profitability by optimizing basket constraints and incentives.
- Implemented various improvements to the product recommender, including the use of fine-grained recorded impressions as a negative signal and more flexibility in handling catalog metadata.
- Built and deployed a foundational analytical backbone for the company in AWS with Kinesis, Redshift, and Spark.
- Integrated continuous data ingestion from key systems into the analytical backbone, whenever practical, through low latency interfaces such as database replication.
- Migrated some interaction tracking systems to the backbone and the recommender.
- Conducted retrospective sourcing performance and pricing analysis by replaying row mutations continuously captured from database replication logs and stored in Redshift (Python/C++).
Software and Data Architect
- Built an image upload/pre-processing pipeline for a media startup using Node.js and MongoDB on AWS. Included single sign-on with a Ruby on Rails app in the back end.
- Created custom, interactive data displays for a bespoke structured messaging application using D3.js. Implemented real-time updates.
- Implemented a structured messaging application. Contributed to the Python/Django back-end and the CoffeeScript front end.
- Built a custom C# distributed data analysis pipeline to perform MATLAB jobs on AWS.
- Designed and implemented a custom interactive data analysis and visualization for economic data along with a Python back-end and D3.js visualization.
- Assembled a custom nurse schedule and route optimization system for a healthcare software startup. Worked on pre-processing and mixed integer model formulation for Gurobi with Python/Pandas/NumPy/PuLP, D3.js visualization of solutions, and Flask API.
- Modernized the system design and implementation of a Java/Spring back end for real-time transport logistics. Improved scalability and performance.
- Designed and implemented a reference application for a high-security network architecture for a banking customer with Scala/Play, Slick, and two-factor authentication.
- Contributed to a large-scale online storage system implementation using Python and PostgreSQL. Contributed to embedded security appliances in C.
- Developed a custom MATLAB system to tune a legacy application from data during black-box optimization (derivative-free).
Chief Software Architect
- Prototyped a voice- and gesture-based user interface for in-car mobile phone usage at various levels of fidelity ranging from Wizard of Oz to software proof-of-concept (Python, Java, Sphinx).
- Defined software architecture for a family of in-car products, with input to hardware platform selection.
- Planned costs, schedule, and execution of multiple new product development scenarios.
- Organized and moderated usability studies for prototype validation and iteration.
- Conducted rigorous feasibility studies and software architecture reviews at Gear.
Senior R&D Manager
- Recruited and ramped up the Maemo application framework team from scratch.
- Defined the application framework architecture and development strategy.
- Led the implementation of three major software generations along with updates.
- Impacted Nokia's entry into open-source development.
- Developed a considerable subcontracting and partnering network for Linux development.
- Contributed to the initial product concept definition.
Senior Software Engineer
- Prototyped a small-footprint relational database for small Linux devices in C++ for the Nokia Research Center.
- Prototyped a personal information manager for handheld devices based on semantic web technology in Python.
- Studied and evaluated architectural options for an application framework aimed at Linux-based handheld devices adopted by the nascent Maemo project.
GIS/Computer Graphics Freelancer
- Built a geographic information system (GIS) to edit the land cadaster for the Portuguese Ministry of Agriculture using C++, Windows, and Oracle technologies.
- Constructed a custom C++ framework for real-time manipulation of topologically integrated geographic vector data.
- Assembled a geographical decision support system for semi-automated execution and optimization of land-consolidation projects for specialized consultancy using C++.
- Developed, licensed, and finally sold a ray-tracing rendering module for use with interior design software written in C++.
- Shaped GIS to edit an olive tree cadaster for the Portuguese Ministry of Agriculture, with integrated olive tree recognition from aerial photography, built with C++ in Windows.
- Designed and implemented flooring tiling algorithms for 3D interior design software.
Keys are binary buffer objects rather than strings. Values are arbitrary objects.
RawHash is built on Google SparseHash and MurmurHash3.
Incremental Random Foresthttps://github.com/pconstr/irf
The forest is maintained incrementally as samples are added or removed - rather than fully rebuilt from scratch every time to save resources.
It is not a streaming implementation, as all the samples are stored and will be re-seen when required to recursively rebuild invalidated subtrees. The effort to update each tree can vary substantially, but the overall effort to update the forest is averaged across the trees and tends not to vary significantly.
Pandas, Node.js, NumPy, Matplotlib, Bottle.py, Eigen, Scikit-learn, SciPy, Theano, D3.js, TensorFlow, Keras, PyMC, PySpark, REST APIs, Spark ML, PyTorch, GitHub API, Slick, Beautiful Soup, Shapely
Data Science, Distributed Computing, Parallel Computing, Distributed Programming, ETL, Microservices, Functional Programming, Usability Testing, Agile Project Management, DevOps
Linux, Amazon Web Services (AWS), Docker, Apache Kafka, Jupyter Notebook, Oracle, Embedded Linux, Kubernetes
Redis, Redshift, MongoDB, PostgreSQL, Databases, Database Architecture, Amazon S3 (AWS S3), MySQL, RocksDB, Elasticsearch
Machine Learning, Amazon Kinesis, Software Engineering, Mathematics, FastAPI, Deep Learning, Bayesian Statistics, PuLP, Optimization, Recommendation Systems, Software Architecture, IT Project Management, Open Source, Random Forests, Data Engineering, Scripting, Data Extraction, Command-line Interface (CLI), Data Architecture, Architecture, Technical Leadership, Back-end, Leadership, SaaS, Algorithms, Natural Language Processing (NLP), Tornado, MLflow, Prometheus, Delta Lake, Statistics, Mixed-integer Linear Programming, Front-end, CVXOPT, Bluetooth, Planning, Usability, Speech Recognition, Due Diligence, Recruitment, Semantic Web, Graphical User Interface (GUI), Computational Geometry, Computer Graphics, DOM, GeoPandas, RESTful Microservices, Boost.Asio
Apache Spark, Hadoop, Flask, Ruby on Rails (RoR), Django, Qt, .NET, Spring
Git, Redshift Spectrum, Spark SQL, Apache Airflow, GitHub, StatsModels, Seaborn, Pytest, MATLAB, Amazon Elastic MapReduce (EMR), AWS IAM, AWS ELB, AWS CLI, Amazon Athena, Grafana, Confluence, Gurobi, GNOME, GTK+, GIS
Master's Degree in Computer Science
Universidade Nova de Lisboa - Lisbon, Portugal