Pedro Alves Nogueira

Helped build the Computer Vision algorithm for an iOS app designed to automate body size measurements.
The solution uses a combination of Computer Vision and anthropomorphic science to deduct the positions and measurements of several body points from still images taken by the users and matches these to clothing vendor's item sizes for a clear-cut shopping experiences.

Defined and supervised the implementation of a novel MCTS-based AI for The Octagon Theory, a game with a complexity orders of magnitude above chess.

MCTS AI was also later dotted with Bayesian opponent modelling capabilities that allowed it to not only surpass any known human players but also all other known/previously implemented AIs. Work was featured at AIGameDev.com (https://aigamedev.com/broadcasts/session-mcts-tot/).

In this project, I single-handedly developed a fully automated processing pipeline for processing confocal microscopy imaging using computer vision and machine learning techniques. The process was adopted into existing software to save thousands of manual annotation hours on drug research trials aimed at identifying a cure for Leishmania. The process is completely parameterisable to account for different magnification levels, signal-to-noise ratios and background assumptions, achieved partly due to a modular architecture. The project was implemented fully in Java, with all core computer vision algorithms manually implemented, optimized, and tuned, and models were trained in Weka (and later imported into the Java code).

Implemented new features and performed bug fixes on CleverTech’s internal hiring system. Also defined and provided technical supervision on parallel and subsequent development. The system was responsible for processing all new applications to the company’s job openings, as well as perform applicant tracking, rating and interview process reporting. Employed technologies include: Angular.js, Node.js, MongoDB and Twitter Bootstrap.

Developed a suite of parallel, multilevel models that were able to transform an individual’s physiological data (skin conductivity, heart rate and facial muscle activation) into emotional states (arousal and valence). Employed methods include, but were not limited to: linear/polynomial regression, decision trees, artificial neural networks, support vector machines, fuzzy rules, random forests, and (weighted) ensemble models. Development was done in R and Python for fuzzy rules and data pre-processing.

A video/physiological data synchronization and processing tool for aiding in direct observation UX studies. The tool integrated a configurable emotion detection system previously developed to process the physiological data into emotional states for visualization. Users could annotate relevant events from a (user-defined) list of existing events and add optional detailed feedback to each event. Once annotated, the session could be saved in a specific format (csv or eet) and later loaded for further inspection/validation by other team members. The main selling point of the tool was the ability to employ a peak detection algorithm to automatically determine emotional reactions to the annotated events using the physiological/emotional data. This reduced intra- and inter-personal bias while also saving man-hours. The project was implemented in C#, also using ZedGraph.

This work involved the development of player models' affective reactions in regards to specific game events based on observed past reactions. Related activities involved analyzing the collected data, testing several feature extraction and selection algorithms, and machine learning classifier training. We finally settled on a more structured/grounded approach that took into account the data’s nature using a matrix of regression models for each event/emotional dimension. Having dealt with idiosyncratic reactions, scaling issues and data sparsity through this approach, the created models were then used to create distance metrics between player pairs, enabling a hierarchical clustering approach followed by a fuzzy cluster approximation formula. Development was done in R, but the models were later fully mapped so they could be loaded into memory for direct access (see GOAD project). For details see: http://www.aaai.org/ocs/index.php/AIIDE/AIIDE14/paper/viewPDFInterstitial/8947/8939.

Developed a simple, symbolic simulator to automatically play out all possible variants of a digital game with a specific range of configurations. This simulator was used to automatically assess the effectiveness of suggested gameplay adaptations dictated by affective player models. Given a model of how a specific player might react to game events, this model identifies the set of best gameplay parameters to drive the player towards a specified emotional state or pattern. The project was fully implemented in Java and ran on a headless mode (configuration files were generated by a separate GUI). This allowed us to speed up the system, which generated over 36GB or pure text log data on a 1 hour simulation run over several hundreds of millions of simulation steps.

This is a web application that enables users to generate an "A Capella" for any given music track. To do this, the user needs only to chose an audio or movie file (the audio track is automatically extracted) and load the lyrics he wants to embed in the music track. The application will then synthesise the given lyrics, apply a parameterisable pitch deformation algorithm (i.e. auto-tune the lyrics) and generate a new audio/video file that combines the music track, lyrics and video - if available.

In recent years, video game companies have recognized the role of player engagement as a major factor in user experience and enjoyment. This encouraged a greater investment in new types of game controllers such as the WiiMote, Rock Band instruments and the Kinect. However, the native software of these controllers was not originally designed to be used in other game applications. This work addresses this issue by building a middleware framework, which maps body poses or voice commands to actions in any game. This not only affords a more natural and customised user-experience but it also defines an interoperable virtual controller. In this version of the framework, body poses and voice commands are respectively recognized through the Kinect's built-in cameras and microphones. The acquired data is then translated into the native interaction scheme in real time using a lightweight method based on spatial restrictions. The system is also prepared to use Nintendo's Wiimote as an auxiliary and unobtrusive gamepad for physically or verbally impractical commands. System validation was performed by analyzing the performance of certain tasks and examining user reports. Both confirmed this approach as a practical and alluring alternative to the game's native interaction scheme. In sum, this framework provides a game-controlling tool that is totally customizable and very flexible, thus expanding the market of game consumers.

Invited to participate in a live, premium broadcast on AIGameDev pertaining our on MCTS AI algorithms for The Octagon Theory.

Been a member for the AAAI society for 2 consecutive years now due to continued publications in their conferences.

Successfully supervised over 10 M.Sc. theses in the digital games and game theory domains with an average of A rankings.

Representative sample of topics include:
- Direct biofeedback shooter games
- Affective movies
- Biofeedback horror games
- Emotional NPCs and human player mimicking
- Affective PCG
- MCTS for complex board games (orders of magnitude above chess)
- DBF framework
- Stealth games mcts
- PCG for platform games