José Serra

I worked on capturing body and facial movements for Aardman BBC’s Is Anna OK? [https://www.aardman.com/work/is-anna-ok/]. This was achieved using Vicon Shogun and Vicon’s Cara HMC. Within this project, I was also involved in the implementation of the tools used to drive the facial animation of the characters.
For more information: https://www.camera.ac.uk/cameras-facial-motion-capture-helps-bring-vr-documentary-to-life/

Captured and previewed the motion for Aardman Bandai Namco’s 11-11 Memories Retold. We used Vicon Shogun to capture the motion of half of the actors and Unity to preview the retargeted motion.

Captured the motion Karah Son for Rewind/Welsh National Opera’s Madame Butterfly project.
We used Vicon Blade to capture facial and body motion.
For more information, check CAMERA’s blog pos here [https://www.camera.ac.uk/welsh-national-opera-project-rewind/].

Internal collaboration (Co-Lab) between different departments of the University of Bath (CAMERA/CS department and The Edge) to create Semblance. We worked on the full project from inception to completion. The project itself takes the form of a VR dancing experience, where the dancer (captured used Vicon) drives an avatar in UE and leaves behind a ghostly trail.
I worked on assembling the Vicon to the UE pipeline and implemented the UE experience.
For more, information check CAMERA’s blog post and video here [https://www.camera.ac.uk/virtual-worlds-dance-event-22-april-2017/].

Helped develop a visual speech animation system for Golem and LIFEisGAME projects in collaboration with Microsoft’s MLDC in Lisbon. In this project, we implemented a modular speech animation framework that started with speech recognition to identify the phonemes and their timings, and then using a phone-to-viseme map; the animation was assembled and blended. We explored different phone-to-viseme maps and used MFCCs to vary the poses according to the intensity of the speech signal. The framework was also general enough so that different modules (text-to-speech, co-articulation) could be switched without affecting all the other steps required to generate the animation. This led to the iberSPECCH publication and my Master of science thesis.