Ovunc Tuzel

A machine learning model trained using PyTorch that can precisely detect when a user is about to jump off a ledge. The model was used for a VR skydiving experience. Detecting the jump at the right time was crucial for an immersive and comfortable user experience, and the existing rule-based triggers were sometimes causing false triggers.

The machine learning model was trained using data gathered from customers. 6DOF tracking data was collected from hand, head, foot, and back trackers. This data was used for jump prediction.

The model was successfully integrated into the Unreal/C++ project, and inference could be done in only a few milliseconds. The jump prediction was very accurate and eliminated the false positives seen in the former rule-based approach.

A custom optical character recognition (OCR) model was designed and deployed to read digits on labels in retail environments. The model outperformed off-the-shelf OCR tools and was over 95% accurate in detecting long-digit sequences.

The model could accurately read blurry, out-of-focus, damaged, or slightly occluded labels.

The model was purely trained on synthetic data. A tool was built that could essentially generate infinite variations of the labels used as training data. This eliminated a costly labeling step, saving significant resources and time.

Training, evaluation, and deployment were streamlined using GitLab CI/CD jobs and cloud-based model storage.

Built a unique framework in C++ and Unreal blueprints that allowed remapping the topology of a virtual space in an intuitive way. Using this framework, virtual reality content creators could create experiences that match a physical space and trick the user into believing they are in a completely different environment.

For example, a common use case was to trick users into walking in circles while they thought they were walking in a long straight corridor.

Another use case was convincing the user that they were in a large room while, in reality, they were in a smaller space.

A blueprint API was built for the project that hid the complex math from the content creators, who could focus on the experience flow and not the mathematical transformations occurring in the background.