Skip to main content
Story

KE Staff Awards 2022 Innovation of the Year: InteractML for Unreal

a dual screen with a person wearing VR glasses and the display showing the alphabet and flag positions
  • Written byYana Kasa
  • Published date 30 March 2023
a dual screen with a person wearing VR glasses and the display showing the alphabet and flag positions
Gibson-Martelli using InteractML VR for artwork, Semaphore, photography by Gibson-Martelli, 2022
Knowledge Exchange is how we work with external partners to create positive change in our society, economy and environment. The KE Staff Awards 2022 celebrated and made visible the diverse range of Knowledge Exchange practices at UAL.

The winners of the Innovation of the Year award were both, InteractML and Ananas Anam Circular and Sustainable by Design.

InteractML won the Innovation of the Year award, for its potential to break new ground. InteractML is an open-source plugin for developers and creators, it allows for interactive machine learning within the Unreal Engine software. Unreal Engine is currently the most powerful real-time 3D creation tool used across creative industries.

Born from disabled technologist Phoenix Perry’s research on creating more accessible interfaces with game engines and Rebecca Fiebrink’s passion for making Machine Learning accessible to all, InteractML expands what inputs developers can use in games to create dynamic, gesture-based user experiences. The project is headed by Phoenix Perry with Rebecca Fiebrink. The Unreal version also includes the development efforts of Sam Swain and Bruno Martelli.

InteractML expands the functionality of the Unreal software to allow game developers and creators to build machine learning systems using the visual programming tool, Blueprints. Visual coding uses nodes hooked together via cables. It does not require that game developers or environmental designers have an in-depth understanding of C++. The three machine learning algorithms are Classification, Regression and Dynamic Timewarping. Developers can build a training set by recording examples, which can be movement or other inputs like a position of a character in a 3D space. The output of the running model is then used to drive the desired in-engine systems and effects. InteractML empowers game developers who want to expand the range of input systems to be inclusive of their player’s bodies verses forcing them to conform to standardised controllers and their inbuilt biases.

The potential applications include creating custom control schemes within the software, designing specific gesture recognition, fuzzy control systems and developing accessibility tools. Having been designed by a disabled developer, this plugin produces new accessibility options within the Unreal Engine Software.

InteractML has expanded interactive virtual creation to new immersive horizons. Since launching in Spring 2022, the plugin has been adopted by the creative community and has been downloaded by over 55,000 creators from the Epic Store. Students are also exploring the plugin in classrooms, testing the possibilities of gesture-based control in games.

Find out more about the winners of the KE Staff Awards 2022 (PDF 3.8MB).