Enhancing conventional human-machine interfaces with XR
Leslah Garland 21 February 2023 
A new research project THEIAXR aims to improve human-machine interaction in mobile machinery by enhancing user technology fit and extended reality (XR) technologies and functionalities.
The three-year project is funded with a total of six million euros from the European Commission’s Horizon Europe program and includes 11 project partners from industry and academia from six EU countries, including project coordinator TTControl, a joint-venture company of TTTech Group and HYDAC International
The project’s goal is to make the invisible visible and extend the perceivable range of the human operator without impacting their performance. THEIAXR will be validated and tested in three use cases in the off-highway domain: snow grooming, logistics, and construction.
Christiana Seethaler, Vice President of Product Development at TTControl is very excited to coordinate this highly relevant project: “Integrating XR technologies into TTControl’s HMI products for mobile machinery is another step towards more intelligent, trustworthy off-highway operations.”
Mobile machines, such as excavators, snow groomers, or cranes, are often large and bulky, and sometimes controlled remotely. It can be a challenge for a human operator to maintain an overview of the machine and its environment. XR technologies can expand the operator’s field of vision and improve confidence in human-machine interaction.
For instance, augmented reality (AR) and virtual reality (VR) technologies may be used to display machine status in the operator’s immediate field of vision. Haptic feedback could be used to send tactile signals to operators, for example via a joystick, and acoustic feedback can be used to provide warning signals to the user. Combined with external sensors, these technologies can also help detect obstacles and other factors of the environment, improving not just the safety of the operator and bystanders, but also increasing efficiency as work can be completed with more foresight.
The project applies a transdisciplinary co-design approach, where all relevant stakeholders of the use case partners are involved in the specification, implementation, testing, and validation activities and decisions of the project, contributing as experts in their respective domains.
Martijn Rooker, Innovation Projects & Funding Manager at project lead TTControl explains: “The XR technology used in mobile machinery must be intuitive enough to enable low-threshold operation in many different scenarios. Therefore, THEIAXR will apply a human-centered, scenario-based co-design methodology. This means that real end-users and real-life data from industrial environments will be integrated to guarantee all solutions are as intuitive and user-friendly as possible.”
This way, the project will ensure a positive impact of extended reality technologies on the safety, security, trustworthiness, and societal aspects of the interaction between the machine, the human operator, and the public. Rooker continues: “Being able to interact with the machine using their inherent human intuition, operators should experience an increased sense of self-efficacy and meaningfulness in their work. Ultimately, that is our goal.”
