ACE4ACES | Multimodal interfaces for future combat aircraft

Region:

Federal

Financed by

Advanced displays, gaze and gesture tracking, and AI tosupport decision-making

Combat aircraft operate in increasingly connected environments, where pilots must manage growing volumes of information and interact with unmanned systems. ACE4ACES develops multimodal cockpit interfaces to reduce this workload and facilitate decision-making. The project combines advanced displays, gaze and gesture tracking, and AI-assisted communications.

Target group

The project is primarily aimed at:

Aerospace and defence companies
Aircraft manufacturers and avionics suppliers
Developers of cockpit and human-machine interface technologies
Companies specialising in displays, sensors, AI and communication systems

Context

Air operations are becoming increasingly connected and complex. Manned aircraft, unmanned systems and other platforms need to exchange increasing amounts of information in real time. Pilots must interpret and prioritise these multiple data streams to maketimely, informed decisions.

Current interfaces are reaching their limits in the face of this complexity. New forms of interaction are needed to present relevant information at the right time, while meeting strict requirements for reliability, safety, latency and availability.

Objectives & results

The project aims to:

Improve situational awareness and information management
Facilitate interaction between pilots, onboard systems and unmanned systems
Demonstrate and evaluate the developed technologies at Technology Readiness Level 4 (TRL 4)

Key results:

A prototype helmet-mounted augmented reality display combining holographic technology, gaze tracking and gesture-based interaction
A fault-tolerant Large Area Display integrating multiple modes of interaction
Improved gaze, head and gesture tracking technologies
AI-assisted solutions for processing and prioritising communication and data streams
Demonstrators to evaluate the technologies in different human-machine interaction scenarios
Technologies not subject to ITAR restrictions, contributing to Belgian and European technological sovereignty
Publications and workshops to share the project results

Approach

ACE4ACES brings together Belgian partners with expertise in displays, sensors, artificial intelligence, communications and applied research. The project focuses on three complementary technology areas.

1. Advanced cockpit displays
The partners are developing a high-resolution, fault-tolerant Large Area Display. In parallel, the project is exploring a helmet-mounted augmented reality system based on holographic display technology with multiple focal planes, a wide field of view and high brightness.

2. Gaze and gesture-based interaction
The project investigates gaze and head tracking as well as event-based sensors to enable fast, contactless interaction. The research focuses in particular on accuracy, robustness, latency and computational requirements.

3. AI-assisted communications
Natural language processing and intelligent data prioritisation are used to select, enhance and transcribe communication streams according to their context and relevance.

These technologies will be integrated into demonstrators and evaluated across a range of human-machine interaction scenarios.