An automotive interactive interface was proposed by integrating floating projection, ultrasonic haptic feedback, gesture recognition, and driving information. Drivers can interact with the floating interface by using virtual components and gestures. The integration of the ultrasonic haptic feedback device ensured that drivers experience detailed tactile feedback during operation without notification at the display to enhance driving safety.

Figure 1. Advantage of Holographic Automotive Electronic Control System

The system was divided into a floating projection interface and an ultrasonic haptic feedback system. The floating projection employed dual orthogonal reflector technology, projecting the display screen through a special optical structure to create a mid-air interface without any physical medium. Leap Motion is integrated for hand recognition, enabling direct hand operation in three-dimensional space. The ultrasonic haptic feedback system is based on an array of 512 ultrasonic transducers arranged to converge strong sound pressure. By using amplitude and spatio-temporal focal modulation, tactile points perceptible to the human body were generated. In addition, the system controlled the phase, PWM, and modulation frequency of the driving signals to produce various tactile sensations, simulating the feel of objects within the floating interface. This enhanced user interaction feedback and operational efficiency.

The system primarily interacted with the floating interface using fingertips and finger pads, supported by gestures such as tapping, grabbing, rotating, and dragging. To enhance the accuracy of gesture and fingertip positioning, the system incorporated coordinate overlay and weighting algorithms, and employs a Kalman linear filter to eliminate coordinate jitter.

To improve safety while the vehicle is in motion, the interaction algorithm integrated a Random Forest (RF) algorithm for hand movement recognition, providing a variety of interactive gesture operations, aiming to reduce the time that drivers spend looking at the interface and to provide tactile feedback for interface information. The RF algorithm classifies the current hand skeletal point coordinates into various actions to determine the user's gesture for interface operation. RF has the capability to extract and select the most important features to build decision trees. A RF consists of multiple decision trees generated from random training data, which vote to determine the final classification result. This approach effectively mitigates the overfitting issue of single decision trees. Due to its low computational complexity, the RF algorithm achieves real-time gesture classification, enhancing the diversity and usability of interface operations.

The control interface for automotive systems included a main menu, vehicle information, navigation, music player, and air conditioning. The interface featured various interactive elements such as buttons for selecting different functions, sliders for adjusting music playback progress and air conditioning airflow, knobs for adjusting volume, and a dual-zone keyboard for adjusting air conditioning temperature.