| I. Research objectives An AI pathology interaction platform was proposed by integrating the Microsoft HoloLens 2 MR glasses with an ultrasonic haptic feedback interface. This platform utilized mixed reality and image streaming to visualize virtual control elements and floating pathology slide images. The modular ultrasonic haptic feedback system provided interactive feedback, allowing users to intuitively operate the platform and reduce cognitive errors. Th digital pathology platform offered physicians an advanced and intuitive pathology operation experience by using big data processing for pathology display technology and AI-assisted interpretation of pathology slides. |
|
|
|
|
Figure 1. Architecture of contactless interaction pathology system. |
|
| II. Research content Our laboratory utilized the Leap Motion camera for gesture and skeletal point recognition and Unity for designing virtual control elements to construct MR interaction platform [2], [3]. Leap Motion captures skeletal feature points using a hand joint recognition model. The feature points were processed through 3D joint world coordinate transformation and a Kalman filter to reconstruct the user's hand and interact with platform control elements in the virtual environment. |
||
|
||
Figure 2. Architecture of MR interaction technology |
||
|
||
Figure 3. Control components for MR interaction. |
||
| III. Achievement |
||
|
||
Figure 4. Silver Medal Award of 22nd Macronix Golden Silicon Awards Semiconductor design and Application Competition |
||
|
||
Figure 5. Certificate of Scholarship of CTCI Foundation |
||
|
||
Figure 6. Silver Award of 2022 Advanced Display Technology (AR/VR/MR/XR) Innovative Project Contest |
||
