Abstract
We revealed the optimal feedback parameters of visual, auditory, and tactile modalities in the operation of aerial display and determined that aerial display could be further improved for use of as a contactless human interface by multimodal feedback combining these parameters. We proposed applying a vibration motor to the sole of the foot instead of fingertips to provide vibration as tactile feedback to operate the aerial display. Sensory evaluation tests were carried out to evaluate the optimal feedback parameters for operating the aerial display in three senses: visual (image change); auditory (touch tone); and tactile (vibration), and to evaluate the effect on the operation with a combination of these parameters. We analyzed the variance of the experimental data using Scheffé’s method for obtaining significance levels of paired comparisons. We found that the combination of two or three modalities rather than only a single modality makes the aerial display easy to operate.
Similar content being viewed by others
Change history
17 February 2024
A Correction to this paper has been published: https://doi.org/10.1007/s10043-024-00869-5
References
Abe, E., Yasugi, M., Takeuchi, H., Watanabe, E., Kamei, Y., Yamamoto, H.: Development of omnidirectional aerial display with aerial imaging by retro-reflection (AIRR) for behavioral biology experiments. Optical Rev. 23, 221–229 (2019)
Yamamoto, H., Tomiyama, Y., Suyama, S.: Floating aerial LED signage based on aerial image by retro-reflection (AIRR). Opt. Express 22(22), 26919–26924 (2014)
Miyazaki, D., Hirano, N., Maeda, Y., Yamamoto, S., Mukai, T., Maekawa, S.: Floating volumetric image formation using a dihedral corner reflector array device. Appl. Opt. 52(1), A281–A289 (2013)
Yoshida, T., Shimizu, K., Kurogi, T., Kamuro, S., Minamizawa, K., Nii, H., Tachi, S.: RePro3D: full-parallax 3D display with haptic feedback using retro-reflective projection technology. IEEE International Symposium on VR Innovation 49–54 (2011)
Hasegawa, K., Shinoda, H.: Aerial vibrotactile display based on multiunit ultrasound phased array. IEEE Trans. Haptics 11, 367–377 (2018)
Hoshi, T., Takahashi, M., Iwamoto, T., Shinoda, H.: Noncontact tactile display based on radiation pressure of airborne ultrasound. IEEE Trans. Haptics 3(3), 155–165 (2010)
Rakkolainen, I., Sand, A., Raisamo, R.: A survey of mid-Air ultrasonic tactile feedback. IEEE International Symposium on Multimedia, 94–98 (2019)
Ochiai, Y., Kumagai, K., Hoshi, T., Rekimoto, J., Hasegawa, S., Hayasaki, Y.: Fairy lights in femtoseconds: Aerial and volumetric graphics rendered by focused femtosecond laser combined with computational holographic fields. ACM Trans. Graphics 35(2), 1–14 (2016)
Suzuki, Y., Kobayashi, M.: Air jet driven force feedback in virtual reality. IEEE Comput. Graphics Appl. 25, 44–47 (2005)
Gupta, S., Morris, D., Patal, S.N., Tan, D.: AirWave: Non-contact haptic feedback using air vortex rings. In: Proceedings of the 2013 ACM International Joint Conference on Pervasive and ubiquitous computing 419–428 (2013)
Brewster, S., Chohan, F., Brown, L.: Tactile feedback for mobile interactions. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems 159–162 (2007)
Politis, I., Brewster, S., Pollick, F.: Evaluating multimodal driver displays under varying situational urgency. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems 4067–4076 (2014)
Lee, J., Spence, C.: Assessing the benefits of multimodal feedback on dual-task performance under demanding conditions. In: Proceedings of the 22nd British HCI Group Annual Conference on People and Computers Culture, Creativity, Interaction 1, 185–192 (2008)
Lee, J., Poliakoff, E., Spence, C.: The effect of multimodal feedback presented via a touch screen on the performance of older adults. In: Haptic and Audio Interaction Design, 4th International Conference, HAID 2009, Dresden, Germany, Proceedings 128–135 (2009)
Rosam, N., Hurst, W., Vos, W., Werkhoven, P.: The influence of visual cues on passive tactile sensations in a multimodal immersive virtual environment. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction 327–334 (2015)
Scheffe, H.: An analysis of variance for paired comparisons. J. Am. Stat. Assoc. 47(259), 381–400 (1952)
Acknowledgements
This work was supported by JSPS Grants-in-Aid for Scientific Research JP19H04155, JP20H05702, JP20K21817, JP20K11919, JP22H00535 and JP23H03485, and by Hoso Bunka Foundation
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The original publication has been updated for revision in Table 1.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Terao, Y., Mizushina, H. & Yamamoto, K. Evaluation of usability improvement of contactless human interface with visual, auditory, and tactile sensation for aerial display. Opt Rev 31, 126–133 (2024). https://doi.org/10.1007/s10043-023-00853-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10043-023-00853-5