Document Open Access Logo

Assessing the Use of Mixed Reality as a Valid Tool for Human-Robot Interaction Studies in the Context of Space Exploration

Authors Enrico Guerra , Sebastian Thomas Büttner , Alper Beşer , Michael Prilla

PDF HTML 5

Files

Thumbnail PDF
PDF
OASIcs.SpaceCHI.2025.27.pdf
  • Filesize: 474 kB
  • 11 pages
HTML5 Logo HTML (experimental)
OASIcs.SpaceCHI.2025.27.html

Document Identifiers

Subject Classification

ACM Subject Classification
  • Human-centered computing → Mixed / augmented reality
Keywords
  • Mixed Reality
  • Augmented Reality
  • Human-Robot Interaction
  • Space Exploration
  • Validity

Metrics

Abstract

Mixed Reality (MR) is a technology with strong potential for advancing research in Human-Robot Interaction (HRI) for space exploration. Apart from the efficiency and high flexibility MR can offer, we argue that its benefits for HRI research in space contexts lies particularly in its ability to aid human-in-the-loop development, offer realistic hybrid simulations, and foster broader participation in HRI research in the space exploration context. However, we believe that this is only plausible if MR-based simulations can yield comparable results to fully physical approaches in human-centred studies. In this position paper, we highlight several arguments in favour of MR as a tool for space HRI research, while emphasising the importance of the open question regarding its scientific validity. We believe MR could become a central tool for preparing for future human-robotic space exploration missions and significantly diversify research in this domain.

Cite As Get BibTex

Enrico Guerra, Sebastian Thomas Büttner, Alper Beşer, and Michael Prilla. Assessing the Use of Mixed Reality as a Valid Tool for Human-Robot Interaction Studies in the Context of Space Exploration. In Advancing Human-Computer Interaction for Space Exploration (SpaceCHI 2025). Open Access Series in Informatics (OASIcs), Volume 130, pp. 27:1-27:11, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/OASIcs.SpaceCHI.2025.27

Author Details

Enrico Guerra
  • Interactive Systems, University of Duisburg-Essen, Duisburg, Germany
Sebastian Thomas Büttner
  • Department of Computer Science and Communications, Westfälische Hochschule – , Westphalian University of Applied Sciences, Gelsenkirchen, Germany
Alper Beşer
  • Interactive Systems, University of Duisburg-Essen, Duisburg, Germany
Michael Prilla
  • Interactive Systems, University of Duisburg-Essen, Duisburg, Germany

Funding

This work was funded by the federal state of Lower Saxony, Germany, as part of the project 11-76251-1337/2022 Kognitiv und Empathisch Intelligente Kollaborierende Roboter - "KEIKO" of the SPRUNG funding instrument.

References

  1. Basel Alhaji, Sebastian Büttner, Shushanth Sanjay Kumar, and Michael Prilla. Trust dynamics in human interaction with an industrial robot. Behaviour & Information Technology, 0(0):1-23, 2024. Publisher: Taylor & Francis _eprint: URL: https://doi.org/10.1080/0144929X.2024.2316284.
  2. Basel Alhaji, Michael Prilla, and Andreas Rausch. Trust Dynamics and Verbal Assurances in Human Robot Physical Collaboration. Frontiers in Artificial Intelligence, 4:103, 2021. URL: https://doi.org/10.3389/frai.2021.703504.
  3. Apple. Apple Vision Pro. [Online; accessed 29. Mar. 2025]. URL: https://www.apple.com/apple-vision-pro.
  4. Anna Bolder, Stefan M Grünvogel, and Emanuel Angelescu. Comparison of the usability of a car infotainment system in a mixed reality environment and in a real car. In Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology, pages 1-10, 2018. URL: https://doi.org/10.1145/3281505.3281512.
  5. Robert R. Burridge, Jeffrey Graham, Kim Shillcutt, Robert Hirsh, and David Kortenkamp. Experiments with an EVA Assistant Robot. In Experiments with an EVA Assistant Robot, United States, May 2003. NTRS Author Affiliations: Georgia Inst. of Tech., Titan Beta, NASA Johnson Space Center, Metrica, Inc. NTRS Report/Patent Number: JSC-CN-7773 NTRS Document ID: 20100033095 NTRS Research Center: Johnson Space Center (JSC). URL: https://ntrs.nasa.gov/citations/20100033095.
  6. Xiaoxia Chen, Liang Gong, Anton Berce, Björn Johansson, and Mélanie Despeisse. Implications of virtual reality on environmental sustainability in manufacturing industry: A case study. Procedia CIRP, 104:464-469, 2021. Google Scholar
  7. Martial Costantini, Flavie Rometsch, Andrea Emanuele Maria Casini, Aidan Cowley, Stephen Ennis, Christopher Scott, Stephan Ghiste, Jonathan Scott, and Lionel Ferra. eXtended Reality applications for human spaceflight: the ESA-EAC XR Lab. In 72nd International Astronautical Congress (IAC 2021, Dubai, United Arab Emirates, 2021. ISSN: 0074-1795. URL: https://elib.dlr.de/144773/.
  8. Fabio Vinicius de Freitas, Marcus Vinicius Mendes Gomes, and Ingrid Winkler. Benefits and challenges of virtual-reality-based industrial usability testing and design reviews: A patents landscape and literature review. Applied Sciences, 12(3):1755, 2022. Google Scholar
  9. Lorin Dole and Wendy Ju. Face and Ecological Validity in Simulations: Lessons from Search-and-Rescue HRI. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pages 1-8, Glasgow Scotland Uk, May 2019. ACM. URL: https://doi.org/10.1145/3290605.3300681.
  10. Florian Dufresne, Tommy Nilsson, Geoffrey Gorisse, Enrico Guerra, André Zenner, Olivier Christmann, Leonie Bensch, Nikolai Anton Callus, and Aidan Cowley. Touching the Moon: Leveraging Passive Haptics, Embodiment and Presence for Operational Assessments in Virtual Reality. In Proceedings of the CHI Conference on Human Factors in Computing Systems, pages 1-18, Honolulu HI USA, May 2024. ACM. URL: https://doi.org/10.1145/3613904.3642292.
  11. Chris Elsden, David Chatting, Abigail C. Durrant, Andrew Garbett, Bettina Nissen, John Vines, and David S. Kirk. On speculative enactments. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI '17, pages 5386-5399, New York, NY, USA, 2017. Association for Computing Machinery. URL: https://doi.org/10.1145/3025453.3025503.
  12. ESA-DLR. Home - Luna Analog Facility, January 2025. [Online; accessed 17. Mar. 2025]. URL: https://luna-analog-facility.de.
  13. Sharon L Farra, Matthew Gneuhs, Eric Hodgson, Burhan Kawosa, Elaine T Miller, Ashley Simon, Nathan Timm, and Jackie Hausfeld. Comparative cost of virtual reality training and live exercises for training hospital workers for evacuation. CIN: Computers, Informatics, Nursing, 37(9):446-454, 2019. Google Scholar
  14. Terrence Fong, Jean Scholtz, Julie A. Shah, Lorenzo Fluckiger, Clayton Kunz, David Lees, John Schreiner, Michael Siegel, Laura M. Hiatt, Illah Nourbakhsh, Reid Simmons, Brian Antonishek, Magda Bugajska, Robert Ambrose, Robert Burridge, Alan Schultz, and J. Gregory Trafton. A Preliminary Study of Peer-to-Peer Human-Robot Interaction. In 2006 IEEE International Conference on Systems, Man and Cybernetics, volume 4, pages 3198-3203, October 2006. ISSN: 1062-922X. URL: https://doi.org/10.1109/ICSMC.2006.384609.
  15. Fabio Grandi, Luca Zanni, Margherita Peruzzini, Marcello Pellicciari, and Claudia Elisabetta Campanella. A transdisciplinary digital approach for tractor’s human-centred design. International Journal of Computer Integrated Manufacturing, 33(4):377-397, 2020. URL: https://doi.org/10.1080/0951192X.2019.1599441.
  16. Enrico Guerra and Michael Prilla. Virtually Real Robots: XR as a Proxy for Physical Human-Robot Interaction. In Virtually Real Robots: XR as a Proxy for Physical Human-Robot Interaction, page 10.18420/muc2024. Gesellschaft für Informatik e.V., 2024. URL: https://doi.org/10.18420/MUC2024-MCI-WS06-204.
  17. Aakar Gupta, Bo Rui Lin, Siyi Ji, Arjav Patel, and Daniel Vogel. Replicate and Reuse: Tangible Interaction Design for Digitally-Augmented Physical Media Objects. In ACM Conferences, pages 1-12. Association for Computing Machinery, New York, NY, USA, April 2020. URL: https://doi.org/10.1145/3313831.3376139.
  18. Kimberly Hambuchen, Jessica Marquez, and Terrence Fong. A Review of NASA Human-Robot Interaction in Space. Current Robotics Reports, 2(3):265-272, September 2021. URL: https://doi.org/10.1007/s43154-021-00062-5.
  19. Zhao Han, Yifei Zhu, Albert Phan, Fernando Sandoval Garza, Amia Castro, and Tom Williams. Crossing Reality: Comparing Physical and Virtual Robot Deixis. In Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction, HRI '23, pages 152-161, New York, NY, USA, March 2023. Association for Computing Machinery. URL: https://doi.org/10.1145/3568162.3576972.
  20. Hampus Hansen, Måns Holmgren, Carl Johan Gribel, Joakim Ingrell, and George Palamas. Utilizing virtual reality to enhance situational awareness in swedish police training. In Eva Brooks, Emma Edstrand, Anders Kalsgaard Møller, and Thomas Bjørner, editors, Design, Learning, and Innovation, pages 33-50, Cham, 2025. Springer Nature Switzerland. Google Scholar
  21. Nathalie Harz, Sebastian Hohenberg, and Christian Homburg. Virtual reality in new product development: Insights from prelaunch sales forecasting for durables. Journal of Marketing, 86(3):157-179, 2022. Google Scholar
  22. Barbara Imhof, Waltraut Hoheneder, Stephen Ransom, René Waclavicek, Bob Davenport, Peter Weiss, Bernard Gardette, Virginie; Taillebot, Thibaud; Gobert, Diego Urbina, Tom Hoppenbrouwers, Thomas Vögele, Mathias Höckelmann, Jakob Schwendner, Knut R. Fossum, and Victor Parro García. Moonwalk - Human Robot Collaboration Mission Scenarios and Simulations. In AIAA SPACE 2015 Conference and Exposition, AIAA SPACE Forum. American Institute of Aeronautics and Astronautics, August 2015. URL: https://doi.org/10.2514/6.2015-4531.
  23. Hiroko Kamide, Yasushi Mae, Tomohito Takubo, Kenichi Ohara, and Tatsuo Arai. Direct comparison of psychological evaluation between virtual and real humanoids: Personal space and subjective impressions. International Journal of Human-Computer Studies, 72(5):451-459, May 2014. URL: https://doi.org/10.1016/j.ijhcs.2014.01.004.
  24. Lee Kent, Chris Snider, James Gopsill, and Ben Hicks. Mixed reality in design prototyping: A systematic review. Design Studies, 77:101046, 2021. Google Scholar
  25. Xiangfei Kong and Zhao Han. Do results in experiments with virtual robots in augmented reality transfer to physical robots? an experiment design. In Proceedings of the 4th Workshop YOUR Study Design, 2024. Google Scholar
  26. Rui Li, Marc van Almkerk, Sanne van Waveren, Elizabeth Carter, and Iolanda Leite. Comparing Human-Robot Proxemics Between Virtual Reality and the Real World. In 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pages 431-439, March 2019. ISSN: 2167-2148. URL: https://doi.org/10.1109/HRI.2019.8673116.
  27. Joseph Lindley and Paul Coulton. Back to the future: 10 years of design fiction. In Proceedings of the 2015 British HCI Conference, British HCI '15, pages 210-211, New York, NY, USA, 2015. Association for Computing Machinery. URL: https://doi.org/10.1145/2783446.2783592.
  28. Chao Ma and Ting Han. Combining virtual reality (vr) technology with physical models-a new way for human-vehicle interaction simulation and usability evaluation. In HCI in Mobility, Transport, and Automotive Systems: First International Conference, MobiTAS 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Orlando, FL, USA, July 26-31, 2019, Proceedings 21, pages 145-160. Springer, 2019. URL: https://doi.org/10.1007/978-3-030-22666-4_11.
  29. Meta. Meta Quest 3. [Online; accessed 29. Mar. 2025]. URL: https://www.meta.com/de/quest.
  30. Tonia Mielke, Mareen Allgaier, Danny Schott, Christian Hansen, and Florian Heinrich. Virtual Studies, Real Results? Assessing the Impact of Virtualization on Human-Robot Interaction. In Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, CHI EA '25, pages 1-8, New York, NY, USA, April 2025. Association for Computing Machinery. URL: https://doi.org/10.1145/3706599.3719724.
  31. Paul Milgram, Haruo Takemura, Akira Utsumi, and Fumio Kishino. Augmented reality: A class of displays on the reality-virtuality continuum. In Telemanipulator and telepresence technologies, volume 2351, pages 282-292. Spie, 1995. Google Scholar
  32. NASA. Analog Missions - NASA, March 2025. [Online; accessed 17. Mar. 2025]. URL: https://www.nasa.gov/analog-missions.
  33. NASA. Neutral Buoyancy Laboratory, April 2025. [Online; accessed 1. Apr. 2025]. URL: https://www.nasa.gov/johnson/neutral-buoyancy-laboratory.
  34. Federica Nenna, Valeria Orso, Davide Zanardi, and Luciano Gamberini. The virtualization of human–robot interactions: a user-centric workload assessment. Virtual Reality, 27(2):553-571, June 2023. URL: https://doi.org/10.1007/s10055-022-00667-x.
  35. Tommy Nilsson, Leonie Bensch, Florian Dufresne, Flavie Rometsch, Paul de Medeiros, Enrico Guerra, Florian Saling, Andrea EM Casini, and Aidan Cowley. Out of this World Design: Bridging the Gap between Space Systems Engineering and Participatory Design Practices. In Proceedings of SpaceCHI 3.0, A Conference on Human-Computer Interaction for Space Exploration (SpaceCHI 3.0). MIT Media Lab, MA, USA, 2023. URL: https://creating-worlds.com/wp-content/uploads/2024/01/Out_of_this_World_Design.pdf.
  36. Tommy Nilsson, Flavie Rometsch, Leonie Becker, Florian Dufresne, Paul Demedeiros, Enrico Guerra, Andrea Emanuele Maria Casini, Anna Vock, Florian Gaeremynck, and Aidan Cowley. Using Virtual Reality to Shape Humanity’s Return to the Moon: Key Takeaways from a Design Study. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1-16, Hamburg Germany, April 2023. ACM. URL: https://doi.org/10.1145/3544548.3580718.
  37. Divya Dolly Patel and Denise Y. Geiskkovitch. The Space Between Us: Bridging Human and Robotic Worlds in Space Exploration. In Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, HRI '24, pages 833-836, New York, NY, USA, 2024. Association for Computing Machinery. URL: https://doi.org/10.1145/3610978.3640727.
  38. Florian Saling, Andrea Emanuele Maria Casini, Andreas Treuer, Martial Costantini, Leonie Bensch, Tommy Nilsson, and Lionel Ferra. Testing and validation of innovative eXtended Reality technologies for astronaut training in a partial-gravity parabolic flight campaign, October 2024. arXiv:2410.14922 [cs]. URL: https://doi.org/10.48550/arXiv.2410.14922.
  39. EXPLORATION KJ Snook. The need for analogue missions in scientific human and robotic planetary. Lunar and Planetary Science, 2024. Google Scholar
  40. Ryo Suzuki, Adnan Karim, Tian Xia, Hooman Hedayati, and Nicolai Marquardt. Augmented Reality and Robotics: A Survey and Taxonomy for AR-enhanced Human-Robot Interaction and Robotic Interfaces. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI '22, pages 1-33, New York, NY, USA, April 2022. Association for Computing Machinery. URL: https://doi.org/10.1145/3491102.3517719.
  41. Unknown. Astronaut training in the land of volcanoes, March 2025. [Online; accessed 25. Mar. 2025]. URL: https://www.esa.int/Science_Exploration/Human_and_Robotic_Exploration/Astronaut_training_in_the_land_of_volcanoes.
  42. Varjo. Varjo XR-4 Series. [Online; accessed 29. Mar. 2025]. URL: https://varjo.com/products/xr-4.
  43. Valeria Villani, Beatrice Capelli, and Lorenzo Sabattini. Use of Virtual Reality for the Evaluation of Human-Robot Interaction Systems in Complex Scenarios. In 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pages 422-427, Nanjing, August 2018. IEEE. URL: https://doi.org/10.1109/ROMAN.2018.8525738.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact