Document Open Access Logo

Multidimensional Usability Assessment in Spaceflight Analog Missions

Authors Shivang Shelat , Katherine E. Homer, John A. Karasinski, Jessica J. Marquez

PDF HTML 5

Files

Thumbnail PDF
PDF
OASIcs.SpaceCHI.2025.3.pdf
  • Filesize: 1.05 MB
  • 9 pages
HTML5 Logo HTML (experimental)
OASIcs.SpaceCHI.2025.3.html

Document Identifiers

Subject Classification

ACM Subject Classification
  • Human-centered computing → HCI design and evaluation methods
Keywords
  • space usability
  • crew autonomy
  • self-scheduling software

Metrics

Abstract

Software planning tools enable the self-scheduling of operational timelines during spaceflight, reducing reliance on ground support crews. Here, we assess analog crew perceptions of a planning tool’s usability across two space mission analogs with two validated questionnaires: the unidimensional System Usability Scale and the multidimensional User Experience Questionnaire. Critically, half the missions had assistive countermeasures integrated into the planning software interface whereas the other half did not. Correlation tests revealed high convergence between usability measures in the spaceflight analog setting. Group comparisons showed that the interface countermeasures enhanced several dimensions of usability, particularly for perceptions of the tool’s efficiency and dependability. These findings highlight the utility of a multidimensional approach to characterizing usability in order to capture fine-grained shifts in human-software dynamics, especially in spaceflight environments.

Cite As Get BibTex

Shivang Shelat, Katherine E. Homer, John A. Karasinski, and Jessica J. Marquez. Multidimensional Usability Assessment in Spaceflight Analog Missions. In Advancing Human-Computer Interaction for Space Exploration (SpaceCHI 2025). Open Access Series in Informatics (OASIcs), Volume 130, pp. 3:1-3:9, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/OASIcs.SpaceCHI.2025.3

Author Details

Shivang Shelat
  • San Jose State University, Moffett Field, CA, USA
Katherine E. Homer
  • NASA Ames Research Center, Moffett Field, CA, USA
John A. Karasinski
  • NASA Ames Research Center, Moffett Field, CA, USA
Jessica J. Marquez
  • NASA Ames Research Center, Moffett Field, CA, USA

Funding

This research was funded by the NASA Human Research Program’s Human Factors and Behavior Performance Element (NASA Program Announcement number 80JSC017N0001-BPBA) Human Capabilities Assessment for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR) effort.

References

  1. NASA Human Integration Design Handbook (HIDH), 2010. Google Scholar
  2. ISO 9241-210:2019, 2019. URL: https://www.iso.org/standard/77520.html.
  3. Renee Abbott, John Karasinski, and Jessica J. Marquez. Characterizing Spontaneous Self-Scheduling in NASA’s Human Exploration Research Analog Campaign 6. In IEEE Aerospace Conference, 2025. Google Scholar
  4. Kritina Anderson and Ian Robertson. Development and validation of the NASA modified System Usability Scale (NMSUS): a brief summary, November 2022. URL: https://ntrs.nasa.gov/citations/20220017395.
  5. John Brooke. SUS — a quick and dirty usability scale. Usability evaluation in industry, 189:4-7, 1996. Google Scholar
  6. Tom Bullock, James C. Elliott, John T. Serences, and Barry Giesbrecht. Acute exercise modulates feature-selective responses in human cortex. Journal of Cognitive Neuroscience, 29(4):605-618, April 2017. URL: https://doi.org/10.1162/jocn_a_01082.
  7. Kelly A. Burke, David J. Wing, and Mark Haynes. Flight test assessments of pilot workload, system usability, and situation awareness of tasar. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 60(1):61-65, September 2016. URL: https://doi.org/10.1177/1541931213601014.
  8. Jacob Cohen. Statistical power analysis for the behavioral sciences. Taylor and Francis, Hoboken, 2nd ed edition, 2013. Google Scholar
  9. Andreas Hinderks, Martin Schrepp, Maria Rauschenberger, and Jörg Thomaschewski. Reconstruction and validation of the UX factor trust for the User Experience Questionnaire Plus (UEQ+):. In Proceedings of the 19th International Conference on Web Information Systems and Technologies, pages 319-329, Rome, Italy, 2023. SCITEPRESS - Science and Technology Publications. URL: https://doi.org/10.5220/0012186700003584.
  10. John A. Karasinski, Shivang Shelat, and Jessica J. Marquez. Validation of self-scheduling countermeasures in NASA’s HERA Campaign 6. In AIAA SciTech Forum and Exposition, January 2025. URL: https://ntrs.nasa.gov/citations/20240015124.
  11. Andreas M. Klein, Jessica Kollmorgen, Andreas Hinderks, Martin Schrepp, Maria Rauschenberger, and Maria-Jose Escalona. Validation of the UEQ+ scales for voice quality. Computer Standards & Interfaces, 93:103971, April 2025. URL: https://doi.org/10.1016/j.csi.2024.103971.
  12. Bettina Laugwitz, Theo Held, and Martin Schrepp. Construction and evaluation of a user experience questionnaire. In Andreas Holzinger, editor, HCI and Usability for Education and Work, volume 5298, pages 63-76. Springer Berlin Heidelberg, Berlin, Heidelberg, 2008. URL: https://doi.org/10.1007/978-3-540-89350-9_6.
  13. James R. Lewis and Jeff Sauro. Item benchmarks for the system usability scale - jux, May 2018. URL: https://uxpajournal.org/item-benchmarks-system-usability-scale-sus/.
  14. Jessica J. Marquez, Tamsyn Edwards, John A. Karasinski, Candice N. Lee, Megan C. Shyr, Casey L. Miller, and Summer L. Brandt. Human performance of novice schedulers for complex spaceflight operations timelines. Human Factors: The Journal of the Human Factors and Ergonomics Society, 65(6):1183-1198, September 2023. URL: https://doi.org/10.1177/00187208211058913.
  15. Jessica J. Marquez, Steven Hillenius, Bob Kanefsky, Jimin Zheng, Ivonne Deliz, and Marcum Reagan. Increasing crew autonomy for long duration exploration missions: Self-scheduling. In 2017 IEEE Aerospace Conference, pages 1-10, Big Sky, MT, USA, March 2017. IEEE. URL: https://doi.org/10.1109/AERO.2017.7943838.
  16. Jessica J. Marquez, Shivang Shelat, and John A. Karasinski. Promoting crew autonomy in a human spaceflight earth analog mission through self-scheduling. In Accelerating Space Commerce, Exploration, and New Discovery (ASCEND) 2022, October 2022. URL: https://ntrs.nasa.gov/citations/20220013438.
  17. David Meza and Sarah Berndt. Usability/sentiment for the enterprise and enterprise. In Text Analytics World 2014 Conference. NASA, 2014. URL: https://ntrs.nasa.gov/api/citations/20140007413/downloads/20140007413.pdf.
  18. Jakob Nielsen. Usability engineering. AP Professional, Cambridge, Mass, 1993. Google Scholar
  19. S. Camille Peres, Tri Pham, and Ronald Phillips. Validation of the System Usability Scale (SUS): SUS in the wild. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 57(1):192-196, September 2013. URL: https://doi.org/10.1177/1541931213571043.
  20. Martin Schrepp, Andreas Hinderks, and Jörg Thomaschewski. Applying the user experience questionnaire (UEQ) in different evaluation scenarios. In Aaron Marcus, editor, Design, User Experience, and Usability. Theories, Methods, and Tools for Designing the User Experience, volume 8517, pages 383-392. Springer International Publishing, Cham, 2014. URL: https://doi.org/10.1007/978-3-319-07668-3_37.
  21. Shivang Shelat, John A. Karasinski, Erin E. Flynn-Evans, and Jessica J. Marquez. Evaluation of user experience of self-scheduling software for astronauts: defining a satisfaction baseline. In Don Harris and Wen-Chin Li, editors, Engineering Psychology and Cognitive Ergonomics, volume 13307, pages 433-445. Springer International Publishing, Cham, 2022. URL: https://doi.org/10.1007/978-3-031-06086-1_34.
  22. Shivang Shelat, Jessica J. Marquez, Jimin Zheng, and John A. Karasinski. Collaborative system usability in spaceflight analog environments through remote observations. Applied Sciences, 14(5):2005, February 2024. URL: https://doi.org/10.3390/app14052005.
  23. Dominique Winter, Andreas Hinderks, and Jörg Thomaschewski. Applicability of user experience and usability questionnaires. Journal of Universal Computer Science, January 2020. Google Scholar
  24. Jimin Zheng, Shivang M. Shelat, and Jessica J. Marquez. Facilitating crew-computer collaboration during mixed-initiative space mission planning. In SpaceCHI 3.0 A Conference for Human-Computer Interaction for Space Exploration, June 2023. URL: https://ntrs.nasa.gov/citations/20230008619.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact