MUSE: Designing Immersive Virtual Realities for Spaceflight UX Research

Virtual Reality (VR) and Extended Reality (XR) technologies are increasingly prominent tools in astronaut training and space mission preparation, primarily due to their development flexibility, reduced physical risks to testers, and cost efficiency at large. Traditional astronaut training methods, such as analogue environments, neutral buoyancy pools, and parabolic flights remain invaluable, but they are also increasingly resource-intensive, may require additional specific training, entail physical risks, and may have limitations concerning repeatability and scenario realism. In contrast, VR and XR offer highly controlled and reproducible simulation environments that facilitate extensive rehearsal of operational tasks with significantly lower resource demands [19, 3]. There remain several challenges, too, regarding the use of VR for training, particularly aspects of technology constraints, user comfort and user interface design needs [16].

With these challenges and opportunities in mind, VR and XR technologies are studied to anticipate their potential to support human and robotic mission training in the field of space systems development. Immersive technologies, such as immersive VR experiences, have been identified for their opportunity to support task-focused training and machine operation, largely focused on areas of skill development and performing specific procedural tasks that support memorization and recall [12]. Other examples of immersive VR use include engaging with the user in a more personalized level in areas of customized training, team building, inspiration and motivation. Various studies also demonstrate VR environments’ potential for design and evaluation of usability [10, 19, 18, 8]. The need for immersive enrichment in VR-based scenario training has been clearly identified within European Space Agency’s (ESA) operational contexts, extensively used for both astronaut training and as an aspects of operational ergonomics during early mission design. For example, in a case study on EL3 lander done at the European Astronaut Center (EAC) by Nilsson et al (2022) the aspects of structured timelines and scenario-based interaction in benefit of enhancing immersion and crafting meaningful scenarios are highlighted. Structuring the VR experience narrative for immersive realism was seen crucial for the value of the experience [19].

Narrative elements in VR experiences are more commonly employed in VR games and cultural visual experiences, where they can host aspects of psychological realism, storytelling, and emotionally engaging action [11, 17]. Recent research indicates that immersion within narrative-rich virtual experiences significantly enhances user creativity and the capacity for insight-based problem-solving by evoking awe and promoting creative problem-solving through deeper cognitive and emotional engagement [17, 11, 25, 14].

Game design theory and narrative design methodologies offer useful tools for structuring immersive virtual reality (VR) environments, particularly in contexts of training new tasks and speculative scenario development. Frameworks like Freytag’s Pyramid and the Hero’s Journey, are commonly used in interactive media and serve not only as storytelling devices but also as models for guiding emotional pacing, cognitive engagement, and decision-making within a simulated context [21, 4, 1]. These narrative structures can be aligned with Csikszentmihalyi’s theory of Flow, where users experience a high level of focus and involvement when the balance between task difficulty and user skill is carefully managed [7]. When integrated into VR simulations, these frameworks can support emotional involvement in the simulated world and promote intuitive responses to emerging events. This aligns with Carpenter’s description of insight as a moment of sudden understanding born from unconscious cognitive processing, contributing to adaptive thinking and creative way to solve problems [5].

From a user interface (UI) and user experience (UX) perspective, narrative immersion offers several practical benefits. It can reduce cognitive load, support attentional focus, and enable users to respond more effectively in high-pressure operational contexts [2, 23]. Narrative components in user interfaces may also help with memory retention and emotional relevance, which can be especially helpful in training settings where task repetition alone may not address all learning goals. By applying narrative-oriented methods from game design, developers can create virtual training tools that foster adaptability, creativity, and engagement. These qualities are relevant for astronaut task and mission preparation, particularly in complex scenarios involving different tasks [15, 25, 19]. Including game design methodology more centrally in design of VR experiences can further advance the development of systematic design practices that link performance requirements of these experiences with the cognitive and emotional dimensions of users in space environments.

The scenario developed for this research adopted iterative and participatory design methods, explicitly structured around the well-established narrative framework called Freytag’s Pyramid, consisting of clearly defined phases including a rising action, climax and falling action, to create a dramatic and comprehensible narrative arc. This structured approach (see Fig. 1) facilitated operational realism of having a structured phasing to the mission and immersive storytelling by providing users with emotional and cognitive pacing for a deeper immersion and enhanced user engagement throughout the experience [21, 4].

Based on ESA’s Human Inspirator Co-Engineering (HICE) study, the MUSE VR experience aimed to assess the technical and programmatic feasibility of extended human missions, including existing and new opportunities to Low Earth Orbit (LEO), Lunar Gateway station, and the lunar surface. This initiative offered multiple realistic use-cases, of which one scenario was adopted as the narrative framework for this research (see Fig. 2). Following the recommendations and preliminary findings of the HICE study, the scenario closely adhered to the identified challenges and opportunities of crewed missions starting from LEO and culminating in docking operations at the Lunar Gateway station, including astronaut transfer from the spacecraft to the station. The architectural elements of the spacecraft, particularly its capsule environment and its configurations, were likewise informed by the HICE study outcomes, ensuring that the virtual experience maintained authenticity and relevance to current real-world considerations while offering valuable insights into user interface (UI) and user experience (UX) design.

Given that mission timelines and specific operational details within the HICE study remain speculative, the scenario and spacecraft design required considerable adaptability to consider the needs of several stakeholders and technology opportunities. Thus, a modularization strategy was adopted (see Fig. 3 ). Leveraging the flexibility provided by Unreal Engine 5.3. VR development pipeline the modularity enabled interchangeable scenario components, and 3D model configurations, addressing evolving knowledge and iterative refinements throughout development. This approach allowed scenario elements such as spacecraft modules and narrative mission parts to be quickly adapted or replaced as new insights emerged, enhancing responsiveness to changing scenario understanding and facilitating future scalability and refinements in the training experience [6].

While inspired by existing and conceptual spacecraft designs, such as the Orion and Crew Dragon capsules, the spacecraft developed for the virtual scenario in MUSE incorporated further findings from the HICE study, such as the earlier mentioned need for adaptability to evolving mission design requirements. The design aimed to balance narrative-driven immersion with operational authenticity by managing constraints like spacecraft scale, visual fidelity, and internal functionality through strategic planning of interchangeable and adjustable parts (see Fig. 3 and Fig. 5). This modular methodology aligns with established best practices from previous immersive virtual experiences developed at ESA, where structural flexibility and adaptability have supported ongoing iteration for mission prototyping and user testing [19, 2].

In addition to the physical and spatial design, narrative immersion was further grounded through the integration of a voice-loop system modeled on real docking procedures with Crew Dragon spacecraft and the International Space Station (ISS). Audio material was referenced from public Apollo mission archives and supplemented by live dialogue data recorded during a recent Crew Dragon docking operations. This audio was simplified, processed and adapted into a three-way communication structure that reflects the style and content of authentic operational voice loops. By embedding this system into the VR narrative, the experience preserved operational realism, for example in simulating realistic challenges in docking operations, while introducing new narrative elements, effectively bridging fictional scenario development with authentic astronaut communication dynamics.

Special emphasis was placed on the spacecraft capsule design, carefully refining both external geometry and internal layouts to align closely with the HICE study’s identified goals, including habitability, ergonomics, and operational efficiency. Modularity of 3D parts ensured that the spacecraft interior and scenario interactions could remain dynamically aligned with evolving mission requirements and new ergonomic insights as they arose through ongoing research and user feedback. Modular design allows for targeted testing and training scenarios, offering insights into user interaction design, mission planning, and decision-making effectiveness across diverse user groups, from mission planners to astronauts [6].

Interior design specifically emphasized immersive user experiences, aligning narrative interactions with functional and ergonomic realities of the spacecraft environment (see Fig. 4), such as identified division of onboard workspace intended for the VR user [20]. Environmental storytelling techniques, such as integrating a narrator’s voice, was added to enhance intuitive decision-making processes. This narrative approach aimed to complement traditional heuristic-based training methods by actively facilitating intuitive, insight-driven strategies among users, and fostering deeper cognitive engagement and situational awareness [5, 11, 10].

The evaluation framework combined semi-structured quantitative assessments and qualitative exploratory approaches, employing think-aloud protocols, observational methods, structured surveys, and more detailed qualitative interviews regarding the overall experience.

Our first research question required understanding both when and how much the user was engaged with the experience. We combined in our methodology the monitoring of users movements and engagement activity to understand cognitive investment at any given situation. Similar technique has been described for example in research by Sun et al. [24], to observe how users physically engaged with critical scenario events. User behaviors, such as turning towards audio cues, manipulating interface elements, or physically leaning into tasks, were logged and cross-referenced with quantitative post-experience assessment. Makransky et al. (2019) emphasize the value of combining self-report measures (e.g., enjoyment, motivation, and self-efficacy) with behavioral transfer tests to comprehensively evaluate immersive VR training [14]. Emphasis was therefor placed on capturing intuitive decision-making processes in the moment and conducting an after testing qualitative interview [23, 16, 19].

To explore our second research question, the VR simulation incorporated a simplified Situation Awareness Global Assessment Technique (SAGAT) to assess user responses to modular capsule environments. In particular this was done by measuring user responsiveness to operational, mostly audio cues through timed reactions, underlining how narrative-driven scenarios influenced intuitive decision-making during operation. Furthermore a situational awareness rating technique was used as a quantitative method and self assessment after the testing, keeping in mind that self-rating after testing is subject to for example poor recall rates [6, 9, 22].

These methods enabled us to look at immersion patterns and align user reactions with narrative turning points within the experience. This UX assessment strategy was the foundation for exploring cognitive processes like intuitive decision-making and situational awareness in narrative-driven VR scenarios.

The participants were selected with a view to having a high degree of role variability as stakeholders for the MUSE experience. Before the actual testing the users were identified for their potential stakeholder role and experience of using VR, and invited individually into the testing session. Upon entering the testing session the user was greeted and offered a brief overlook into the upcoming testing, and given a quick introduction to using the VR-headset and hand controls before donning the headset. Users were also informed about the general narrative goal of the experience, flying onboard a small spacecraft to the Lunar Gateway and being tested for situational awareness during docking operations. Similarly to the testing setup done by Nilsson et al. for the EL3 lander, the users were encouraged to verbalize their experiences, and reasoning during the docking operations [19]. After the testing session, which took about 10-15 minutes, depending on the users curiosity to explore the experience longer, the users were asked to fill in two questionnaires right after the testing, and further verbalize their thoughts in a general manner regarding the two research goals we had.

The user testing involved surveys, observational protocols, and emotional and physical response tracking to help heuristic evaluation and apply a multi-level assessment [23, 16, 8]. Collecting such varied data however posed a challenge of how to combine the data in a meaningful way to offer insight in the best way. The testing did not include tools, such as eye or position tracking, or measure precise moment when users pressed for example a button. The test included two monitors who marked the information regarding these events manually. The users were instructed before the testing of the correct way to react during the situational awareness test to tick a box (press a button) in the VR interface on the capsules command panel. Outside the docking operations the users were monitored also for their other physical reactivity, such as naturally occurring reactions towards narrative elements when there was no particular voiced command to interact with the Command Panel.

The experience was structured into sequential narrative stages: beginning with mission preparations in Low Earth Orbit (LEO), followed by lunar flyby transit, and concluding with approach, docking operations and entering the Lunar Gateway. While one phase was focused on situational awareness and explicitly incorporated a task-based cognitive evaluation (see Fig. 6 and Fig. 7), user feedback and observations across the entire experience consistently revealed high levels of emotional and cognitive engagement. In our assessment of the outcomes it seemed that the relative experience of an individual dealing with similar operational scenarios correlated with faster and more precise reaction times during docking operations.

Participants described a strong sense of excitement and curiosity throughout, particularly in quieter moments before and after operational tasks. Many explored the interior environment of the capsule early to the experience, moving between seats and physically leaning toward the windows to observe landmarks passing by the window such as Earth, the Moon, and the Gateway station (see Fig. 8). These patterns mirror findings by Sun et al. [24], who demonstrated that heightened movement within VR scenarios correlates with stronger presence and task engagement. While functional interaction was concentrated around the command panel due to its central role in the experience management, user movement patterns also indicated spontaneous interest in non-operational elements of the environment. However, participants generally avoided interacting with passive props like cargo racks, indicating selective engagement shaped by narrative relevance.

These behavioral insights align with Makransky et al.’s [14] findings on immersive learning, which emphasize that immersive VR environments are most effective when users are emotionally engaged and immersed, even if not all elements contribute to learning actively. In our study, physical engagement often increased in anticipation of narrative turning points, reinforcing the important role of storytelling in guiding embodied presence. This also resonates with Carpenter’s work on insight and creative cognition, which highlights that intuitive realizations of information can arise during periods of passive observation or cognitive rest [5]. These moments of exploration and reflective curiosity may thus support the conditions necessary for intuitive insight, underscoring the value of designing narrative structures that allow for such psychological incubation.

Despite the overall success in immersion and reports of generally good user experience, several usability concerns emerged during the whole length of the experience (see Fig. 9). Participants occasionally struggled to return to optimal viewing positions after moving around the capsule (spatial navigation issue), sometimes leading to confusion or diminished visibility of essential UI elements like the mission timeline or command panel. During the situational awareness testing, some users inadvertently marked mission steps as complete prematurely. These incidents were often due to misunderstandings of audio cues (audio instruction issue), over-anticipation of events to come, or accidental controller contact with the user interface in the limited capsule space represented in the VR experience around them. Such errors emphasize the importance of multi-modal narrative design, combining clear audio cues, visual reinforcement, and spatial UI alignment to support reliable user interaction. Participants also reported cases of immersion being abruptly disrupted by external environmental noises, highlighting the fragile nature of deep presence and the need for consistent sensory control in high-fidelity training simulations, especially for experiences that rely on audio ques and concentrating on listening activities [8, 19, 2, 25].

Importantly, instances of intuitive decision-making were observed, especially when users acted under uncertainty, when they were either misinterpreting technical terminology in the audio loop or attempting to multitask. These moments, although sometimes resulting in operational errors, are informative for understanding how narrative and scenario structure influence not only task accuracy but also individual intuitive cognitive processing. Awareness of such insight-based behavior complements earlier findings by Makransky et al. [14], who advocate for multi-level assessments for VR use. In narrative VR contexts, this represents an insight towards testing individual situational adaptation in both active and passive stages of the experience, rather than merely during the actual mission procedure.