Virtual Reality Prototyping Environment for Concurrent Design, Training and Rover Operations

The CASIMAR student project aims to develop a rover as a testing platform to study tools to provide EVA (Extravehicular Activity) support, to carry out search and rescue missions, and be able to carry out stand-alone missions, as well. It’s intended testing target is the LUNA analog facility, which is a joint project between the European Space Agency and the German Aerospace Center (DLR). The LUNA hall contains about $700m^2$ of regolith simulant up to a depth of $4m$ and therefore recreates the lunar surface and its environmental conditions like dust, illumination, reduced gravity and ground communication. The aim for LUNA is to develop, test, and train exploration activities, processes, and relevant technologies [27].

The renewed interest and drive towards lunar exploration provides unique challenges to establish a human presence on Earth’s moon. Space systems and tools have been designed by building and iteratively testing analog models. This resource-intensive approach is not viable for the student associations designing and implementing the CASIMAR project, which is heightened by this multi-organizational design challenge. A viable virtual testing and training platform can alleviate these problems by providing an interactive environment in a virtual space that supports the project throughout its design, by making the rover accessible in a believable simulation of the moon, without requiring a partial or complete prototype or access to LUNA or a ticket to the moon.

Predecessors of this research have applied VR to assist in the designing of systems, training of personnel and allowing to support astronauts in operations. Bensch et al. showcase the use of an VR environment to aid in testing the design for the Argonaut Lunar Lander in a design study. It is concluded that widely accessible VR technology allows a wide range of individuals to contribute to the early design stages, but is still restricted by being limited to only visuals and audio [5].

Costantini et al. explored with JIVE (Joint Investigation into VR for Education) the capabilities of VR for teaching, by delivering a unique visualization tool [7]. In a collaboration between NASA’s JPL, Johnson Space Center, and the Marshall Space Flight Center, the Cold Atom Laboratory on the ISS was upgraded by an astronaut assisted with an AR overlay [22].

Researching existing projects, the application of the VR environment in the aspect of this project could solve some open questions. The VR prototype environment creation will serve as a platform for design review and validation, human-robotic interaction (HRI) testing and implementation of components in a 3D design of the rover.

For the proposed solution, immersive VR will be used. Immersive virtual reality describes a fully enclosed, computer-generated environment that completely surrounds the user. This is usually achieved through a head-mounted display (HDM) and equipment such as hand-held controllers. This type of VR blocks perception of the real world and may achieve a sense of presence within the virtual environment [17]. Some examples of immersive VR devices include the HTC Vive and Meta Quest.

The subsequent sections elaborate on how the VR environment will assist the project throughout its project life cycle, with its respective challenges.

Within the CASIMAR project, special emphasis will be placed on the human-robot interaction (HRI), i.e. the interaction between the astronaut and the rover. HRI will be included as an integral part from early on, governing the design and development of the rover. The VR environment will be one tool to serve this goal, ensuring the inclusion of HRI in the whole development process.

Generally, research has shown VR to be an effective tool for testing HRI in contexts outside of space research. For example, in [29] the same experiment was conducted in reality and VR. They found similarities in all reported metrics, e.g. effectiveness, mental fatigue, perception and impressions of interaction, between both conditions. In another experiment, where reality and VR were compared, the authors concluded VR to be a “great way of simulating robotic scenarios” [9]. Similar results were found by [31] and [30].

Furthermore, through VR, user research can be conducted in the early stages, supporting informed decision making. Prototypes in the form of 3D models can be implemented in the environment to test the interaction with and perception of the rover. As research shows, important HRI measures can be effectively assessed within VR.

In 2006, Steinfeld et al. [24] defined common metrics for HRI which proved their importance since then. For the human side of HRI, they list situational awareness, workload and accuracy of mental models as important indicators. These can be assessed through questionnaires, i.e. the Situation Awareness Global Assessment Technique (SAGAT) for situational awareness or the NASA-Task Load Index (NASA-TLX) [15] for workload. Next to these query-based techniques, VR enables behavior-based assessment of metric through e.g. eye-tracking to assess workload, e.g. [19]. It is important to note that there is evidence that cognitive load, which is important for learning, can be reliably assessed in VR, e.g. [14] and [10].

Furthermore, Steinfeld et al. [24] also defined metrics regarding the rover, namely efficiency and effectiveness. Degrees of these two metrics can be simulated within the VR environment, and their consequences on the human-robot interaction investigated. These results can then be used to improve requirements and set benchmarks for the rover to ensure adequate HRI.

In a literature review, Lei, Su & Cheng [17] summarized eight key advantages of using VR for HRI. Some of these advantages include:

• $\mathrm{■}$

  Immersive experiences: The ability to create immersive experiences, putting users in situations that resemble reality as closely as possible. This is especially relevant for environments like the moon, where simulation environments are limited.

• $\mathrm{■}$

  Realistic Training: These immersive experiences create opportunities for realistic training environments. This can improve training quality, as the context of training is close to reality.

• $\mathrm{■}$

  Remote Interactions: The possibility to enable remote interactions with robots in different locations, opening the opportunity to test teleoperation within the VR environment.

• $\mathrm{■}$

  Customizable and cost-effective: The VR simulation can be tailored precisely to the needs of the project and modifications can be made throughout the project cycle - all while keeping costs low.

• $\mathrm{■}$

  Increased safety: Within VR, there is reduced risk of accidents and injuries. A simulation can be used to teach users how to securely interact with a robot in a dangerous setting, enabling them to make errors and learn from them without facing real consequences.

These advantages will be considered within the project. This concludes that VR emerges as an effective opportunity to test HRI in the context of space exploration.

As part of the BVSR project, the aim is to develop software that benefits all student groups. To achieve this, the diverse domains and requirements involved must be considered. One way we address this challenge is by following the European Cooperation for Space Standardization (ECSS). Here, especially the ECSS-M-ST-10C [26] will be important for hardware development and applying the Concurrent Engineering (CE) approach.

Concurrent Engineering is a work methodology that emphasizes the parallel execution of tasks, also known as simultaneous engineering or integrated product development (IPD) – using an integrated product team approach. This method integrates design engineering, manufacturing, and other disciplines to reduce the time required to bring a new product to the market [32].

The European Space Agency (ESA) applies CE in its Concurrent Design Facility (CDF) as an iterative process where multiple disciplines work simultaneously on a project, making real-time adjustments to specifications and designs. The CDF is a state-of-the-art facility equipped with networked computers, multimedia tools, and software, enabling experts from different fields to collaborate efficiently. This setup ensures high-quality, consistent results in significantly less time [11].

At Technical University Darmstadt, the student group has access to a scaled-down version of ESA’s CDF, located in the Mechanical Engineering building on Lichtwiese Campus. The CEL Facility allows the implementation of the same collaborative project development approach.

The benefits of Concurrent Engineering in space mission design are proven and this methodology will be applied within this project. The ECSS Standard provides an excellent framework for managing an interdisciplinary hardware development process, covering all hardware design phases from 0 to F. The goal is to iterate through these phases, leveraging real-time feedback from the VR environment to enhance the final product’s quality.

Since the ECSS phases follow a traditional waterfall model, they are adapted for software development by integrating a more agile methodology, a more suitable approach for iterative software development. This creates a synergy between ECSS’s structured phases, the agile approaches flexibility, and Concurrent Engineering’s iterative design process.

By combining these methodologies, it is ensured that the project stays on scope and schedule, benefiting all teams involved and contributing to the success of the project. This approach streamlines collaboration, improves design efficiency, and enhances the overall project outcomes.

This combined roadmap helps understand that the VR system isn’t a post-hoc visualization tool, but a concurrent design and validation environment that grows with the rover, aiding every phase – from concept to training and operations.

The application of VR environments in the context of space exploration can take on different forms, which has already been explored in related works: It can be a tool to evaluate early designs in the prototyping and design phase [20], support and facilitate mission training as shown for lunar EVAs, be used for learning the operation and interaction with robotic components and be used to conduct simulations and tests on the software of interacting or inter-actable agents such as a lunar rover.

All of these use cases have been applied to different engineering projects at different stages of design and development, while having intersecting elements. In the context of the CASIMAR project, which is currently in the mission and requirements definition phase, a VR environment is proposed that leverages the above-mentioned use cases into one singular environment. This environment will support the CASIMAR project throughout the different phases up until the rover operations while growing along with it. The approach takes inspiration from the works of Nilsson et al. and their EL3 study. Here, the team examined the design concept of the European Large Logistics Lander (EL3; now the argonaut lander) in VR [20], [21].

This section of the current study defines the scope of the VR environment and the iterative additions and milestones for each stage. The purpose for the respective development phases is described with a particular focus on human-computer interaction (HCI) and human-robotic interaction (HRI) aspects. Subsequently the design phases are being referred to by ECSS-M-ST-10C from phase B to E, which describe phases of preliminary project definition (B), detailed project definition (C), qualification and production (D), as well as utilization (E).

The first proposed application for the VR environment concerns design phases from early prototyping up until later design stages.

This phase is not directly related to the ECSS standardised project phases, as these are mainly concerned with the development and deployment of the product. Rather, it is concurrent with the end of verification and production and, in particular, utilisation (phases D and E). After supporting the design phase of the project, the VR environment is planned to support the training of astronauts in interaction with the rover. Virtual reality bears various advantages that could faciliate the training phase.

The final proposed addition to the VR environment is to facilitate operations of the actual rover during its utilization (phase E). Part of the CASIMAR mission profile is the operations of the rover by a ground support team.

As the VR environment already poses a digital twin, the last piece to allow for this capability is the synchronization of the real rover with its digital twin. This consists of two parts: Updating the digital twin with telemetry and the system states of the analog rover as well as to inform the operator about what the rover senses and what the state of the rover is.

Ideally, they behave the same in the same environment, however, the digital environment can only ever realistically be an approximation and it is to be expected that the analog environment persistently changes due to the rovers actions, but also especially with an accompanying astronaut. To minimize desynchronization between both rovers, frequent updates are required but are also possibly straining the data budget. On the other hand for effective control of the rover, commanding functionality needs to be added to the VR environment as well: Inputs from the VR appliances, either directly or interpreted by the virtual rover, need to be translated into telecommands and sent to the analog rover. This may be the most experimental of all proposed features.