Exploring the Symbiotic Collaboration Paradigm in Virtual Reality and Its Potential Applications to Human Spaceflight

EXtended Reality XR technologies find a well suited use case for design review and training purposes. This tendency is even more salient in the context of VR experiences, mainly due to its advanced sensorimotor contingency features. The latter facilitates “an inclusive, extensive, surrounding, and vivid illusion of virtual environment” to the users [26]. As presented in the opening of this document, VR is a complex medium on its own. It induces perceptual illusions among users that are referred to as qualia (singular: quale), defined as “a subjective and internal feeling elicited by sense perceptions” [25].

For instance, being immersed through VR technologies elicits among users a subjective experience of presence, meaning the sense of being there in the virtual space [15, 29]. Slater introduced two orthogonal components to presence, namely Place Illusion (PI) and Plausibility Illusion (PsI), that are sine qua non conditions for the users to experience events happening to them in the virtual place as if they were real [27]. This authenticity of the simulation is what drives new training practices in using immersive technologies.

In addition, the users may interact with the digitally recreated space through a virtual representation commonly referred to as an avatar. Experiencing an avatar as being one’s own body is conceptualized as the Sense of Embodiment (SoE) in VR [17]. Kilteni et al. proposed a framework of the SoE made of three dimensions: Self-Location, denoting “one’s spatial experience of being inside a body” (distinct from being in the virtual world) ([17, 9]); Agency, signifying the sensation of having complete motor control over the virtual body, which also encompasses the intention of actions ([7, 16]); Body Ownership, referring to the self-attribution of the virtual body [28, 19]. Developing a SoE may appear critical in some use cases. For instance, Gorisse et al. [13] showed that users with increased ownership towards their avatar where more likely to protect their body integrity.

Therefore, presence and the SoE appear as two major qualia, on top of every immersive experience, that should be carefully considered when designing for practical use cases in the context of human spaceflights and beyond.

As mentioned above, VR could be a perfect ground for design reviews or training activities. But such use cases often involve a supervision setting, in which users are rarely to be left alone with the simulation. Indeed, the aforementioned use cases would imply that one or more operator supervises the immersive session, usually using non-immersive setups for practicality. A concrete example of such practice can be observed when using standalone VR applications that can be monitored thanks to external device screen mirroring features, but that only gives access to the VR user’s viewpoint. Some tailored applications go further by providing the supervisor with interaction means, and even a virtual representation. However, collaborating with non-immersed users has been reported to create occlusion problems and peripersonal space invasion in collaborative settings [22]. More specifically, a literature review of collaborative virtual environments proposed by Derouech et al. [10] may support the claim that avatar representation tend to be understudied in this context, even if the number of papers considered for the review was limited based on the inclusion criteria ($n=44$).

Still, a concrete benefit that arises when making the remote supervisor part of the simulation is the potential emergence of a sense of social presence [6, 30, 25], encompassing the sense of “being there with another” [6]. Leveraging this quale may benefit positive communication outcomes, like persuasion among others [21]. However, when the supervision is operated through a cross-platform setup, the inherent asymmetry between devices and associated virtual representations may limit non-verbal communication. This latter point is exemplified by Olin et al. who designed the mobile user’s avatar to signify their inability to use hand tracking and object manipulation by making its hands static [22].

All things considered, there may be alternatives to these methods to leverage collaboration and non-verbal communication in a supervision context, by taking advantage of devices affordances.

At the frontier of the virtual and real worlds, Piumsomboon et al. [23] explored the question of the supervisor representation through an innovative Mixed Reality (MR) collaboration. Their application allowed for a local trainee in Augmented Reality (AR) and a distant supervisor in VR to collaborate in real-time. They notably reported that having the remote supervisor embodied in a 360° camera mounted on the VR user’s shoulder is well appreciated by the latter. This indicates promising benefits for multiscale collaboration with the remote user in close vicinity of the local user. The concept of virtual co-embodiment is crystallizing such an approach. It was recently introduced by Fribourg et al. [12] as “a situation that enables a user and another entity (e.g., another user, robot, or autonomous agent) to be embodied in the same virtual avatar”. In most virtual co-embodiment settings, two VR users share control over the same avatar. The use of such paradigm shows promising perspectives for motor skills training [18].

Yet, lot of work on co-embodiment focuses on the impact of control distribution over the virtual body [12, 32, 18, 14]. There is therefore room to investigate the sole fact another is co-inhabiting once own avatar. This may especially appear critical as, to the best of our knowledge, the impacts of co-embodiment on the SoE or social presence are yet unknown. Studying such impacts would require first to make users aware someone is inhabiting their body. Venkatraj et al. [32] provided initial insights about mutual awareness cues in their co-embodiment experiment as they operationalized the perceptual cross paradigm through haptic vibrations. It consisted in triggering a vibration when both users hand locations overlapped in space. Even if this attempt yielded a negative effect with respect to perceived agency, the result must be considered carefully as the users were not aware of the vibration feedback signification. Moreover, haptics may be beneficial to co-embodiment on the social presence aspect, like when passing objects to each other, or when the haptic communication supplements the verbal one [21].

Finally, the literature would benefit from a first attempt that could lead to operationalize co-embodiment for cross-platform applications. Indeed, as presented in Subsection 2.2, providing the non-immersed operator with an independent virtual representation may cause collaboration issues. Then, making the non-immersed user inhabit the VR user’s avatar may help bridge the gap between their device interaction possibilities. Still, the consequences on qualia from the VR user perspectives are still to be explored.

In an effort to investigate an innovative way to collaborate between a VR user and non-immersed supervisor, we propose a new collaboration paradigm, namely the symbiotic collaboration paradigm. It would consist in providing the illusion of a spatially co-located non-immersed supervisor, further referred to as symbiont, to the VR user, further referred to as the host. This paradigm is inspired by the co-embodiment one in ways that are explained in the remaining of this section. However, for the first iteration of symbiotic collaboration, we convoke a significant difference between both concepts by preventing shared control between the host and the symbiont. Initial design considerations are as follows:

• $\mathrm{■}$

  The host should have full control over the body: as a consequence, the symbiont would not have agency over parts of the host’s virtual body. However, from the host perspective, she/he may perceive not being in full control of the avatar due to the co-located symbiont in some body parts. On that point, we claim the symbiotic collaboration paradigm requires particular attention.

• $\mathrm{■}$

  The host should always be aware of the symbiont position: the main characteristic and cornerstone of the concept we propose is to solely signify the presence of the symbiont user to the host, audio visually and through haptics. By doing so, the symbiont would be provided with unique viewpoints of the host perspective, which leads to the next requirement.

• $\mathrm{■}$

  The symbiont should be able to freely navigate between body parts: the available migration location should also be communicated to the host to leverage mutual understanding. One could for instance think of a shoulder mounted viewpoint on the host’s shoulder that would give a co-located 360° panorama [23], or a viewpoint in one hand to give access to close-ups. At any moment, the host would be aware of the body part inhabited by the symbiont.

• $\mathrm{■}$

  The host and symbiont should be able to interact in a diegetic way: finally, providing the symbiont with specific viewpoints, and signifying them to the host with adequate cues, would enable innovative interactions. For examples, one could equip the avatar shoulder with a representation of the 360° camera and embedded speakers that would spatialize the symbiont audio source. Additionally, figuring a screen and speakers in one of the avatar’s hand palm would enable live discussion between both collaborators.

Following the aforementionned requirement, we illustrated on Figure 1 a concept of what a symbiotic avatar may look like in the context of the VR experiment we tease on Subsection 3.3. The concept of symbiotic collaboration is hypothetically well positioned to foster collaboration between the host and the symbiont. Even so, it is still conjectural and initial prototypes for such features should contribute to the validation of its advantages and exploration of its limits.

Prior to any evaluation of this new paradigm, it may benefit from the careful design of a device that could help signifying the presence of the symbiont user to the host, namely a “position-aware feedback mechanism” [32]. We designed a haptic device as a candidate to fulfill this goal. The active haptic prop we designed shall be attached to the arm with three key points: on the shoulder, on the back of the hand, and lastly close to the elbow. Each of the aforementioned positions is associated with a cluster of three vibration motors (see Figure 2). Then, each cluster is wrapped between two layers of textile that are sewed together and then to a shoulder patch, a elbow strap and a fingerless glove. An Arduino board provides independent controls for the different clusters. Finally, on top of the active haptic feedback capabilities, the physical hardware may provide additional passive haptic sensations, more specifically the sensation of wearing the gear on the arm for instance.

Building up on our hardware assessment, we plan to conduct a follow-up user study in order to compare the symbiotic collaboration paradigm to a regular cross-platform supervision situation, further referred to as the control condition. By doing so, we aim at investigating whether a symbiotic paradigm between a VR user and non-immersed one foster collaboration and the social presence. In addition, we aim at characterizing the impact of such symbiotic relationship on the experience of the VR user. The collaboration methods will be compared according to two conditions: an omniscient presence of the non-immersed supervisor, and a symbiotic collaboration situation in which the non-immersed user’s presence would be suggested to the VR user through audio-visual cues, and through our haptic device. A concept of the symbiotic paradigm implementation is illustrated on Figure 1.

A virtual scenario is being designed in which the experimenter, acting as the supervisor, will guide participants immersed in the VR simulation in completing complex assembly tasks. At the end of each condition, the VR user will have to answer questionnaires assessing their physical presence, sense of embodiment, and social presence as well as items translating how they identify with the avatar in the different configurations. Semi-structured interviews will ultimately be conducted to collect insights about participants experience of the collaboration setup.