Perception of Space in Virtual and Augmented Reality (Invited Talk)

Virtual and Augmented Reality (VR and AR) methods provide both opportunities and challenges for research and applications involving spatial cognition. The opportunities result from the ability to immerse a user in a realistic environment in which they can interact, while at the same time having the ability to control and manipulate environmental and body-based cues in ways that are difficult or impossible to do in the real world. The challenge comes from the notion that virtual environments will be most useful if they achieve high perceptual fidelity - that observers will perceive and act in the mediated environment as they would in the real world. Consider two approaches to the use of VR/AR for in cognitive science. The first is to serve applications. For this, I argue in many cases we need to achieve and measure perceptual fidelity. Specifically, perceiving sizes and distances similarly to the real world may be critical for applications in design or training where the accuracy in scale matters. The second approach is to use VR/AR to manipulate environment-body interactions in ways that test perception-action mechanisms. Our lab and collaborators take both of these approaches, as they often mutually inform each other. I will present two examples of this dual approach to the use of VR that take advantage of the body-based feedback available in immersive virtual environments, in adults and children. The study of children’s spatial cognition is an important new direction in VR research, now feasible with the emergence of head-mounted-display technologies that fit those with smaller heads. Immersive VR has great potential for education, specifically in advancing complex spatial thinking, but a foundational understanding of children’s perception and action must first be established. This is particularly important because children’s rapidly changing bodies likely lead to differences compared to adults in how they represent and use their bodies for perception, action, and spatial learning. Even with rapidly advancing VR technologies, one continuing challenge is how to accurately update one’s spatial position in a large virtual environment when real walking is constrained by limited physical space or tracking capabilities. In my first example, I will present research that compares different modes of locomotion that vary the extent of visual or body-based information for self-motion, and tests the ability of users to keep track of their positions during self-movement. Differences in adults and children suggest reliance on different cues for spatial updating. Research in space perception in VR suggests that viewers underestimate egocentric distances in VR as compared to the real world, although the new commodity-level head-mounted-displays have somewhat reduced this effect. In a second example, I will present research that examines the role of bodies in scaling the affordances of environmental spaces. We use judgments of action capabilities both to evaluate the perceptual fidelity of virtual environments and to test the role of visual body representations on these judgments. Finally, I will present extensions of the use of affordances to evaluate perceptual fidelity in VR to new possibilities with AR, in which virtual objects are embedded in the real world. This work demonstrates that augmented reality environments can be acted upon as the real world, but some differences exist that may be due to current technology limitations.