SportsHCI

As the leader of an educational community focused on challenge-based learning for sports, wellbeing and preventive health (Vitality Squad, TU Eindhoven), I have been supervising or leading a large variety of projects related to SportsHCI. Being part of a department of industrial design, our vision revolved around designerly ways to trigger healthy lifestyles with solutions contributing to both physical and mental vitality. Relying on aesthetics of interaction principles and the use of data as a creative material, we design interactive artefacts to support motivation and change. My work, done in collaboration with several Ph.D. candidates, covers several application areas and target audiences, with a main focus on exercising motivation (Daphne Menheere), sports injury prevention (Juan Restrepo Villamizar), office vitality through active ways of working (Ida Damen, Roy van den Heuvel), and the design of (inter)active urban environments (Loes van Renswouw [1, 2, 3]). We see an important role for data in the design process, to gain insight into these complex new behavior patterns in everyday life.

In my talk, I presented several projects to illustrate our main design approaches. Based on the Runners’ journey [4, 5], projects were developed to trigger exercising motivation using qualitative interfaces [6, 7], data physicalization [8], and aesthetics of friction [9, 10]. We approached injury prevention by on-skin interfaces [11] or the concept of interaction-through-negotiation [12]. I concluded my talk with two challenges for SportsHCI: 1/ the necessity to further develop non-normative views on sports and motivation and to address audiences who are not already intrinsically motivated by physical activity. 2/ the responsible and meaningful use of data.

My work in Sports Interaction Technology is shaped around Things that Interact for a Purpose against a background of Theory and Domain Knowledge. Each of these may be the focus of my research in some projects, but may as easily serve as the mere context for other projects. The “things” are typically digital technologies, often with expressive qualities. Holographic virtual reality for telepresence; smart objects with sensors and lights on a sports field; game-like settings in immersive virtual reality; robots and embodied agents; music technology; and many more. When the “thing” is the central concern, research questions involve finding good architectures, algorithms, and models, or working on novel creative, and playful designs. Alternatively, the “thing” may just be something used as a vehicle to explore other questions. The “interaction” in my work involves the sense-think-act cycle that is always embedded in the technology that I develop or use. Research questions at this level are about modifying elements in that cycle (what is sensed, what form do responses take, and how are the two mapped in an unfolding interactive dialog with the technology) and asking how this changes the immediate social and physical behaviour and experience of users. The larger “purpose” of my work tends to involve contributions to play, care, and learning – sports can be said to be all three of those. Research questions that focus on the purpose mostly concern longer-term impact: does the technology indeed change social relations, health, and well-being, learning, etcetera? These questions are typically very intensive to answer and thus less frequently put in the center of my own work. The “theory and domain knowledge” about sports, play, and movement, finally, is present in two ways. On the one hand, it serves as a necessary foundation to inform the other three pillars. On the other hand, sports interaction technology can be used as a lab setup to explore and answer more fundamental questions that contribute to sports and movement theory.

As a former elite gymnast, it paved my way into sports science, and I challenged myself to re-train the backward somersault to show off on my 60-year birthday. Despite being a trained carpenter with dyslexia, I pursued building houses, incorporating the double diamond as a window design of my house. With philosophy and a sports-physiology master’s, I entered interaction design, creating SportsPlanner without knowing the discipline. This journey led to establishing the first Sports Innovation and Entrepreneurship program in sports science education, encompassing technology. Research has circled around developing games for an interactive back-yard trampoline, making a platform for filming and using visual data to strengthen learning in school physical education. As these two projects we have also in the Kids n’ Tweens project developed the iMO-LEARN interactive stool for move-and-learn. In the field of physical activity for special needs we have co-designed the Handi-Wall with the local company PlayAlive inc. We have developed the “Real Man” platform using gamification for social connecting and enhancing behavioral change among blue workers. We have co-designed the feedback elements of the Eye4Talent football coaching tool. Considering health crises and talent development, technology presents ethical dilemmas. We see reality stars being online coaches having no knowledge and compassion and thus coaching towards fertilised eating disorders. It could also be that talent is produced from the many, but too much data may exclude and restrict motivational fellowship, reducing the talent pool. How will AI affect and on which values should we build ethical future HCIsport?

The field of SportsHCI holds great potential to contribute to a more active and sportful society. Ubiquitous computing, wearable sensors, and artificial intelligence allow for on-the-fly, real-time interactions to support motor learning, skill acquisition, performance, and long-term engagement in sports. To live up to these promises, the field of SportsHCI needs to address a number of challenges to be able to tackle the big societal issues that we face today. The first of these challenges is to join forces within the HCI community and beyond to get a multidisciplinary perspective on wicked problems like: physical inactivity, decreasing physical activity, and impoverished physical literacy. Together, we need to rethink sensing techniques, processing approaches, and actuation paradigms to match the capabilities of interactive technologies to the needs of sports and movement.

Improving the data accuracy and performance metrics is one of the driving forces of Sports-HCI. Research shows that the introduction of performance and biomechanical measurements altered the act and practice of sports. However, experience of the data collection technology (e.g., sports trackers) changes the way the athletes experience and immerse themselves in sports. I argue that athlete’s subjective experience of objective measures is richer and more complex than their looking into performance and biomechanical data. For example, athletes can misinterpret data or may feel frustrated when the meaning derived from sports data does not match with their expectations or feeling of their performance. Furthermore, such mismatch and misinterpretation may negatively influence athletes’ perceptions of self, such as self-worth and self-care. Hence, I challenge the sports-HCI with going beyond making data readable, understandable and interpretable, and investigating the practices of when the numbers and the data representation are not aligned with what an athlete feels about their body and performance. In my work, I investigate how sports technology influences athletes’ experience and sensemaking of sports-data, their sporting and bodily experiences. I presented an initial framework that can shed light on investigating athlete’s data sensemaking practices and concluded with the grand challenge of further investigating the “subjective experiences of objective measures”.

SportsHCI provides an additional opportunity to engage in sports. Our SportsHCI projects include virtual rowing, tag, skiing and volleyball. These projects showed how VR and projections might visualise elements of our bodily behaviour and might steer us to behave in different ways. It also made us rethink what sports is and what SportsHCI could be. This familiarised us with the notion of sportification amidst a un(-)ification triangle of playification, gamification, and sportification. Adding or removing sportslike elements might be one starting point of what SportsHCI technologies could add in, SportsHCI and beyond. We will continue to explore where technologies can really be of added value and will provide new types of motivations. I see interactions can be aimed at getting people to actively move to exhaustion and excellence, but also as technology which could unlock feelings of awe and enjoyment of moving physically just a little for entrance level athletes. Furthermore, new and not yet existing mixed reality sports, such as GPT4’s teamsport SpectraBall where rugby, beatsaber and football are combined, not only provide new experiences for athletes but also open up untapped potential for relatedness through the act of jointly (watching) sports.

In this research project I developed a simple instance of breath coach that supports runners with breathing. Specifically we support Locomotor-Respiratory Coupling (LRC), also known as Rhythmic Breathing. Locomotor-Respiratory Coupling (LRC), a breathing technique in which the breath is coupled to steps, can positively impact running experience and economy. Locomotor-Respiratory Coupling (LRC), typically is done in whole-integer ratios, counted in steps per inhalation and exhalation. For example, one could use a 2:2 ratio, implying that the runner performs two steps per inhalation and two steps per exhalation: An Android application picks up the runner’s steps and guides the breath through sound feedback. Based on a proof of concept a full fledged Android application is designed and developed to support longitudinal studies. Follow-up steps will explore feeding the system with breath data to further support the runner at individual level when needed. Additionally we aim to explore explicit interactions during the running activity, exploring interactions that do not interfere with the running experience or the running motion.

Human-computer interaction has potential to impact physical activity through two key ways: Making it easier to learn movement skills and improving the motivation to keep learning and moving. In my work, I have been mostly focusing on motivation, although my group has also done some learning interventions. I mostly build on intrinsic motivation theories such as Self-Determination Theory and the work of Silvia et al. [1] on curiosity. These highlight the importance of satisfying basic psychological needs such as competence, autonomy, social relatedness, and novelty. In my talk, I present examples of a stream of research originating in my own doctoral research, almost 20 years back: Creating “superhero movement experiences” with both physical and digital manipulations that aim to make one feel more competent as a mover. For instance, one might become the protagonist of a kung-fu game where one’s body appears stronger, faster, and more flexible, and where one can execute backflips so that one only does the movement up to a point that is safe, and the game then takes over and guides the virtual body through the rest of the skill. I will also briefly comment on supporting other needs, such as curious exploration of novel movement opportunities in an urban environment, approached through an AI system that recognizes parkour spots and creates spot maps based on Google Street View images.

The project “Digital Motion in Sports, Fitness, and Wellbeing” (DiMo) is motivated by the belief that by digitizing the quality of movement and making emotion visible during movement, we can support people in their movement and lead them to optimal performance and motion experiences. The project aims to give people a better understanding of their physical activity by linking motion and emotion, resulting in an overall new (digital) experience. Within this project, I worked on two streams: 1. exploring respiration as an interaction modality while in motion, and 2. working with visual respiration data feedback while running. Here, my focus lay on supporting breath pacing in treadmill runners while also trying to address impacts on breathing awareness. In the seminar, I am really interested in conversations on how technology can be used to build or increase body awareness to support training, recovery, as well as pleasure and enjoyment during sports. I am also keen to dive into discussions about ethical questions that arise in the field of emerging technologies in sports.

Different skills need to be acquired in sports to perform well. Also, following the correct technique will not only enhance performance but also will avoid unnecessary injuries. Hence, my research has focused on integrating sensing and feedback into our bodies for skill learning and enhanced performance. For example, GymSoles allow our body to feel where our centre of pressure is to keep the correct body posture while lifting weight. The interface is integrated with our body taking foot pressure as input and creating vibrio tactile patterns to internalise our centre of pressure allowing the body to keep the correct posture. Similarly, in CricketCoach, we explored how feedback from SMA wires could be used in a cricket training session to render the hand feedback of a cricket coach, allowing a more naturalistic experience in self-cricket learning. In the future, I want to explore how body integrations can automatically be adjusted according to the contextual changes of the player’s environment.

Attuning to, cultivating, and enhancing body perception is crucial in sports. Advances in sensing, actuating, and wireless technology offer new ways to understand body physiology and movements. Further, wearable technology facilitates integration in challenging sporting contexts. My work in SportsHCI focuses on designing and evaluating sensory technologies to enhance and transform body perception. I presented two aspects of my work. First, Intercorporeal Biofeedback, a type of technology that enhances individual and mutual body perception, and that has been proven to improve understanding, communication, performance, and movement learning in yoga, strength training and circus training. Secondly, I presented Body Perception Transformation technologies, which utilize sensory feedback to create illusions of body changes, like feeling faster, lighter, or more fluid. Evaluations in everyday physical activity contexts and with dancers have shown that these technologies yield empowering, creative and motivating experiences for individuals. I concluded presenting current SportsHCI challenges, and perspectives I wish to discuss.

Interactive computing is reshaping the athlete experience in sports and in outdoor recreation more broadly defined. This talk will discuss these issues in the context of figure skating and day hiking. In figure skating, the primary issues revolve around injecting data into the coach-athlete relationship where the athlete is a child and the parent is also involved. In the context of day hiking we present self-reported motivations for headphone use and non-use in the United States. These motivations involve safety, mediating social interaction and creating specific experiences. Before the seminar, I uploaded an essay and a research paper for pre-seminar reading. The essay frames cars in nature and foreshadows modern discussions of smartphones and nature. The research paper is an inductive study of sources of enjoyment in figure skating for elite skaters published in 1989. It focuses on enjoyment–an important concept in sports at all levels. My tentative pre-seminar grand challenge is: redefine SportsHCI to enhance the wellness benefits of lifelong participation in sports.

Data visualization is an established field of HCI. One of the main goals is to help users to understand and analyze data through visualization techniques and interaction. Visualizations have the property of being processed in parallel by the human perceptive system. Good visualizations should allow the observer to quickly understand the data and make decisions. Classic approaches, such as Shneiderman’s Mantra, cannot be implemented anymore due to the massive amount of data. AI might help, but in many cases, the role of the user is not clear. Sports data are multivariate and are produced massively, making them challenging to compute and visualize. We aim to investigate the feasibility and utility of data visualization in sports, analyzing first soccer data visualization and interactive interfaces for video analysis.

Water’s pleasant nature and associated health benefits have captivated the interest of HCI researchers. Prior WaterHCI work mainly focused on advancing instrumental applications, such as improving swimming performance, and less on designing systems that support interacting with technology in water in more playful contexts. In this regard, I explore the somaesthetic design of playful interactive water experiences, specifically, floating, surfing and diving, aiming to enrich the experiential aspect of the water activity through technology. Employing somaesthetic design, I aim to develop different playful prototypes to understand the integration of the soma (human mind and body), the technology and the water, towards a synergy that enriches the water experience. To date, I have designed an extended reality system for a flotation tank experience, which suggests interesting opportunities for the technological enrichment of water experiences. Ultimately, I hope that our WaterHCI work supports people to be playful and benefit from the many advantages of being in the water.

Sitting in front of computers has become a major part of our workaday routines, challenging us in maintaining active and healthy lifestyles. This challenge becomes even more salient and more relatable to many of us after COVID-19. Is it possible to design interface and interaction so that the health behaviour intervention is integrated into the target users’ established lifestyles and daily routines? We argue that by leveraging existing social mechanisms, a SportsHCI design can enable health interventions in a natural and effective way. One simple example could be facilitating a social context where users feel being invited for micro health gains. To better illustrate this design mechanism, we use several design instances (Step-by-Step, Anti-Sedentary Robot, Co-Drink, Co-Coffee) to exemplify our proposal. We would also like to learn your advice to inspire us in figuring out the design, evaluation and scaling-up methods of this persuasive strategy.

Cycling is a great way to remain physically active, maintain cardiovascular health, and improve physical shape. While cycling outdoors is de facto a standard that includes cycling tours, training, or commuting, cycling indoors has become more popular over the last decade. In my research, I explore technological improvements for indoor and outdoor cycling. I will present how we employed multimodal and extended reality user interfaces and new tandem-based cycling simulation methods to facilitate cycling safety and realism. The results of these works demonstrate how technology can improve the cycling experience and reduce motion sickness indoors and facilitate safety outdoors.

SportsHCI can benefit from the integration of MR (Mixed Reality) interfaces with AI (Artificial Intelligence) algorithms, particularly CV (computer vision) and ML (Machine Learning algorithms). We discuss the blend of such technologies in use cases related to training high performing athletes in table tennis, albeit ideas and principles can be extended to other racket sports and, in fact, to a variety of other sports, as well. We are also very interested in the interplay of such technologies with humans in the loop (athletes and coaches), specifically wrt their acceptance of such technologies, their perception of user experience and the opportunities for such technologies to be assisting athletes and para athletes.

In the last years, sports technology has become ubiquitous and there has been a large body of work in HCI and Ubicomp as well as commercial products including apps, wearables and smart sports environments. SportsHCI has the potential to analyze complex human movements and provide guidance when learning a new motor skill. In our research we investigate intelligent assistants to support skill acquisition and improve performance in athletes. To achieve this goal we investigate different feedback techniques, both on the athlete’s body and in their training environment. This includes smart environments using intelligent sports equipment and projections as well as wearables and virtual reality headsets, which enable us to provide in-situ feedback to enhance motor learning. We are particularly interested in implicit, in-situ and real-time feedback for example by using EMS but also playful approaches in mixed realities.

Much of the historic focus of physical activity-related HCI has either been related to: those starting out their journey; or, to performance, particularly in (semi-) professional athletes. Over time this has broadened to include a range of sports, fitness levels, and technologies. However, more work remains to be done to establish a more inclusive SportsHCI research agenda that caters for all athletes, as well as promotes enjoyment and continued engagement regardless of gender, race, background, ability, body size, or time-pressures. While there is a broader movement in HCI around designing inclusive and accessible technologies, this should be more explicit in SportsHCI. Here I will bring examples from the cycling communities to highlight this need. I argue that a Feminist HCI framework can help emphasise inclusivity and diversity from an intersectional standpoint, allowing SportsHCI to better address individual needs and preferences, challenge stereotypes, and create a more welcoming environment for all.

SportsHCI benefits from the integration of the human body and interactive technology. We demonstrate this through a series of research design works around integrated cycling experiences, integrated entertainment experiences, and integrated arts experiences. The results of these works suggest interesting ways forward for SportsHCI research, in particular how the design of integrated SportsHCI can highlight experiential aspects, facilitating playful exertion experiences. Ultimately, with our work, we want to enhance our knowledge around the design of integrated SportsHCI experiences to help people understand who they are, who they want to become, and how to get there.

Data – qualitative or quantitative – has become central for Sports HCI. My personal take on data is from the perspective of digital twins. The goal of digital twins is to have a full representation of an artefact, process, machine or even a digital replica of a human. The identified key challenge from my perspective is that time, evolvement over time and especially how to enable people with the future. Digital twins have the ability to simulate and predict in real time – making it imperative for people to interact with future states and alternative futures. We know that humans are not very good when it comes to planning, predicting the future and knowing what to come is sometimes disturbing for people and how they react. It will be important to consider the ethics when designing with digital twins, to ensure that sports systems will be a positive contributor to humans health and their planning of sport activities and does not become a burden or a scary prediction instrument that is foreshading possible diseases or injuries. The key topic emerging from this challenge is to understand overall on how to interact with data and representations that relate to the future: “future interactions”.

After having read about the Reverie lens from the paper, the group summarized what they understood about the concept and the three subfacets (Void, Solitude, and Mini-Holidays). We highlighted in the text a few elements that could act as design triggers to create reverie (e.g., removing stimulation in the environment or adding extra stimulation). Then, each participant shared their own experiences of reverie, in a sport or non-sport context. Void and solitude were often hard to distinguish:

• $\mathrm{■}$

  Walking in the forest, foraging = safe solitude

• $\mathrm{■}$

  Cycling to work with a toddler in a cargo bike = co-experience of solitude (alone together)

• $\mathrm{■}$

  Solo hot air balloon flight = feeling insignificant in the greater scheme of things

• $\mathrm{■}$

  Rhythm of breathing during a run

• $\mathrm{■}$

  Slackline, being in one’s bubble despite the audience (note: for some, it is rather the opposite, where the audience is intimidating)

Personal experiences resonating with the mini-holidays sub-facet were:

• $\mathrm{■}$

  Using swings on children’s playground as an adult.

• $\mathrm{■}$

  Listening to an audiobook while using an indoor bike trainer

• $\mathrm{■}$

  Knitting (automatized gestures), wood-turning, pottery

The group tried to identify higher-level concepts within these examples. The ideas of contemplation, patterns/rhythm, craft/material experience, and purposeless activity were emphasized. However, here again, interindividual differences in the appreciation of craft (fear of non-mastery) or purposeless activities emerged.
Besides a general audience, possible target groups of reverie technology were discussed as a means to reach a more concrete level in ideation:

• $\mathrm{■}$

  Trauma survivors, where vulnerability and fears of specific situations are a challenge

• $\mathrm{■}$

  Eating disorders, with opposing needs where refraining from compulsive physical activity is a goal

• $\mathrm{■}$

  Aging population, with the opportunity to reframe solitude as positive

The group identified some opportunities linked to reverie Tech, e.g., overcoming obstacles in exercising related to the security to embrace an activity (yoga class), the body awareness or the threshold to get started.

First, the group spent time reading the description of the pleasure lens from the paper. Then, we discussed our understanding about the lenses and how we conceive the definition of pleasure in different ways than the ones provided by the paper. Next, we provide our first person experiences about the pleasure we encounter from physical activity, such as intense exertion through sports like cycling and crossfit, very similar to the discomfort subcategory in the paper. On the contrary, we also discussed the pleasure we get from less intense exertion but more expressive physical activity, such as dance, which is not described in the paper. We then discussed how this last one is less explored in our field and then we brainstormed in which scenarios we encounter pleasure from dance and music. Finally, we came out with the case of a work office environment in which workers are seated all day and are under great amounts of stress. Most of the time these workers don’t have the chance to have active breaks, sometimes because they don’t have access to facilities that allowed them to have a physical active break, or sometimes they don’t have the time. Therefore it would be useful to provide and active break through dance, allowing them to get rid of the stress while performing bodily movements following the music. We also imagined a system, for example VR, that allows the worker to switch off the external work environmental and encourage them to move however they want avoiding feeling judged by others.

First, we spent time reading 2.5 pages about the beauty lens from the paper. Second, we freely associated and reflected on the main aspects of beauty such as rhythm, expressed our opinions about them, and proposed possible extensions, such as a timely unfolding of a sports performance and the beauty of time in sports. Then, we did solitary ideation: keeping the lens and those extra associations in mind, what ideas come up to design for beauty? We spent five minutes writing down our ideas about how technology can possibly facilitate beauty in sport activities and ended up with six ideas. The descriptions of the lens and sub-facets and the concrete design suggestions in the paper provided an entry point to start thinking about design ideas once we identified a goal besides the “beauty”. In addition, the associative concepts that we came up with, made the design possibilities feel more nuanced and detailed. Once we had a number of concepts, we presented them to each other and shortly discussed and extended them. Afterward, we briefly discussed which of these ideas can be clearly presented via a pantomime or a bodily representation. In the end, we decided to pick an idea that would fit the best for bodily representation and would be easy for an audience to understand.

The group first reviewed the section on “pain” as a lens in the Sports HCI Lenses paper. We discussed the meaning of the word pain as constructed in that paper as well as in another paper Mike is working on. We focused on the concept that pain is a voluntary experience in sports.

• $\mathrm{■}$

  We reflected on our personal experiences in sport which included table tennis and running.

• $\mathrm{■}$

  We reflected on our experiences with parents and children in sport. This included some experiences with our own children and some experiences we’d seen involving other children.

• $\mathrm{■}$

  We generated several ideas about how to generate pain in several contexts.

We discussed what the lens of humility was about and went through the three types, mostly directly in relation to some of the examples given in the paper. We reiterated that the big mountain in front of you brings a sense of humility, talked about the humility from competing against someone who is expert or better, only later more as a side note for completion we briefly mentioned but not really discussed trying again. Instead, we did discuss trash talk and bluffing in relation to humility. There is a difference of knowing your skills/cards (e.g. in poker you might exactly know what you have in the last round), versus not yet knowing your current capabilities of that day before going all out. The trash talks was also a picture in one of the slides shown after it was mentioned in the discussion, we related this also to the current sports event of the Tour de France. For instance, the coach sort of coaxing by saying “you can do it!” to Pogačar who was already completely beat. We later mentioned exemplars from media/every day use and own experiences. This led to the focus on experiencing actual parts of being a pro, which interestingly enough is also in the paper but more as a almost comparison rather than a more complete feeling of humility. Through the generation of the design space we saw a variation in first person experiences of pro sports which might make one humbled. After the discussion we wondered how other theoretical contributions or theory would play a role, or could help to shape the design space. Here, Newell’s theory of constraints, phenomenology, and remediation/ post-phenemonology were mentioned but not discussed in too much depth.

Reflecting on the process, we see a recall and sharing of exemplar 1st/2nd/3rd person experiences, some focusing on the somaesthetic qualities (e.g., going a ridiculous speed in real or virtual life). There is a back and forth of ideas that would be interesting to experience humility, ideas that inspire a direct transformation of that idea into something related (e.g., from wavelight to seeing someone descend in the Tour de France), then talking these and associated ideas through we see other examples that already exist (e.g. cycling at higher speed due to wrong weight setting, experiencing downhill footage in connection to simulator) which we can build on but require and instigate a transformation of the last proposed own idea into something new (e.g., camera-based system currently not keeping up if you don’t break downhill, rather than seeing the pace align the pace changing you with your “known” speed). This thus naturally transitions between inspiration, associated idea generation, to converging or actually taking steps sideways, see Figure below.