Inclusive Data Visualization

In this talk, I will discuss the various kinds of deafblindness and the ways in which people who are deafblind communicate. I will also discuss two barriers deafblind people are currently facing. The first is the independent use of technology such as computers. The second is gaining information about the objects in their immediate environment and navigating through this space. Finally, I will discuss how tactile maps drawn on the body are used to provide spatial information.

Accessible visualization for physical disability is an under-researched area that is often invisible in discussions of making data visualizations accessible. Physical disabilities include fine and gross motor movement and can be congenital, acquired, progressive or temporary so methods for accessibility may need to be adapted dynamically. There are three roles that people can take in the visualization process: passive viewer, active user and creator. In the role of passive viewer people with physical disabilities face minimal barriers to access but as soon as interactions are required to access or create content the barriers can be significant. The modalities used to design and use data visualization can significantly impact the ability of people with physical disabilities experience with visualization but research has not been conducted that establishes best practice for this group.

Stroke, an injury to the brain from disrupted blood flow, often leads to lasting impairments such as speech difficulties (aphasia), one-sided weakness (hemiparesis), and cognitive issues. These changes can complicate daily tasks. A large body of work exists for stroke rehabilitation, designed to enhance stroke survivors’ functional recovery. Of particular interest to the visualization community is mobile self-tracking interventions that show personal data to motivate people to engage in rehabilitation. Wearable sensors capture limb performance, providing metrics like use intensity, active arm use duration, and use ratio. Displaying such data in conjunction with personalized goals may encourage patients to utilize the affected limb in their everyday living. I propose multimodal feedback that transcends descriptive data, integrating self-reflective questions, suggestions, and motivational messages, presented via visual and audio narratives. This multimodal approach could enhance an understanding of the data and provide therapeutic support for stroke survivors.

In this personal narrative, I offer a unique perspective on life beyond the visual realm, delving into the nature of dreams, the intricacies of mental math, and the concept of “inclusive data visualization” through my lived experience as a blind individual. I invite you to explore the uncharted landscapes of my dreams, where the absence of sight gives rise to a distinct, immersive experience that challenges typical perceptions. I further share my unique approach to mental arithmetic, demonstrating the adaptive, versatile nature of human cognition in the absence of visual cues. From my vantage point, I critically assess the prevailing concept of “inclusive data visualization,” questioning its true inclusivity for the visually impaired community. Join me in this journey, as we rethink the boundaries of perception and inclusivity in our predominantly visual world.

In 2023, the United States Centers for Disease Control and Prevention, identified cognitive disability as the most prevalent disability across the nation, touching 12.8% of the population surpassing mobility disabilities (12.1%). As datafication of our world proliferates, understanding equity in knowledge translation and access to data visualizations becomes increasingly important. In 2018, the State of the States in Intellectual and Developmental Disabilities Ongoing Longitudinal Data Project of National Significance, partnered with the VisuaLab to validate anecdotal evidence and understand further cognitive accessibility of data visualizations. This research, the first of its kind, established guidelines for making visualizations more meaningful to users with cognitive disabilities. The presentation provided an overview of the partnership, community demands for equity, user-design approaches, and future research topics.

JavaScript visualization libraries are widely used to create online data visualizations but provide limited access to their information for screen-reader users. Building on prior findings about the experiences of screen-reader users with online data visualizations, in this demonstration, we present VoxLens, an open-source JavaScript plug-in that – with a single line of code – improves the accessibility of online data visualizations for screen-reader users using a multimodal approach. Specifically, VoxLens enables screen-reader users to obtain a holistic summary of presented information, play sonified versions of the data, and interact with visualizations in a “drill-down” manner using voice-based information querying.

Olli, an open source library that converts visualizations into a keyboard-navigable structure accessible to screen readers. Using an extensible adapter design pattern, Olli is agnostic to the specific toolkit used to author the visualization. Olli renders a chart as an accessible tree view following the HTML Accessible Rich Internet Applications (ARIA) standard. The fields participating in the visualization serve as branches of the tree, and levels of the tree correspond to different granularities of data (e.g., major axis regions, minor axis regions, individual data values). Users can navigate up and down the tree using the up/down arrow keys, or move between sibling nodes using the left/right arrow keys. Users can also jump to specific positions in the tree via a series of drop down menus, or press the “T” key to invoke a data table view for more traditional row-by-row, column-by-column navigation.

We demonstrate Chart Reader, an accessibility engine that renders web visualizations optimized for screen reader access. By designing and developing Chart Reader during a five-month iterative co-design study with 10 blind or low vision people, we aim to improve accessible visualization experiences. Our approach, realized through three sequentially designed and developed prototypes, allows users to interrogate visualizations using keyboard interactions, resulting in multimodal audio (announcements and sonification) of the chart. The web-based accessibility engine generates bar charts, stacked bar charts, and single-/multi- series line charts.

Tactile media can be produced in various ways including embossers for braille and dynamic tactile displays. We demonstrate embossed tactile renderings of SVGPlot, a tool we developed and is itself accessible. In our evaluations bar graphs and scatter plots are identified as being suitable for exploration by blind people. Embossed tactile graphics can be turned into multimodal systems by a low cost pen with audio feedback (TipToi). We demonstrate a new version of an affordable Hyberbraille dynamic tactile display (portable, lowered height and reduced weight). As an extension for OpenStreetMap we show as a result of project AccessibleMaps maps of indoor buildings that are rendered both tactile aiming at blind people as well as for people with low vision through high contrast colors. The dynamic display is touch sensitive and also provides audio feedback for room names and barriers we identified in user surveys.

Visualization of information is a way to present data in a compact and clear way. People with blindness have access to this visual information only if it is provided by alternative forms of presentation such as audio-tactile. The TPad is a standard hardware, i.e., an iPad pro integrated in a frame made with a laser cutter. By using the frame a tactile graphic can be fixed. An app enables access to the additional digital information stored on the iPad via the audio-tactile system. A study with users with blindness showed that the system is useful to explore audio-tactile building plans and allows an intuitive access to this information.

Tactile graphics, also known as raised line drawings, are best practice for the representation of 2D graphics to convey spatial relationships for people who are blind or have low vision. Tactile graphics can be created using a variety of methods, each with its own advantages and disadvantages. These include pressed paper, collage and other handcrafting, thermoform, swell or microcapsule paper, refreshable tactile displays and 3D printing on paper. Graphics must be simplified and redesigned for tactile reading, however little research has been conducted on the design of the many different data visualisations and their presentation using the various tactile graphic methods.

Visualization amplifies cognition and helps a viewer see the trends, patterns, and outliers in data. However, conventional visualization tools and guidelines do not actively consider the unique needs and abilities of people with Intellectual and Developmental Disabilities (IDD), leaving them excluded from data-driven activities and vulnerable to ethical issues in everyday life. This work explores the challenges and opportunities of cognitively accessible visualization. Through mixed-method approaches and close collaboration with people with IDD, our group ran experiments and developed guidelines to improve current visualizations. We interviewed people with IDD and gained initial understandings of their daily data experiences, and we are currently in the process of running a participatory design workshop to create accessible visualizations for and with this population. We hope to further expand our knowledge of cognitively accessible visualization, translating what we have learned into a graphical user interface that supports people with IDD with better data analytics, and finally make this population more visible in the inclusive data visualization space.

I showed some knitted Selbu mittens from Norway. Noeska Smit used these to inspire her Data Knitualization, in which she uses wool (material) and knitting needles (technology) to encode information in physical form. She knits abstract representations – line graphs, and woollen models of real structures (body organs). I did so to differentiate between material, technology and data artefact, which we might consider to be a technology enabled manipulation of material that encodes data. This process is subjective, and so I like the notion of exposing this with the explicit use of the term “Design.” It’s a human process that involves intent. I forget who described “Design” as imagining a better World and doing something about it. But I like that perspective. I think it’s what we try to do. So I wonder whether knitualization is PHYSICAL Data Design? Or perhaps we think about the material and consider this to be WOOL Data Design? So is visualisation VISUAL Data Design, and can we then think about alternative materials, modes, approaches and characterise these as: TEXTUAL, HAPTIC, EDIBLE, OLFACTORY, AUDIO, etc. Data Design? All of these can be interactive, collaborative, and combined in artefacts and analytical environments.

It feels to me as though this perspective has some advantages:

• $\mathrm{■}$

  it exposes the SUBJECTIVITY of the encoding and the artefact

• $\mathrm{■}$

  it reduces the cultural DOMINANCE of visualization among other forms of data depiction & representation

• $\mathrm{■}$

  it encourages MULTIMODAL designs and alternative encodings – why not do PHYSICAL WOOL Data Design or EDIBLE AUDIO Data Design?

These seem to be good things for reliable (consistent?) interpretation and accessible data. Many modes, many representations, many senses, many perspectives. But I look forward to hearing about disadvantages. Subsequently, I wonder whether Visual Data Design is actually a process in which we use light, and technologies that manipulate it to help us engage in Light Data Design to Data Design with Light. Maybe I have discovered that I am a Light Designer, and I use materials and technology that interact with light to depict data that represent aspects of the World. I hadn’t thought of that before. Useful?

I also wonder whether Inclusive Data Visualization is achievable or even what we want to do. What we may really want to do is provide inclusive (diverse, comparable) Access to Data, and perhaps what this is really about is Access to Decision Making Processes and Influence, Empowerment.

I presented a study and showed two demos in the context of visually impaired Indian users. The first was a study that aimed to improve the accessibility of bar graphs. In this study we compared the speeds and accuracy of four techniques of auditory bar graphs of two lengths, namely Parallel-Tone, Parallel-Speech, Serial-Tone and Serial-Speech. The study included both sighted and visually impaired users. I also showed another demo of a 3D printable, modular tool that visually impaired people can use to both create and consume line charts. Lastly, I showed a demo of an accessible interaction technique that enables visually impaired users to enter numerical passwords without the need for using headphones. The technique was found to be shoulder-surfing-proof and can allow visually impaired users to confidently enter passwords in a public place. I skipped a demo on an accessible text input mechanism in Indian languages because that was a bit off-topic for this seminar, but it is available in the slides.

Billions of people in the world watch sport media broadcasts to follow their favourite sport. They enjoy the actions captured by cameras and microphones on and around the fields. They socialise with other fans at home, at a local club, in a stadium and on social media. However, if you are blind or have low-vision (BLV), your overall experience is limited. While TV is providing the state of the art experience for sighted people, BLV need to tune in to radio broadcasts. Although used widely by BLV, radio can not provide all the actions in real time, in particular the movement of the objects such as the ball, puck, and players. Action Audio is a world-first system designed for the BLV to watch games in a broadcasting environment that is augmented with 3D sound. It provides alternative modalities to allow BLV access all the actions on sports broadcasting as well as live events in sport venues.

We conducted an exploratory study with four blind people about the possibility of representing depth in images through variations in pitch and loudness of a tone. The blind participants were able to explore an image on the touchscreen of an iPad. Three sound conditions to represent depth were investigated: pitch variation; loudness variation; a fusion of pitch and loudness. All four blind participants strongly preferred the fusion option. As this was an initial exploratory study, more objective measures such as time and errors were not taken. However, these results are very encouraging and we will continue with this line of research.

Access to visual information is compromised for people who are blind or have low vision. This impacts not only information access but general engagement with day-to-day activities that most take for granted. 3D printing has the potential to provide access to content that can be difficult with traditional tactile graphics. This demonstration provides an overview of work undertaken exploring how 3D printing can be used to convey traditionally visual content in the contexts of education, orientation and mobility, and cultural institutions. It hopes to inspire data visualisation designers to think about how this technology can be useful in creating accessible data visualisations.

This talk presents two project, led by Sandra Bae in collaboration with Ellen Do, Michael Rivera, and Danielle Szafir, that offer potentially interesting physical platforms for accessible data representation. The first, the Data-Is-Yours toolkit, is a toolkit made from everyday materials (paper, mirrors, and cardboard) coupled with a cell phone to create basic interactive hybrid physical-digital visualizations. The goal of the toolkit is to leverage constructionist principles to inform data literacy in children through making; however, the hybrid physical-digital platform may also support accessible literacy programs as well as collaborative sensemaking through data. The second, sensing networks, provides a integrated pipeline for fabricating capacitive touch responsive node-link diagrams using 3D printing. The work enables people to readily construct physical models which can be immediately integrated into existing visualizations to provide a tangible input device grounded in data.

Makerspaces enable people to create digital items for themselves. They also provide the opportunity to build creative thinking and problem solving skills. Unfortunately people with disabilites are often excluded from participating because they are inaccessible. Research into how to make Makerspaces more inclusive is required for people with a range of different skills and abilities. An example of an inclusive circuit making activity is TapeBlocks which consists of chunky foam blocks wrapped in conductive tape with electronic components inserted under or on top. People with physical disabilities can push them together; blind people can feel the vibration and fan versions and people with intellectual disabilities can learn how to make them. TronicBoard are a flat version of TapeBlocks that enable a wider range of activities because they are easier to build into artifacts. There are lots of opportunities to make Making more inclusive but we need research to create and evaluate accessible tools.

The most common approach to identify dysgraphia in children is based on an interview with the pupil made by an expert in the field, who manually analyzes handwritten sentences. To this purpose, one widely adopted evaluation is based on the Brave Handwriting Kinder (BHK) test that takes into account features of handwriting produced by children that are manually annotated by an examiner. One important factor to consider is that during the test execution, the examiner can take into account the way the handwriting is produced (e.g., the posture of the pupils or how they handle the pen). However, the analysis of the handwritten specimen can be time-consuming and subjective, posing challenges in accurate and efficient diagnosis. In our recent research, we used smart-pens to perform the test and machine-learning based approaches to assess the dysgraphia level. The smart-pen allows us to capture features related to the speed of writing and pressure on the pen, in addition to the writing trajectory. Concerning the handwriting analysis, we implemented an algorithmic version of the BHK test and compared its performances with those achieved by an approach relying on deep-learning architectures. The children’s handwritings have been also analyzed and scored, according to their potential level of dysgraphia by elementary school teachers.

This research direction delves into the crucial intersection of user needs and abilities for crafting inclusive data visualizations. It addresses a comprehensive spectrum of disabilities, spanning visual, cognitive, physical, and auditory impairments, while also considering broader user preferences and individual lived experiences. Future research should recognize the inadvertent biases and exclusions in current practices that often arise from an assumption that individuals with disabilities are solely consumers, rather than potential producers of data and visualizations. Future research should address these problems by focusing on the following key dimensions: (1) Scrutinize unique and common requirements of diverse disability groups, aiming to uncover shared needs and barriers across the spectrum. Research is needed to identify tasks users want to accomplish and the challenges they encounter, thereby fostering an understanding of potential synergies among user groups. (2) Explore where and how individuals with disabilities interact with data, eliciting meaningful tasks while addressing systemic barriers. (3) Amplify user participation by advocating for the involvement of diverse user groups through co-design methods and community outreach. Research should also balance the broader needs of user groups with individual lived experiences by incorporating personalization.

The working group took some time to finalize the scope of this working group as the initial discussion focused on modalities and associated sensory channels/variables which overlapped with the working group on Representation. One interesting point was that variables may be multisensory, e.g. smoothness is a mix of haptic and visual [1]. We then decided to focus more on the underlying technologies.

We identified the following presentation technologies to support more inclusive data visualization: speech, sonification, magnification/image enhancement, dynamic tactile displays, tactile graphics, 3D models and data physicalization inc dynamic and shape-changing materials, force-feedback/vibrotactile/ultrasound haptic interfaces, tactiles and 3D models with audio-labels, making with everyday materials. The interaction technologies were: gesture/touch, speech, keyboard, mouse/joystick, multimodal combinations.

We discussed the requirements/characteristics of technology and perception (resolution both spatial and temporal, transparency, dynamicity, dimensionality (1-, 2-, 3-D) and the factors impacting technology use/choice (individual abilities, social acceptability, cost, availability, use context).

We recognized that lots had been done in sonification, physiology of haptic perception, color vision deficiencies but still much to do in these areas. The open questions/grand challenges list was huge ranging from how to provide sign language labels for Deaf users, choice of language for IDD users to exploring shape-changing technologies, use of making and dynamic tactile displays. A particular focus was multimodality. Concrete research questions that could/should be addressed now included:

• $\mathrm{■}$

  What is a “grammar of graphics” but for multimodal representations? (What’re the defaults, what is shared between modalities, what is modality specific)?

• $\mathrm{■}$

  How can multimodal representations help sighted and PWDs collaboratively analyze data.

• $\mathrm{■}$

  How do we understand the crossmodal perceptual trade-offs necessary to support effective sensemaking? How might these trade-offs map to different technologies?

• $\mathrm{■}$

  Do physical representations of (personal) data help people with IDD express themselves|think with data?

• $\mathrm{■}$

  How do we design interactions for dynamic tactile displays (zooming, filtering etc) that preserve mental model and make changes salient