Visualization and the Humanities: Towards a Shared Research Agenda

Only one method exists in the humanities: interpretative reading. The method has many modalities (approaches) and techniques (applications) but within the historical tradition from which the humanities derive, the act of reading (mainly) texts is the single approach to knowledge and understanding. This can be tracked in Western culture to the reading of sacred texts in the religious communities of the three Abrahamic faiths: Judaism, Christianity, and Islam. This practice is also central to Eastern traditions in Buddhism, Hinduism, and Confucianism. While the study of images, dance, spatial rituals, and other activities are persistent in humanistic practices, the development of disciplines that considered these as objects of study comes much later than textual study. As a field, art history appears in the late 18th-century. The social sciences such as sociology, anthropology, archaeology are largely the outcome of 19th century syntheses of empirical methods from natural sciences and humanistic topics of research. By analogy, these disciplines perform “readings” of images, practices, and social phenomena that are not constituted solely by texts. The various inflections that nuance the method of interpretative reading – critical theory, deconstruction, queer theory, feminism, decolonizing approaches – each bring a set of distinct and significant perspectives and insights into the ideological frame, calling attention to often unacknowledged assumptions and biases of traditional interpretation – but these are also interpretative reading practices.

With this talk, we enter ongoing traditions of salvaging, repurposing, and re-valuing the remnants of the past. Drawing predominantly from feminist, queer, and archival critical theories, as well as artistic and/or vernacular practices, we offer what we call a “scavenger method”. We focus on scraps that preserve (albeit imperfectly, in different degrees of ruin) material traces of other times and “users”. These scraps offer opportunities to read history “against the grain” of dominant narratives and work as a provocation for visualization and humanities researchers to experiment with practices of re/contextualization of historically specific data.

Scholars in the Humanities often think of data visualization as graphs, maps and trees. In this talk, I argue for the essential role of other, less expected types of visualization, and how humanities hypothesis can extend and diversify the notion of data and visualization, both for the fields of humanities and data visualization. Indeed, for scholars in Early Modern literature, one example of visualization revolutions lies in the ability to access and read Early Modern texts in their original layout, thanks to initiative such as Google Books or Gallica. Considering type size, layout and text formatting as data reintegrate issues of materiality directly into textual analysis, challenging traditional approaches and methods in literary studies. I used the example of La Bruyòre’s Caractères to show how several interfaces designed with a fine-grained understanding of the work in mind enable the work to be visualized differently, profoundly transforming the understanding and perception of a literary masterwork. Such interfaces challenge the preconceived idea of a marmoreal masterpiece, from restoring its inherent dynamics (multiple editions with multiple additions and modifications over a short period), to enabling new approaches to its various translations across Europe.

While more traditional types of neural networks have long been part of digital art history, the epistemic implications and methodological affordances of contemporary large-scale vision models have not yet been systematically analyzed. We focus our analysis on two main aspects that, together, seem to suggest a coming paradigm shift towards a “digital” art history in Johanna Drucker’s sense. On the one hand, the visual-cultural repertoire newly encoded in large-scale vision models has an outsized effect on digital art history. The inclusion of significant numbers of non-photographic images allows for the extraction and automation of different forms of visual logics, from which computational iconographies of almost arbitrary complexity emerge. Large-scale vision models have “seen” large parts of the Western visual canon mediated by Net visual culture, and they continuously solidify and concretize this canon through their already widespread application in all aspects of digital life. On the other hand, based on a technical case study of utilizing a contemporary large-scale visual model to investigate basic questions from the field of urbanism, we suggest that such systems require a new critical methodology that takes into account the epistemic entanglement of a model and its applications. This new methodology reads its corpora through a neural model’s training data, and vice versa: the visual ideologies of research datasets and training datasets become entangled.

My presentation discusses conditions under which research scenarios in the humanities can benefit from abstract data visualization. Using the Damast prototype as an example, the talk demonstrates how the combination of abstract data visualization, the recording of analysis provenance, and long-term data preservation support the reproducibility of interactive visual analyses. However, even when providing a certain level of reproducibility, it is necessary to provide sufficient context when presenting visually abstracted research data. An orthogonal topic are cultural differences between research disciplines in collaborative projects. Publication traditions and the reception of joint research in the individual disciplines can negatively impact early-stage research careers. Another issue are the different requirements regarding the sustainability of research outcomes. If collaborative research should become a success story, these aspects need to be discussed openly.

The reification of data is the bedrock of visualization. Uncertainty adds roadblocks to this process: how do we chart the unknown or the unknowable? Overcoming this roadblock involves the reduction of uncertainty down to probabilities, a “real abstraction” that fails to capture how people experience or reason about uncertainty, but nevertheless has normative force: we often try to make people act as “statistical golems”, and punish/degrade them when they don’t. In this talk I explore the frictions introduced by the probability-based visualization of uncertainty, and the visualization work that highlights these frictions.

Visualization researchers are very familiar with the epistemology of natural sciences and follow its model for exploring data. The social sciences do not follow this epistemology, leading to misunderstandings between the two communities when discussing how visualization tools should support research. There is a need to align the expectations and assumptions between the two communities to converge to usable tools and more constructive criticisms. In particular, the social sciences deal with several levels of beliefs for their research questions; some hypotheses are plausible while others are competing and can remain with these statuses for a long time (for generations). Additionally, the nature of the “data” is not as clear-cut as in the natural sciences. Therefore, this talk is an invitation to visualization researchers and practitioners to better support the subtle levels of truthfulness required by the social sciences as well as mechanisms to intervene in the data to provide contextual information (in a transparent manner).

I will start this talk by discussing the differences and nuances among the concepts of Best Practices, Guidelines, Lessons Learned, and Heuristics, pointing out frequently ignored pitfalls and potentially looming pain points. I will then discuss how the pendulum between advantages and disadvantages can be mitigated by individual attitudes and actions.

In this contribution, I discuss the suboptimal visualizations that textual scholarship applies to transmedialize (historical) text sources in a digital environment. Current digital scholarly editions are text (or glyph) essentialist and ignore for the most part all material information of the documents that they are contained in, ignoring thus also all aspects of non-glyph communication besides sometimes relational information expressed by e.g. marginals, footnotes, etc. This current TEI-XML “good practices” based approach has hermeneutical and pernicious epistemological ramifications as digital scholarly editions are repeating indefinitely a book metaphor, but in a bland abstracted, reductive, and non-creative way. Hermeneutically, this does a disservice to the texts and documents they are trying to re-represent. Epistemologically, they are relatively “poor” and do not inspire exploration of any new engagement or knowledge inference from the texts themselves.

Looking beyond the model of formal research collaborations between disciplined visualization researchers & humanities scholars, this input reflects on expanded ideas of synergy (sun+ergos) as “working together.” In role-based thinking, working together may include a humanities scholar + visualization researcher, or artist… or a hum scholar who is ALSO a vis researcher / artist (& vice versa). In work-based thinking, beyond the “two halves” work model (“I do vis, you do humanities”) is instead the “speak the same language” model: e.g. this specific intersection of the humviz is a domain, so we develop a shared domain-specific language (in a technical or general sense). In each community you afford a different “you” for purposes of synergy – my past collaborative “mes” include artist, domain expert, developer et cetera. Some of the most productive thinking about “working together” is grounded in personal experiences, situating ourselves and reflecting on projects and roles, moving between e.g. the humanities, information visualization, computer science, & fine arts. We collaborators are usually more complex than the simplified role they overtly play, and have more to offer a collaboration. I explore this idea through role-based reflection on four of my visualization+humanities synergies at different career stages: 1) an artist researching historical visualization forms as traditions within community (as PhD researcher, Software Studies Initiative), 2) a Critical Code Studies hermeneuticist using artist-provocateur exploratory visualization (as co-author on Reading Project: A Collaborative Approach to Digital Literary Criticism), 3) the “infrastructure guy” supporting humanities teams with virtual machine clouds while they use visualization to read (grant Co-PI on the WhatEvery1Says Project), and 4) teaching viz to humanities students (as faculty lab director / project lead) in two projects: Panelcode, a minimal markup language for visualizing comics compositions as abstract layouts, and in The Transverse Reading Project, a visual atlas of branching narratives. When “working together” in hum+viz, reflect on your experiences and situate yourself. When seeking synergies, which “you” is the collaborator?

Unlike the paper medical record, which is created by clinicians for their own use, the Electronic Health Record (EHR) is predominantly designed by non-clinicians. The EHR interface is the cornerstone tool through which clinicians review, interpret, create and curate patient data. EHR interfaces are consistently criticized for their poor usability, which results in clinician burnout and sub-optimal, clinically misaligned data being propagated throughout a health system–infinitely and without degradation.

This research aims to realign EHR interface design with the central tenet of user-centredness by better understanding the clinician user. It seeks to achieve this by understanding how the EHR interface can specifically support a clinician’s practice–defined here for the first time as clinical usability (CU).

The hypothesis is that, using a provocative prototype designed by a single clinician as a visual starting point, a Participatory Action Research (PAR) approach can effectively capture, codify and communicate a clinician community’s CU knowledge to inform the trans-disciplinary field of EHR interface design. This leads to the central research question, “How can a clinician community contribute CU knowledge to the design of their EHR interface?”

This research contributes tangible, real-world and communicable new knowledge. First, CU knowledge within a set of novel CU-specific heuristics, which bridge the longstanding gaps between the designer and clinician user, and design and evaluation, that have come to characterize this design field. Secondly, CU knowledge embodied within the Sexually Transmitted Infection Query Interface (STIQI) development prototype and its Representational Epistemic (REEP) design blueprint, not previously seen in either commercial or research settings.

Data visualization, as a practice and as an academic field, has dedicated more time to visualizing uncertainty than ambiguity. Those are not the same thing: although knowledge reduces uncertainty, it amplifies ambiguity. Ambiguity arises as a feature of the things, beings and phenomena we describe and study.

In this short talk, I draw on a minimal case: community detection in networks. Although we can say that communities exist in the network I use for an example, their demarcations are ambiguous. I show five different ways to visualize that ambiguity, and I propose a scale to help people assess the status of ambiguity in their own visualizations, or that of others.

1. 1.

   Ambiguity as noise: it is not visualized

2. 2.

   Ambiguity as an accident: visualized as an anecdote

3. 3.

   Ambiguity as context: it is visualized, but as an afterthought

4. 4.

   Ambiguity as a problem: it is fully visualized, but gets in the way of understanding the rest

5. 5.

   Ambiguity as a feature: it is the purpose of the visualization.

From there, I sketch a possible program at the intersection of the digital humanities and visualization: building radically thick new tools. It may draw on two influences. The “thick description” formalized by anthropologist Clifford Geertz, and “radical empiricism” as stated by the pragmatist philosopher William James.

It is worth noting that during the discussion following the talk, Johanna Drucker proposed to expand this program to other things similar yet distinct from ambiguity: polyvalence, polysemy, multistability, etc.

When we encounter a line graph or a pie chart, we tend to think of the role of visualization – if we think of it at all – as simply revealing the meaning of the data underneath. The reality, however, is that the act of visualizing data generates meaning in and of itself. In this talk, I return to the origins of modern data visualization in order to excavate this meaning. Exploring two examples of early data visualization – the line graphs of British trade data included in William Playfair’s Commercial and Political Atlas (1786) and the Diagram of a Slave Ship (1789) created and circulated by a group of British antislavery activists, I connect Enlightenment theories of visual and statistical knowledge to contemporaneous ideas about race and nation. By examining and re-visualizing the data associated with these charts, I will further show how data visualization always carries a set of implicit assumptions – and, at times, explicit arguments – about how knowledge is produced, and who is authorized to produce it. Placing this work in the context of my larger project, Data By Design: An Interactive History of Data Visualization, I will conclude with a consideration of the ethics of visualization in the present. Through a discussion of contemporary examples, I will show how data visualization can bear witness to instances of oppression at the same time that it can – if intentionally designed – hold space for what cannot be conveyed through data alone.

In this talk, I discuss the power structures that underlie interdisciplinary work between visualisation and humanities research. Funding cuts in the humanities and the soft power of the computational function to create a power imbalance among collaborators that can lead to “disciplinary capture”. I explore three ways that this imbalance affects the research outcomes: through the erasure of humanistic theory, one-sided positioning, and the devaluation of research outcomes. I outline the possible approaches that humanist scan select to engage with visualisation researchers. I end by introducing the concept of “uncomputational thinking” as a method for visualisation research in a humanist context, and point out the importance of slowness and methodological humility in the process.

This breakout group focused on the polysemic concept of complexity in visualization. One challenge in visualizing digital humanities data is the complexity of the analyses and the objects of inquiry. For instance, a simple bar chart might be considered sufficient to surface information about a company’s sales data over the course of financial quarters (Gelman, 2013), but would not be considered sufficient to capture the important features of a novel corpus or musical scores, or at least not in a way that provides sufficient input for hermeneutical study (Windhager, 2018). There is also a connected tension between the oft-stated goal of visualization to simplify (instantiated in principles like Tufte’s maxim to increase the “data-ink ratio” or reduce “chart junk” (Akbaba, 2021) and the hermeneutical impulse to reveal complexity or unpack tacit assumptions. A final related tension was how to manage the complexity “hidden” in superficially simple visualizations (Kostelnick, 2007): for instance, provenance information, methodological choices, and the mathematical sophistication entailed in layout algorithms and dimensionality reduction methods.

The discussions of the group settled on several angles around how complexity can function as a design material. That is, complexity not as an inherently bad or unnecessary component to be reduced as much as possible, but as a material or resource (like color, text, sound, or even machine learning (Dove, 2017; Holmquist, 2017) that can be used to accomplish a number of design goals. Clarifying this concept involved first exploring the different meanings of complexity (Latour, 2008; Latour, 2012; Norman, 2016). For instance, complexity in the size of the data (say, the number of facets or dimensions), complexity of the transformation or representation, visual complexity of the visualization, complexity in the interactive affordances of the visualization, complexity in interpreting the visualization, and the complexity of the communicative goal.

Exploring these different axes of complexity allowed the construction of examples of interesting locations in “complexity space.” A scatterplot that directly encodes two quantitative values that are relatively straightforward to describe–say, the number of pages in a novel plotted against the date of publication–is differently complex than a scatterplot of, say, a two dimensional projection of the same novels from a high-dimensional space, even though both result in a very similar, superficially “simple” visualization. The backing data, and what it means to interpret visual features like clusters or outliers, are fundamentally different, even as the objects of inquiry and the visual design are identical.

The conversation then moved to examples of “beneficial” complexity (Hullman, 2011): using complexity to, say, encourage the reader of the visualization to slow down and de-familiarize themselves with the phenomenon of interest (Bradley, 2016), communicate or disclose uncertainty or provenance information, encourage new or serendipitous ways of interacting with the data [McCurdy, 2015; Thudt, 2012) or even to simply communicate that the phenomenon of interest are, in fact, more complex than they might otherwise appear. The group concluded with a (partly provocative) statement: that, as with Tesler’s law of UX design (Norman, 2016), there is a “conservation of complexity” in visualization design: a visually simple visualization is likely hiding vast methodological or rhetorical complexity, and a visually complex visualization might allow much more straightforward paths to communication or analysis.

The group explored several potential outputs, settling on an initial multi-faceted study of complexity as design material: collecting interesting examples of artifacts (both visualizations and otherwise) that live in interesting areas of complexity space, definitions and framings of the myriad forms of complexity, and potential functions (both positive uses but also abuses) of complexities in design.