Human in the (Process) Mines

Process mining (PM) is the discipline aimed at extracting information from events recorded by information systems executing processes [22]. The operational context of process mining is multi-variate in that data potentially stems from multiple sources and pertains to diverse dimensions (control flow, time, resources, object lifecycle). As argued by the seminal work of Beerepoot et al. [2], a critical, yet largely unaddressed, issue of process mining is the fixed-granularity level of process analysis, namely the inability to navigate through distinct, and possibly domain-specific, dimensions. This problem makes the investigation of process improvement less effective, as studied by Kubrak et al. [13], as it entails a partial view over an inherently complex search space, which is typical of knowledge-intensive processes [5, 6]. Visual Analytics (VA) is the discipline of supporting users’ analytical reasoning through the use of interactive visual interfaces [12]. VA is intended to keep the user within the analysis loop, so that human comprehension and perception actively contribute to generating new insights and increasing confidence with the analysis results. VA has extensively studied the problem of representing complex, multi-faceted phenomena (see, e.g., [11, 16]), which is also the type of phenomena often encountered within real-life business processes.

Thus far, there has been limited cooperation between PM and VA. In this paper, we argue that VA can play a pivotal role in addressing the above limitation of fixed-granularity in process mining. With a cross-pollinating effect, PM can equip VA with a collection of established algorithms and techniques for the automated generation of process-oriented representations of system dynamics.

During the seminar, we envisioned a novel framework that resorts to VA for the interactive investigation of factual evidence of process insights revolving around information mined from data sources that go under the name of sequence event data, to use VA’s terminology, or event logs, as per the PM nomenclature. Our framework will follow a multi-layer approach, providing end users with a context-aware visualization integrating classical PM representation elements (such as workflow nets [19], directly-follows graphs [21], or declarative process maps [7]) with additional diagrams and visual cues tailored for context-variable and metadata representations (e.g., timelines and calendars for time, geographical or building maps for space). We introduce a couple of use cases that could benefit from such a framework. The interconnection of the different layers and their anchorage to a backdrop onto which the representations are projected provides the user with means to foster explainability for process analysis and, eventually, enhancement.

In the remainder of this section, Sec. 4.2.2 describes our framework, illustrating its key concepts and the interplay of its components. Section 4.2.3 showcases two use cases demonstrating the application of the Tiramisù approach in the area of knowledge-intensive processes. Section 4.2.4 acknowledges some known limitations of our work, paving the path for future work discussed in Sec. 4.2.5.

In Tiramisù we envision a visualization framework designed to support sensemaking when dealing with complex PM event sequences. Our goal is to augment existing process models with additional dimensions that can provide further context during the analysis, easing the generation of insights and generally providing a more comprehensive understanding of the process and phenomenon under investigation. With this goal in mind, we envision Tiramisù as a multi-level framework (see Fig. 3), reminiscent of the popular tiramisù cake. The framework will be structured with a “backdrop”, a main layer with the process model, and one or more dimension layers. These are meant to be superimposed on the first two, in a details-on-demand fashion, thus reminding the different layers of a tiramisù.

In this section, we illustrate some scenarios that motivate our work. We focus on two classes of knowledge-intensive processes, pertaining to healthcare [15] and personal information management [4].

Evidence-based discovery approaches suffer from the potential bias given that only a part of processes (especially the knowledge-intensive ones) are recorded by computerized systems [9]. Our framework is in this sense no different. The injection of additional facts stemming from domain experts and users is complementary to this investigation and paves the path for future work (see Section 4.1 for further details in this regard).

Also, we assume the user knows what visualizations best suit their needs, although this may be a challenge on its own. Investigating this aspect goes beyond the scope of this report. However, the interested reader is referred to the work of Sirgmets et al. [18], which presents a framework aimed to guide design choices for the effective visualization of analytical data.

Data-based process analysis tasks often involve a significant amount of time for extracting, reformatting, and filtering event logs from information systems [20]. Our framework requires these preliminary operations too, with the addition that oftentimes knowledge-intensive processes record executions over a heterogeneous set of applications and devices in partially structured or unstructured formats [6]. Furthermore, the notion of a process instance (case) tends to be less defined in such contexts, thus requiring a prior customizable reconciliation of shared events [1]. The emergence of the novel and event data meta-models that are less centered around the concept of case could be beneficial to the information processing we envision in Tiramisù [23].

Finally, our framework covers the visualization aspects related to VA in PM. However, the interaction aspects are only partly discussed, leaving a gap which further iteration of this framework should address. This would be made easier and more effective if a task taxonomy about VA in PM were made available, opening a further research direction with the potential to bring together the two disciplines.

In this report, we have envisioned Tiramisù, a framework based on the multi-layered representation of mined process information helping the user navigate the multi-faceted information at hand while keeping the data under the focus consistently linked and navigable across different dimensions.

We foresee the following research endeavors in the future. A further elaboration of the framework and its application to the use case scenarios identified. An evaluation based on an empirical study involving users is in our plans to assess the efficacy and effectiveness of our solution. The implementation of a working prototype is crucial to this extent and is part of our agenda. We envision the customization and refactoring of layers, their aggregation rules, and backdrops as key challenges to be addressed in the future. Finally, we observe that stepping from a single-backdrop multi-layer design to a tree-like hierarchical structure of backdrops could initiate a new path toward a significant extension of the expressive richness guaranteed by our framework.

Process mining enables process analysts to provide insights into the business processes of organizations. Process mining tools use event logs as data inputs in order to make sense of the organization’s processes. An event log includes data that has been generated by various systems within an organization. Process mining aims to discover, analyze, monitor and ultimately improve business processes, all based on the event log. The process of process mining is a highly interactive process. The general aim faced by analysts is to make sense of the process data, where the data serves as the input coming from the external world. This sense-making process entails an iterative cycle of data exploration toward revealing insights about the process investigated. This process entails a high cognitive load, and as such it requires high cognitive effort. Process analysts are supported by tools that enable the generalization of various visualizations of the business processes as they are executed in real time. Appropriate visualization can reduce cognitive load, and in fact serve as an extended cognition mechanism supporting and enhancing the cognitive process [11].

However, for a visualization to enable this cognitive benefit, we need to ensure that it is a good fit for both the task and the user. We therefore ask the following research question: What comprehension metrics can be used for effectively evaluating visualizations for solving process mining tasks?

In order to answer this question, we looked for relevant metrics from the field of visualization and visual analytics, and found the ICE-T hierarchical evaluation framework for evaluating visualizations as a good starting point. Our aim is to adapt this framework for the process mining domain and validate it as reliable metrics for evaluating the comprehension of a process mining visualizations.

To evaluate the effectiveness of process mining visualizations for solving process mining tasks, we first had to think of tasks that are typically solved by process analysts when using process mining, as well as the various visualizations generated by the different process mining tools. As one of the most typical uses of process mining is process discovery, we decided to take one of the most common process discovery tasks “Discover the most frequent process path” as a starting point. The process mining visualization that is mostly used by a process analyst to perform this task is the directly-follows graph (DFG). The DFG generated from the above-mentioned data set is shown in Fig. 4 and has been generated using the process mining tool Disco. We used the customized ICE-T comprehension metrics to evaluate the effectiveness of the DFG with regard to the task of discovering the most frequent process path. Five experts evaluated the DFG visualization using the four components of the ICE-T framework. From the five experts, one is a process mining practitioner, and four are researchers. The four researchers come from the process mining, conceptual modeling, and visual analytics disciplines.

As a first step, each expert evaluated the DFG from Figure 4 individually. The ICE-T framework uses a 7-point Likert scale. After the individual evaluations were done, the five experts discussed their perceived understanding of the heuristics listed for each ICE-T component, as well as the difficulty to evaluate each of the heuristics against the DFG. As a result, we were able to identify two aspects that should be modified in order to fit the purpose of evaluating process mining visualizations for solving process mining tasks. The first aspect is with regard to the terminology used to describe the heuristics. Certain terms from the visual analytics community share the same semantics but are referred to differently in the process mining community. For example, “data case” in visual analytics is referred to a “process trace” in the process mining community. Therefore, we had to reformulate the terminology into the terminology used and understood by process mining analysts. Second, we found that some heuristics should be split into two heuristics, because the heuristic includes two different concepts that might lead to different evaluations of the same heuristic. We split such composed heuristics into two separate heuristics.

The next steps for continuing this research stream are:

• $\mathrm{■}$

  Fully customize the ICE-T metrics for the process mining domain

• $\mathrm{■}$

  Validate the customized ICE-T (prototypical)

  • –

    Select existing process mining tasks

  • –

    Select existing and emerging visualizations from the other Working Groups of the Dagstuhl seminar to support the process mining tasks

  • –

    Conduct the ICE-T survey for each task and visualization with 5 process mining experts

• $\mathrm{■}$

  Prepare a vision/research paper

• $\mathrm{■}$

  Generalize the customized ICE-T (experiment)

In order to arrive at a well-balanced multi-faceted representation, it makes sense to follow a two-step design procedure [9]. First, a base representation has to be defined for the primary data facet whose depiction will govern the overall display. In alignment with the effectiveness principle [5], the most powerful visual variables should be used for the primary facet (usually, this means mapping the primary facet to the visual variable of position). Second, the additional data facet(s) will be incorporated into the base representation using further visual channels (e.g., color, size, orientation, or shape). In the next section, we describe the categorization of data facets applicable to process mining artifacts.

A wide variety of facets is identifiable in process mining, where the relevance of a facet as a driving force for a visual design largely depends on the perspective from which users need to look at their data. In general, there are key facets to be considered (in some cases to be refined hierarchically) according to the visual analytics literature. We mostly discussed the following facets inspired by [3] and [9] (see Figure 6):

Note that some entities, e.g., activities and objects, can also act as a given attribute. To distinguish between these roles in existing visual representations of process mining data and models, we studied the use of marks and visual channels, as the information-bearing building blocks in visualization idioms. Data that are explicitly represented as marks are interpreted as entities. If data are encoded using visual channels, we interpret them as attributes.

To develop a common understanding of existing process mining visualizations, we worked on a categorization that assigns to commonly used visual designs the data facets that they represent. In addition to the visualization-facet mapping, we also cover the marks and visual channels for selected examples:

• $\mathrm{■}$

  Dotted chart (scatter plot)

  • –

    Primary facet: Time

  • –

    Additional facets: Topology (entities – events) and Attributes

  • –

    Mark: Event $\mapsto$ dot

  • –

    Visual channel: Activity $\mapsto$ color

• $\mathrm{■}$

  Performance spectrum (Marey’s diagram)

  • –

    Primary facet: Time (temporal order)

  • –

    Additional facets: Time (temporal distance) and Attributes

  • –

    Mark: Derived Attributes (temporal distance of two events) $\mapsto$ line

  • –

    Visual channel: end-point positioning $\mapsto$ color

• $\mathrm{■}$

  Maps

  • –

    Primary facet: Space

  • –

    Additional facets: Attribute

• $\mathrm{■}$

  Frequency diagrams (basic charts)

  • –

    Primary facet: Attribute

  • –

    Additional facets: anything

• $\mathrm{■}$

  Variant diagrams

  • –

    Primary facet: Time (temporal order)

  • –

    Additional facets: Topology (entities – activities) and Attributes

• $\mathrm{■}$

  Process matrix

  • –

    Primary facet: Topology (relations)

  • –

    Additional facets: Attribute

• $\mathrm{■}$

  Directly-Follow Graphs

  • –

    Primary facet: Time (temporal order)

  • –

    Additional facets: Topology (relations – activities) and Attributes

• $\mathrm{■}$

  BPMN

  • –

    Primary facet: Time (temporal order)

  • –

    Additional facets: Topology (relations – activities and operators) and Attributes

• $\mathrm{■}$

  Petri Nets

  • –

    Primary facet: Time (temporal order)

  • –

    Additional facets: Topology (relations – activities and operators) and Attributes

• $\mathrm{■}$

  Process Trees

  • –

    Primary facet: Time (temporal order)

  • –

    Additional facets: Topology (relations – activities and operators) and Attributes

• $\mathrm{■}$

  Object-Centric Petri Nets

  • –

    Primary facet: Time (temporal order)

  • –

    Additional facets: Topology (relations – activities and operators) and Attributes (objects)

From the existing process mining visual representations classified according to the earlier defined categorization, we observed the following:

• $\mathrm{■}$

  Time tends to be the primary facet, mostly focusing on temporal order.

• $\mathrm{■}$

  Space is rarely included in visual representations as a data facet.

• $\mathrm{■}$

  Attribute as a primary facet is only used in basic charts; more complex advanced multivariate visualizations are not used (e.g., scatterplot-matrix, parallel-coordinate plot, glyph-based approaches).

• $\mathrm{■}$

  BPMN, Petri Nets, and Process trees all use the same facets from a data visualization point of view, although they use different semantics.

• $\mathrm{■}$

  In contrast to the other process modeling languages, DFGs do not have operators that can be used as a mark.

• $\mathrm{■}$

  Current object-centric process mining maps use the most important element, object, only as a visual channel: if color is removed, it can be considered a traditional Petri Net.

Based on the identified facets and observations, we plan to condense our work into a journal publication. The publication is intended to present the following contributions.

1. 1.

   Propose a multi-faceted framework for the categorization of PM visualizations

2. 2.

   Categorization of existing process mining visualizations

3. 3.

   Identification of blind spots and challenges in PM visualizations

4. 4.

   Outline opportunities how to enhance PM with interactivity and analytical abstractions

As defined in Munzner [5], visual idioms are described with Data, Marks, Visual Channels, and Tasks. With the Facets from the categorization, we cover the Data element. Furthermore, we identify for each of the existing visual process mining representations the Marks and Visual Channels. We did not cover the Tasks in our working group as this was the topic of working group Group 6: Task taxonomies in process mining and visual analytics. By combining the results of both working groups, we can ultimately construct novel visual representations for specific process mining tasks.

The working group also discussed the importance of analytical abstractions and interactivity for advancing existing process mining practices. The visual analytics community provides helpful categorizations of analytical abstractions (see Figure 7), which basically can be applied to each data facet. This would be something to be considered in future work. Similarly, interaction techniques such as view coordination [1], focus+context [2], interactive lenses [8], interactive comparison [7], or dynamic view adaptations [6] would be helpful to integrate into existing process mining tools.

Process mining defines a set of techniques that take event sequence data in the form of event logs as an input in order to produce evidence-based insights into the execution of business processes. Different groups of techniques are distinguished, including automatic process discovery, conformance checking, enhancement, performance analysis, or deviance analysis [10, 3]. Conformance checking is of particular importance in this context. It allows analysts to investigate to which extent observed event sequences deviate from normative process models and which behavior constraints are violated. This can for example be used for compliance checking to check how far the execution of business processes complies with certain regulations, such as the General Data Protection Regulation (GDPR) or clinical guidelines.

The current practice of conformance checking builds on functionality implemented in professional process mining tools. These tools, in essence, support basic conformance checking tasks: They allow users to (i) quantify the conformance of a process, (ii) compare the conformance values of, e.g., different locations, (iii) localize the specific deviations in a process [6], and (iv) explaining those deviations by means of external factors [8]. Hence, we observe that support for conformance checking has been implemented in various ways. So far, research has not investigated which of the various approaches are most effective for analysts in order to fulfill their tasks. Overall, much of conformance checking research has been driven by technical challenges of efficiently calculating alignments between event sequences and process models [2], defining precision and fitness measures with sound formal properties [7], and providing feedback to business users who are not familiar with formal concepts [4].

In general, research on process mining can be framed at different levels [11]. Empirical research has been conducted at the level of organizational anchoring and impact [1] but mostly abstracted from the analyst as a user. What is by and large missing is a research strategy that considers the tasks and user characteristics as a focus of conformance checking research [5]. More specifically, several research questions miss specific answers from empirical and engineering research on process conformance checking, including the following.

• $\mathrm{■}$

  What kind of characteristics do the users have?

• $\mathrm{■}$

  Which are the conformance checking tasks?

• $\mathrm{■}$

  What data is required for each of the tasks?

• $\mathrm{■}$

  Which techniques can be used for specific tasks? What is missing?

• $\mathrm{■}$

  What are the functional/non-functional requirements of the systems?

• $\mathrm{■}$

  Which are effective conformance checking algorithms for the tasks?

• $\mathrm{■}$

  Which are effective visualizations for the conformance checking tasks?

• $\mathrm{■}$

  How to design an effective interaction between the conformance checking algorithms and visualizations?

• $\mathrm{■}$

  How to support annotations and editing to the original data?

• $\mathrm{■}$

  How to support decision-making and actions (i.e., implement interventions) in the real world?

• $\mathrm{■}$

  How to cope with updates of the real world in the conformance checking visualization system?

This report presents the discussion of the working group on conformance checking of the Dagstuhl Seminar 23271 on “Human in the (Process) Mines”, held at Schloss Dagstuhl on 2–7 July 2023. We conclude by summarizing a set of action points that our working group identified.

As a first step, the working group identified the limitations of current visualization techniques adopted by conformance checking tools. It emerged that deviations are often not represented in an effective way that stands out from the normal execution flow. This is typically caused by a gap between visualization producers, users who create the graphical notations, and visualization consumers, users who have to interpret the graphical notation.

The working group then discussed the typical workflow that visual analytics researchers adopt to address these issues [9]. This typically consists in involving domain experts to identify what kind of tasks (actions) that they need to perform to obtain insights. Then conformance checking techniques able to address these tasks are adopted or developed, which then produce output data. Models and visuals will then be built to effectively represent output data. Finally, the visual representation will be checked by domain experts to see if they are effective for accomplishing their tasks, what is missing, and what can be improved.

Following this workflow, the working group analyzed different application domains in which they had direct experience (e.g., by previous research projects) in order to identify the tasks (actions) that the user need to perform to obtain insights on the compliance of its processes. From this analysis, it emerged that, for auditing purposes, it is more relevant to produce a list of cases that caused a deviation and see their impact. If one has a large number of deviations, these need to be grouped into classes. The hypothesis in conformance checking is that if the trace is the same, then they have the same meaning. However, information on resources, data, etc, is currently missing.

This lead to the definition of which information should be the input for a system combining visual analytics with conformance checking, and which interactions with the domain expert this system should support (see Fig. 9).

To apply conformance checking one needs to create a model from regulations and norms, and an event log from information on a database or other data sources. The model can use different formalisms (DFG, BPMN, Declare, etc.), and the logs can be at different levels (task execution, task start/complete, changes in data, etc). Then, the outcome of conformance checking would be deviations. Deviations are grouped in harmful/not harmful, variants, or having additional or missing activities. They are then typically qualified based on risk by domain experts, and quantified by computing measures from the types. Then, deviations are explained. Qualifying, Quantifying and Grouping can be performed multiple times.

Thank to conformance checking, one can perceive how the actual process is performed, then make sense of process-related information, which leads to data-driven decision making, that can eventually lead to implementing interventions on the process.

VA can 1) apply an algorithm and then visualize the results: you choose the algorithm and visualize/interact only with the output. 2) Interact in a cyclic behavior: after observing the result, parameters are changed and the algorithm is rerun. 3) computational steering: while the algorithm is running one changes the parameters and adjusts on-the-fly what is happening.

One can also edit VA results with annotations and corrections. Corrections directly change the data sources (e.g., fixing a wrong timestamp). Annotations highlight relevant behaviors in the data, without changing the data sources (e.g., noting that a regulation has changed, explaining why a trace is non-conformant).

Next, we discussed what input data is available and needed for proper and useful conformance checking. We identified, on the one hand, the event log or a stream of events that represents the process execution. On the other hand, we discussed that the business analyst usually does not provide a standard process model, but rather a set of rules to which the process needs to comply. This can be control flow (activity A must be followed by B), time (activity A must complete within 30 minutes), resource (activity A must be executed by the financial department), and data-related rules (activity A must produce an invoice with a fee not higher than 5000 EUR). These rules need to be formalized, such that conformance checking techniques can automatically identify deviations in the traces of an event log, the individual process executions. Sometimes, business analyst has no rules at hand, but they have hypotheses in mind about how their process should be executed and what should not happen. For such a case, the business analyst needs support in formulating hypotheses and confirming them.

Our discussions this week have led us to believe that visual analytics can provide a completely new angle on process conformance, with a much stronger emphasis on the needs of the user and the tasks that conformance checking is supposed to fulfill. The working group has agreed on two immediate next steps. First, we would like to gather our newly established insights into the potential of visual analytics for the advancement of conformance checking into a workshop paper, outlining the ideas and concepts already presented in this report. Second, we would like to further develop, substantiate, and structure these ideas, with the goal of submitting a joint research proposal.

Concretely, the next steps that we want to take for the substantiation of our ideas are the following:

• $\mathrm{■}$

  Identify additional application scenarios, such as healthcare, purchasing, infection control, ambient assisted living, or logistics, and characterize them with regard to their similarities and differences to obtain a clear set of objectives

• $\mathrm{■}$

  Conduct an design study [9] with process analysts or process managers to verify and extend the preliminary list of tasks that conformance checking should fulfill

• $\mathrm{■}$

  Collect and analyze process-related regulations to develop a better understanding of the different types of rules that business processes are subject to

• $\mathrm{■}$

  Analyze existing visualizations of timed event sequence data, such as [12], from the VA domain to find out which functionalities they already provide

Over the last two decades, process mining has developed into a collection of techniques and tools that have the potential to provide immense value to business users. Although the techniques have gained a significant level of sophistication in terms of their algorithmic approaches and outcomes, supporting the human within the process of process mining [7] has so far received little attention.

Therefore, the goal of this work is to develop a task taxonomy for supporting human-centered process mining with visual analytics.

We argue that the development of such a taxonomy, which we call Milana, needs to be guided by multiple perspectives that include process mining, visual analytics and cognitive underpinnings. The current body of knowledge is lacking a full exploitation of opportunities that sit at the intersection of human-centered process mining activities, the various cognitive aspects of these activities, and visual ways of supporting them.

We envision that Milana can provide a reference to apply a three-way mapping between analytical tasks in process mining, cognitive tasks, and visualization tasks. Such a mapping will establish a common vocabulary and contribute to a shared understanding between the different communities. Moreover, it can be leveraged to justify design choices of process visualizations, identify gaps to inform future work, and be used to inform the measurements for evaluating process visualizations.

A taxonomy (or taxonomic classification) is a scheme of classification, especially a hierarchical classification, in which things are organized into groups or types [8]. Typically, the development of a taxonomy involves a number of steps including (i) identifying the concepts and relationships within the domain(s) of interest, (ii) establishing a nomenclature for the representation of the concepts, (iii) constructing the taxonomy using established notation (usually hierarchical), (iv) evaluating or validating it against its intended goal, and (v) setting up a mechanism for its continued maintenance.

During the seminar, we derived an approach to develop Milana. As shown in Fig 10, it consists of three steps to identify concepts and relationships of the taxonomy:

1. 1.

   Elicit process mining tasks (PMT) (Sec. 4.6.2.1)

2. 2.

   Mapping process mining tasks to cognitive tasks (CT) (Sec. 4.6.2.2)

3. 3.

   Mapping combinations of process mining tasks and cognitive tasks to visualization tasks (VT) (Sec. 4.6.2.3)

Taking these steps as a starting point, we can build on the large body of knowledge on visualization and visual analytics that gives us a proven link on relevant and most suited visual representations which is the main goal of developing Milana.

Considering that the concepts will be derived from three different bodies of knowledge, namely process mining, visual analytics and cognitive theories, establishing the nomenclature is a critical and non-trivial step and will continue to be refined as the taxonomy is further developed. The work on the taxonomy construction, validation and plan for maintenance is yet to be undertaken.

In the future we plan to continue our work as follows.

1. 1.

   Concerning the identification of process mining tasks, we will continue to apply the approach to more questions that are collected from the BPICs until we reach saturation (cf. Sec. 4.6.2.1).

2. 2.

   For mapping the identified process mining tasks to the cognitive tasks that would be required, a critical evaluation of the relevant cognitive tasks that could guide the identification of visual representations is outstanding (cf. Sec. 4.6.2.2).

3. 3.

   After the process mining tasks are mapped to cognitive tasks, visualization tasks will be linked to these. Starting from the taxonomy by Brehmer & Munzner [2] appropriate options will be identified (cf. Sec. 4.6.2.3).

4. 4.

   Finally, Milana will be validated (e.g., case studies, user studies, and experiments).

Milana will pave the way for both evaluating existing process analysis visualizations in terms of their appropriateness for the tasks as well as provide a blueprint for the design of novel visual representations.