Document Open Access Logo

On the Practical Utility of Diagnoses

Authors Ingo Pill , Johan de Kleer

PDF

File

Thumbnail PDF
PDF
OASIcs.DX.2026.9.pdf
  • Filesize: 0.71 MB
  • 20 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Computing methodologies → Causal reasoning and diagnostics
Keywords
  • Model-based Diagnosis
  • Diagnosis
  • Algorithms

Metrics

Abstract

Diagnoses address a universal need for explanations that arises whenever things go wrong or when we observe some unexpected behavior. The ubiquitous need for such explanations has led to an abundance of algorithmic concepts and computation variants for diagnostic processes. Comparisons have mainly focused on highlighting the virtues of individual algorithms or the required computational efforts. In this paper, we focus on evaluating the results of a diagnosis algorithm in the context of an inspection and repair process. We completely ignore the algorithm’s computation concept and whether the observations would justifiably support multiple diagnoses. In particular, we focus on inspecting and discussing the practical utility of diagnoses as a measure of the unnecessary efforts one has to spend when fully repairing a system. We start with assessing the utility of a single diagnosis and then progress to evaluating ambiguity groups, i.e., sets of diagnoses. We propose corresponding metrics that investigate worst case and average performance, contrast them to the utility metric used in the DX Competition 2010, and discuss the metrics' results for several scenarios.

Cite As Get BibTex

Ingo Pill and Johan de Kleer. On the Practical Utility of Diagnoses. In 37th International Conference on Principles of Diagnosis and Resilient Systems (DX 2026). Open Access Series in Informatics (OASIcs), Volume 148, pp. 9:1-9:20, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026) https://doi.org/10.4230/OASIcs.DX.2026.9

Author Details

Ingo Pill
  • Institute of Software Engineering and Artificial Intelligence, TU Graz, Austria
Johan de Kleer
  • c-infinity, Mountain View, CA, USA

Acknowledgements

We would like to thank the anonymous reviewers for their detailed reviews and very valuable comments.

References

  1. R. Alur, C. Courcoubetis, and M. Yannakakis. Distinguishing tests for nondeterministic and probabilistic machines. In Proceedings of the Twenty-Seventh Annual ACM Symposium on Theory of Computing, STOC '95, pages 363-372, 1995. URL: https://doi.org/10.1145/225058.225161.
  2. J. de Kleer and B. C. Williams. Reasoning about Multiple Faults. In 5th National Conf. on Artificial Intelligence Volume 1: Science, pages 132-139, 1986. Google Scholar
  3. A. Feldman, T. Kurtoglu, S. Narasimhan, S. Poll, D. Garcia, J. de Kleer, L. Kuhn, and A. van Gemund. Empirical evaluation of diagnostic algorithm performance using a generic framework. International Journal of Prognostics and Health Management, pages 1-28, 2010. Google Scholar
  4. G. Friedrich, G. Gottlob, and W. Nejdl. Formalizing the repair process. In 10th European Conference on Artificial Intelligence, pages 709-713, 1992. Google Scholar
  5. A. Hou. A theory of measurement in diagnosis from first principles. Artificial Intelligence, 65(2):281-328, February 1994. URL: https://doi.org/10.1016/0004-3702(94)90019-1.
  6. J. Jones and M. Harrold. Empirical evaluation of the tarantula automatic fault-localization technique. In 20th IEEE/ACM International Conference on Automated Software Engineering, ASE 2005, pages 273-282, November 2005. URL: https://doi.org/10.1145/1101908.1101949.
  7. A. Lundgren and D. Jung. Data-driven fault diagnosis analysis and open-set classification of time-series data. Control Engineering Practice, 121:105006, 2022. Google Scholar
  8. I. Pill and J. de Kleer. Assessing Diagnosis Algorithms: Of Sampling, Baselines, Metrics and Oracles. In 36th Int. Conf. on Principles of Diagnosis and Resilient Systems (DX 2025), volume 136 of Open Access Series in Informatics (OASIcs), pages 5:1-5:19, 2025. URL: https://doi.org/10.4230/OASIcs.DX.2025.5.
  9. I. Pill and T. Quaritsch. RC-Tree: A variant avoiding all the redundancy in Reiter’s minimal hitting set algorithm. In IEEE Int. Symp. on Software Reliability Engineering Workshops (ISSREW), pages 78-84, 2015. Google Scholar
  10. R. Reiter. A Theory of Diagnosis from First Principles. Artificial Intelligence, 32(1):57-95, 1987. URL: https://doi.org/10.1016/0004-3702(87)90062-2.
  11. P. Rodler. Sequential model-based diagnosis by systematic search. Artificial Intelligence, 323:103988, 2023. URL: https://doi.org/10.1016/j.artint.2023.103988.
  12. J. Schleich and F. Wotawa. Combining Dynamic Slicing and Spectrum-Based Fault Localization - A First Experimental Evaluation. In 36th Int. Conf. on Principles of Diagnosis and Resilient Systems (DX), volume 136 of Open Access Series in Informatics (OASIcs), pages 3:1-3:18, 2025. URL: https://doi.org/10.4230/OASIcs.DX.2025.3.
  13. R. Stern and M. Kalech. Repair planning with batch repair. In 25th Int. Workshop on Principles of Diagnosis, 2014. URL: http://dx-2014.ist.tugraz.at/papers/DX14_Wed_AM_S1_paper4.pdf.
  14. R. Stern, M. Kalech, and H. Shinitzky. Implementing troubleshooting with batch repair. CEUR Workshop Proceedings, 1507:113-118, 2015. 26th Int. Workshop on Principles of Diagnosis. URL: https://ceur-ws.org/Vol-1507/dx15paper15.pdf.
  15. H. S. Steude, A. Diedrich, I. Pill, L. Moddemann, D. Vranješ, and O. Niggemann. Data-driven diagnosis for large cyber-physical-systems with minimal prior information. In 2026 IEEE Conference on Artificial Intelligence (CAI), pages 325-332, 2026. Google Scholar
  16. S. Thiébaux, M. Cordier, O. Jehl, and J. Krivine. Supply restoration in power distribution systems: a case study in integrating model-based diagnosis and repair planning. In 12th Int. Conf. on Uncertainty in Artificial Intelligence, pages 525-532, 1996. Google Scholar
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2026 Schloss Dagstuhl – LZI GmbH Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact