Document Open Access Logo

Using SLMs and LLMs for Diagnosis

Authors Sophia Plank, Franz Wotawa

PDF

File

Thumbnail PDF
PDF
OASIcs.DX.2026.14.pdf
  • Filesize: 3.18 MB
  • 17 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Computing methodologies → Artificial intelligence
  • Computing methodologies → Machine learning
  • Computer systems organization → Maintainability and maintenance
Keywords
  • LLM diagnosis performance
  • SLM diagnosis performance
  • comparative study

Metrics

Abstract

Large Language Models (LLMs) have rapidly transformed how organizations approach complex problem solving, raising the question of whether they can also be applied to technical reasoning tasks. In this paper, we investigate whether LLMs can be used for system diagnosis, i.e., the localization of faults in systems comprising interacting components. In particular, we conducted an initial study using examples ranging from electric circuits to heating systems, evaluating 13 different language models of varying sizes with respect to the correctness and completeness of their diagnosis. The experimental evaluation revealed that language models with more parameters achieve superior performance.

Cite As Get BibTex

Sophia Plank and Franz Wotawa. Using SLMs and LLMs for Diagnosis. In 37th International Conference on Principles of Diagnosis and Resilient Systems (DX 2026). Open Access Series in Informatics (OASIcs), Volume 148, pp. 14:1-14:17, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026) https://doi.org/10.4230/OASIcs.DX.2026.14

Author Details

Sophia Plank
  • Graz University of Technology, Austria
Franz Wotawa
  • Institute of Software Engineering and Artificial Intelligence, Graz University of Technology, Austria

Funding

  • Wotawa, Franz: The research described in this paper has been partially carried out within the FFG project FO999923190 SAGE – Scalable Agents for Facility Management and Energy Efficiency funded by the Austrian Federal Ministry for Climate Action, Environment, Energy, Mobility, Innovation and Technology (BMK).

References

  1. Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Qin Cai, Vishrav Chaudhary, Dong Chen, Dongdong Chen, Weizhu Chen, Yen-Chun Chen, Yi-Ling Chen, Hao Cheng, Parul Chopra, Xiyang Dai, Matthew Dixon, Ronen Eldan, Victor Fragoso, Jianfeng Gao, Mei Gao, Min Gao, Amit Garg, Allie Del Giorno, Abhishek Goswami, Suriya Gunasekar, Emman Haider, Junheng Hao, Russell J. Hewett, Wenxiang Hu, Jamie Huynh, Dan Iter, Sam Ade Jacobs, Mojan Javaheripi, Xin Jin, Nikos Karampatziakis, Piero Kauffmann, Mahoud Khademi, Dongwoo Kim, Young Jin Kim, Lev Kurilenko, James R. Lee, Yin Tat Lee, Yuanzhi Li, Yunsheng Li, Chen Liang, Lars Liden, Xihui Lin, Zeqi Lin, Ce Liu, Liyuan Liu, Mengchen Liu, Weishung Liu, Xiaodong Liu, Chong Luo, Piyush Madan, Ali Mahmoudzadeh, David Majercak, Matt Mazzola, Caio César Teodoro Mendes, Arindam Mitra, Hardik Modi, Anh Nguyen, Brandon Norick, Barun Patra, Daniel Perez-Becker, Thomas Portet, Reid Pryzant, Heyang Qin, Marko Radmilac, Liliang Ren, Gustavo de Rosa, Corby Rosset, Sambudha Roy, Olatunji Ruwase, Olli Saarikivi, Amin Saied, Adil Salim, Michael Santacroce, Shital Shah, Ning Shang, Hiteshi Sharma, Yelong Shen, Swadheen Shukla, Xia Song, Masahiro Tanaka, Andrea Tupini, Praneetha Vaddamanu, Chunyu Wang, Guanhua Wang, Lijuan Wang, Shuohang Wang, Xin Wang, Yu Wang, Rachel Ward, Wen Wen, Philipp Witte, Haiping Wu, Xiaoxia Wu, Michael Wyatt, Bin Xiao, Can Xu, Jiahang Xu, Weijian Xu, Jilong Xue, Sonali Yadav, Fan Yang, Jianwei Yang, Yifan Yang, Ziyi Yang, Donghan Yu, Lu Yuan, Chenruidong Zhang, Cyril Zhang, Jianwen Zhang, Li Lyna Zhang, Yi Zhang, Yue Zhang, Yunan Zhang, and Xiren Zhou. Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone, August 2024. arXiv:2404.14219 [cs.CL]. URL: https://doi.org/10.48550/arXiv.2404.14219.
  2. Deepak Bhaskar Acharya, Karthigeyan Kuppan, and B. Divya. Agentic ai: Autonomous intelligence for complex goals—a comprehensive survey. IEEE Access, 13:18912-18936, 2025. URL: https://doi.org/10.1109/ACCESS.2025.3532853.
  3. Milton Friedman. The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American Statistical Association, 32:675-701, 1937. Google Scholar
  4. Gerhard Friedrich, Georg Gottlob, and Wolfgang Nejdl. Physical impossibility instead of fault models. In Proceedings of the National Conference on Artificial Intelligence (AAAI), pages 331-336, Boston, August 1990. Also appears in Readings in Model-Based Diagnosis (Morgan Kaufmann, 1992). URL: http://www.aaai.org/Library/AAAI/1990/aaai90-051.php.
  5. Soodeh Hosseini and Hossein Seilani. The role of agentic ai in shaping a smart future: A systematic review. Array, 26:100399, 2025. URL: https://doi.org/10.1016/j.array.2025.100399.
  6. Sarah Lukens, Matthew Bishof, Nadir Siddiqui, and Destiny West. An Evaluation Framework for Fault Diagnosis Using Technical Manuals in Retrieval-Augmented Large Language Models. Annual Conference of the PHM Society, 17(1), October 2025. URL: https://doi.org/10.36001/phmconf.2025.v17i1.4549.
  7. Md Mahade Hasan, Muhammad Waseem, Kai-Kristian Kemell, Jussi Rasku, Juha Ala-Rantala, and Pekka Abrahamsson. Assessing small language models for code generation: An empirical study with benchmarks. Journal of Systems and Software, 236:112815, June 2026. URL: https://doi.org/10.1016/j.jss.2026.112815.
  8. Hamzah A. A. M. Qaid, Bo Zhang, Dan Li, See-Kiong Ng, and Wei Li. FD-LLM: Large Language Model for Fault Diagnosis of Machines, December 2024. arXiv:2412.01218 [cs.AI]. URL: https://doi.org/10.48550/arXiv.2412.01218.
  9. Raymond Reiter. A theory of diagnosis from first principles. Artificial Intelligence, 32(1):57-95, April 1987. URL: https://doi.org/10.1016/0004-3702(87)90062-2.
  10. Peter Struss and Oskar Dressler. Physical negation - Integrating fault models into the general diagnostic engine. In Proceedings 11^th International Joint Conf. on Artificial Intelligence, pages 1318-1323, Detroit, August 1989. URL: http://ijcai.org/Proceedings/89-2/Papers/075.pdf.
  11. Anna Sztyber-Betley, Elodie Chanthery, Louise Travé-Massuyès, Silke Merkelbach, Karol Kukla, Maxence Glotin, Alexander Diedrich, and Oliver Niggemann. Are Diagnostic Concepts Within the Reach of LLMs? In Marcos Quinones-Grueiro, Gautam Biswas, and Ingo Pill, editors, 36th International Conference on Principles of Diagnosis and Resilient Systems (DX 2025), volume 136 of Open Access Series in Informatics (OASIcs), pages 2:1-2:20, Dagstuhl, Germany, 2025. Schloss Dagstuhl - Leibniz-Zentrum für Informatik. URL: https://doi.org/10.4230/OASIcs.DX.2025.2.
  12. Huanting Wang, Jingzhi Gong, Huawei Zhang, Jie Xu, and Zheng Wang. Ai agentic programming: A survey of techniques, challenges, and opportunities, 2025. URL: https://doi.org/10.48550/arXiv.2508.11126.
  13. Franz Wotawa. Model-Based Diagnosis—European Perspectives and Contributions. The European Journal on Artificial Intelligence, 39(3):557-576, 2026. URL: https://doi.org/10.1177/30504554251403464.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2026 Schloss Dagstuhl – LZI GmbH Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact