Document Open Access Logo

Efficient Explanations for Rule Ensembles

Authors Hao Hu , Alexey Ignatiev , Joao Marques-Silva

PDF

File

Thumbnail PDF
PDF
LIPIcs.CP.2026.29.pdf
  • Filesize: 1.13 MB
  • 20 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Theory of computation → Constraint and logic programming
  • Computing methodologies → Artificial intelligence
Keywords
  • Explainable Artifical Intelligence
  • Rule Ensembles
  • Boolean Satisfiability

Metrics

Abstract

Tree ensembles (TEs) are among the most widely used machine learning models, yet explaining their predictions remains a computational challenge. Recent work in formal explainable AI (FXAI) has focused on computing abductive explanations for TEs using Boolean satisfiability (SAT) and maximum satisfiability (MaxSAT). However, these methods often fail to scale with the growth of the number and depth of the trees in the ensemble. This paper addresses these scalability limitations by shifting focus to Rule Ensembles (REs), a structurally simpler alternative to TEs. We make three primary contributions. First, we adapt an existing MaxSAT-based explanation framework designed for TEs to function with general REs. Second, we devise a dedicated logic encoding for REs combining SAT solving with pseudo-Boolean (PB) constraints for determining the winning class. Finally, empirical experiments on standard tabular and image datasets demonstrate a significant advantage of the proposed SAT-based approach for REs over the state-of-the-art MaxSAT-based approach for TEs.

Cite As Get BibTex

Hao Hu, Alexey Ignatiev, and Joao Marques-Silva. Efficient Explanations for Rule Ensembles. In 32nd International Conference on Principles and Practice of Constraint Programming (CP 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 379, pp. 29:1-29:20, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026) https://doi.org/10.4230/LIPIcs.CP.2026.29

Author Details

Hao Hu
  • University of Lleida, Lleida, Spain
Alexey Ignatiev
  • Monash University, Melbourne, Australia
Joao Marques-Silva
  • ICREA, University of Lleida, Lleida, Spain

Funding

This work is supported by Spanish Government the grant PID2023-152814OB-I00.

Acknowledgements

The authors express their sincere thanks to all reviewers for their supportive questions and suggestions.

Supplementary Materials

References

  1. Ignasi Abío, Robert Nieuwenhuis, Albert Oliveras, Enric Rodríguez-Carbonell, and Valentin Mayer-Eichberger. A new look at bdds for pseudo-boolean constraints. J. Artif. Intell. Res., 45:443-480, 2012. URL: https://doi.org/10.1613/JAIR.3653.
  2. Gilles Audemard, Steve Bellart, Jean-Marie Lagniez, and Pierre Marquis. Computing abductive explanations for boosted regression trees. In IJCAI, pages 3432-3441. ijcai.org, 2023. URL: https://doi.org/10.24963/IJCAI.2023/382.
  3. Gilles Audemard, Jean-Marie Lagniez, Pierre Marquis, and Nicolas Szczepanski. Computing abductive explanations for boosted trees. In AISTATS, Proceedings of Machine Learning Research, pages 4699-4711. PMLR, 2023. URL: https://proceedings.mlr.press/v206/audemard23a.html.
  4. Gilles Audemard, Jean-Marie Lagniez, Pierre Marquis, and Nicolas Szczepanski. PyXAI: An XAI library for tree-based models. In IJCAI 2024, Jeju, South Korea, August 3-9, 2024, pages 8601-8605, 2024. URL: https://www.ijcai.org/proceedings/2024/989.
  5. Gilles Audemard, Jean-Marie Lagniez, and Laurent Simon. Improving glucose for incremental SAT solving with assumptions: Application to MUS extraction. In SAT, Lecture Notes in Computer Science, pages 309-317. Springer, 2013. URL: https://doi.org/10.1007/978-3-642-39071-5_23.
  6. Fahiem Bacchus, Matti Järvisalo, and Ruben Martins. Maximum satisfiability. In Handbook of Satisfiability, Frontiers in Artificial Intelligence and Applications, pages 929-991. IOS Press, 2021. URL: https://doi.org/10.3233/FAIA201008.
  7. Olivier Bailleux, Yacine Boufkhad, and Olivier Roussel. New encodings of pseudo-boolean constraints into CNF. In SAT, Lecture Notes in Computer Science, pages 181-194. Springer, 2009. URL: https://doi.org/10.1007/978-3-642-02777-2_19.
  8. Armin Biere, Marijn Heule, Hans van Maaren, and Toby Walsh, editors. Handbook of Satisfiability - Second Edition, volume 336 of Frontiers in Artificial Intelligence and Applications. IOS Press, 2021. URL: https://doi.org/10.3233/FAIA336.
  9. Leo Breiman. Bagging predictors. Mach. Learn., 24(2):123-140, 1996. URL: https://doi.org/10.1007/BF00058655.
  10. Leo Breiman. Random forests. Mach. Learn., 45(1):5-32, 2001. URL: https://doi.org/10.1023/A:1010933404324.
  11. Leo Breiman, J. H. Friedman, Richard A. Olshen, and C. J. Stone. Classification and Regression Trees. Wadsworth, 1984. Google Scholar
  12. Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In KDD, pages 785-794. ACM, 2016. URL: https://doi.org/10.1145/2939672.2939785.
  13. William W. Cohen. Fast effective rule induction. In ICML, pages 115-123. Morgan Kaufmann, 1995. URL: https://doi.org/10.1016/B978-1-55860-377-6.50023-2.
  14. William W. Cohen and Yoram Singer. A simple, fast, and effictive rule learner. In AAAI/IAAI, pages 335-342. AAAI Press / The MIT Press, 1999. URL: http://www.aaai.org/Library/AAAI/1999/aaai99-049.php.
  15. Adnan Darwiche. Logic for explainable AI. In LICS, pages 1-11, 2023. URL: https://doi.org/10.1109/LICS56636.2023.10175757.
  16. Niklas Eén and Niklas Sörensson. Temporal induction by incremental SAT solving. In BMC@CAV, number 4 in Electronic Notes in Theoretical Computer Science, pages 543-560. Elsevier, 2003. URL: https://doi.org/10.1016/S1571-0661(05)82542-3.
  17. Yoav Freund. Boosting a weak learning algorithm by majority. Inf. Comput., 121(2):256-285, 1995. URL: https://doi.org/10.1006/INCO.1995.1136.
  18. Jerome H. Friedman. Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29:1189-1232, 2000. Google Scholar
  19. Fred Hemery, Christophe Lecoutre, Lakhdar Sais, and Frédéric Boussemart. Extracting MUCs from constraint networks. In ECAI, pages 113-117, 2006. URL: http://www.booksonline.iospress.nl/Content/View.aspx?piid=1657.
  20. Hao Hu, Marie-José Huguet, and Mohamed Siala. Optimizing binary decision diagrams with maxsat for classification. In AAAI, pages 3767-3775. AAAI Press, 2022. URL: https://doi.org/10.1609/AAAI.V36I4.20291.
  21. Hao Hu, Mohamed Siala, Emmanuel Hebrard, and Marie-José Huguet. Learning optimal decision trees with maxsat and its integration in adaboost. In IJCAI, pages 1170-1176. ijcai.org, 2020. URL: https://doi.org/10.24963/IJCAI.2020/163.
  22. Xuanxiang Huang, Yacine Izza, Alexey Ignatiev, Martin C. Cooper, Nicholas Asher, and João Marques-Silva. Tractable explanations for d-dnnf classifiers. In AAAI, pages 5719-5728. AAAI Press, 2022. URL: https://doi.org/10.1609/AAAI.V36I5.20514.
  23. Xuanxiang Huang, Yacine Izza, Alexey Ignatiev, and João Marques-Silva. On efficiently explaining graph-based classifiers. In KR, pages 356-367, 2021. URL: https://doi.org/10.24963/KR.2021/34.
  24. Xuanxiang Huang, Yacine Izza, Alexey Ignatiev, and Joao Marques-Silva. Uncovering bugs in formal explainers: A case study with PyXAI. CoRR, abs/2511.03169, 2025. URL: https://doi.org/10.48550/arXiv.2511.03169.
  25. Xuanxiang Huang and João Marques-Silva. On the failings of Shapley values for explainability. Int. J. Approx. Reason., 171:109112, 2024. URL: https://doi.org/10.1016/J.IJAR.2023.109112.
  26. Alexey Ignatiev. Towards trustable explainable AI. In IJCAI, pages 5154-5158, 2020. URL: https://doi.org/10.24963/IJCAI.2020/726.
  27. Alexey Ignatiev, Yacine Izza, Peter J. Stuckey, and João Marques-Silva. Using maxsat for efficient explanations of tree ensembles. In AAAI, pages 3776-3785. AAAI Press, 2022. URL: https://doi.org/10.1609/AAAI.V36I4.20292.
  28. Alexey Ignatiev, António Morgado, and João Marques-Silva. Pysat: A python toolkit for prototyping with SAT oracles. In SAT, Lecture Notes in Computer Science, pages 428-437. Springer, 2018. URL: https://doi.org/10.1007/978-3-319-94144-8_26.
  29. Alexey Ignatiev, António Morgado, and João Marques-Silva. RC2: an efficient maxsat solver. J. Satisf. Boolean Model. Comput., 11(1):53-64, 2019. URL: https://doi.org/10.3233/SAT190116.
  30. Alexey Ignatiev, Nina Narodytska, and João Marques-Silva. Abduction-based explanations for machine learning models. In AAAI, pages 1511-1519. AAAI Press, 2019. URL: https://doi.org/10.1609/AAAI.V33I01.33011511.
  31. Alexey Ignatiev, Zi Li Tan, and Christos Karamanos. Towards universally accessible SAT technology. In SAT, volume 305 of LIPIcs, pages 16:1-16:11. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2024. URL: https://doi.org/10.4230/LIPIcs.SAT.2024.16.
  32. Yacine Izza, Xuanxiang Huang, António Morgado, Jordi Planes, Alexey Ignatiev, and João Marques-Silva. Distance-restricted explanations: Theoretical underpinnings & efficient implementation. In Pierre Marquis, Magdalena Ortiz, and Maurice Pagnucco, editors, KR, 2024. URL: https://doi.org/10.24963/KR.2024/45.
  33. Yacine Izza, Alexey Ignatiev, and João Marques-Silva. On explaining decision trees. CoRR, abs/2010.11034, 2020. URL: https://arxiv.org/abs/2010.11034.
  34. Yacine Izza, Alexey Ignatiev, and João Marques-Silva. On tackling explanation redundancy in decision trees. J. Artif. Intell. Res., 75:261-321, 2022. URL: https://doi.org/10.1613/JAIR.1.13575.
  35. Yacine Izza and João Marques-Silva. On explaining random forests with SAT. In IJCAI, pages 2584-2591. ijcai.org, 2021. URL: https://doi.org/10.24963/IJCAI.2021/356.
  36. Ulrich Junker. QUICKXPLAIN: preferred explanations and relaxations for over-constrained problems. In AAAI, pages 167-172, 2004. URL: http://www.aaai.org/Library/AAAI/2004/aaai04-027.php.
  37. Markelle Kelly, Rachel Longjohn, and Kolby Nottingham. UCI machine learning repository. http://archive.ics.uci.edu/ml, 2013. Accessed: 2021-08-01.
  38. Trang T. Le, William G. La Cava, Joseph D. Romano, John T. Gregg, Daniel J. Goldberg, Praneel Chakraborty, Natasha L. Ray, Daniel S. Himmelstein, Weixuan Fu, and Jason H. Moore. PMLB v1.0: an open source dataset collection for benchmarking machine learning methods. CoRR, abs/2012.00058, 2020. URL: https://arxiv.org/abs/2012.00058.
  39. Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998. URL: https://doi.org/10.1109/5.726791.
  40. Chu Min Li and Felip Manyà. Maxsat, hard and soft constraints. In Handbook of Satisfiability, Frontiers in Artificial Intelligence and Applications, pages 903-927. IOS Press, 2021. URL: https://doi.org/10.3233/FAIA201007.
  41. Zachary C. Lipton. The mythos of model interpretability. Commun. ACM, 61(10):36-43, 2018. URL: https://doi.org/10.1145/3233231.
  42. João Marques-Silva. Logic-based explainability in machine learning. In Reasoning Web, pages 24-104, 2022. URL: https://doi.org/10.1007/978-3-031-31414-8_2.
  43. João Marques-Silva and Alexey Ignatiev. Delivering trustworthy AI through formal XAI. In AAAI, pages 12342-12350, 2022. URL: https://doi.org/10.1609/AAAI.V36I11.21499.
  44. João Marques-Silva, Mikolás Janota, and Anton Belov. Minimal sets over monotone predicates in boolean formulae. In CAV, pages 592-607, 2013. URL: https://doi.org/10.1007/978-3-642-39799-8_39.
  45. João Marques-Silva, Inês Lynce, and Sharad Malik. Conflict-driven clause learning SAT solvers. In Handbook of Satisfiability, Frontiers in Artificial Intelligence and Applications, pages 133-182. IOS Press, 2021. URL: https://doi.org/10.3233/FAIA200987.
  46. Hayden McTavish, Zachery Boner, Jon Donnelly, Margo I. Seltzer, and Cynthia Rudin. Leveraging predictive equivalence in decision trees. In ICML, 2025. URL: https://proceedings.mlr.press/v267/mctavish25a.html.
  47. Tobias Paxian, Sven Reimer, and Bernd Becker. Dynamic polynomial watchdog encoding for solving weighted maxsat. In SAT, Lecture Notes in Computer Science, pages 37-53. Springer, 2018. URL: https://doi.org/10.1007/978-3-319-94144-8_3.
  48. Tobias Philipp and Peter Steinke. Pblib - A library for encoding pseudo-boolean constraints into CNF. In SAT, Lecture Notes in Computer Science, pages 9-16. Springer, 2015. URL: https://doi.org/10.1007/978-3-319-24318-4_2.
  49. Michael Rapp. BOOMER - an algorithm for learning gradient boosted multi-label classification rules. Softw. Impacts, 10:100137, 2021. URL: https://doi.org/10.1016/J.SIMPA.2021.100137.
  50. Michael Rapp, Eneldo Loza Mencía, Johannes Fürnkranz, Vu-Linh Nguyen, and Eyke Hüllermeier. Learning gradient boosted multi-label classification rules. In ECML/PKDD (3), Lecture Notes in Computer Science, pages 124-140. Springer, 2020. URL: https://doi.org/10.1007/978-3-030-67664-3_8.
  51. Olivier Roussel and Vasco Manquinho. Pseudo-boolean and cardinality constraints. In Handbook of Satisfiability, Frontiers in Artificial Intelligence and Applications, pages 1087-1129. IOS Press, 2021. URL: https://doi.org/10.3233/FAIA201012.
  52. Andy Shih, Arthur Choi, and Adnan Darwiche. A symbolic approach to explaining bayesian network classifiers. In IJCAI, pages 5103-5111. ijcai.org, 2018. URL: https://doi.org/10.24963/IJCAI.2018/708.
  53. Jinqiang Yu, Alexey Ignatiev, Peter J. Stuckey, Nina Narodytska, and Joao Marques-Silva. Eliminating the impossible, whatever remains must be true: On extracting and applying background knowledge in the context of formal explanations. In AAAI, 2023. Google Scholar
  54. Zhi-Hua Zhou. Ensemble Methods: Foundations and Algorithms. Chapman & Hall/CRC, 1st edition, 2012. Google Scholar
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2026 Schloss Dagstuhl – LZI GmbH Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact