SAT-Based Learning of Compact Binary Decision Diagrams for Classification

Authors Pouya Shati, Eldan Cohen, Sheila McIlraith

Thumbnail PDF


  • Filesize: 1.54 MB
  • 19 pages

Document Identifiers

Author Details

Pouya Shati
  • Department of Computer Science, University of Toronto, Canada
  • Vector Institute, Toronto, Canada
Eldan Cohen
  • Department of Mechanical and Industrial Engineering, University of Toronto, Canada
Sheila McIlraith
  • Department of Computer Science, University of Toronto, Canada
  • Vector Institute, Toronto, Canada

Cite AsGet BibTex

Pouya Shati, Eldan Cohen, and Sheila McIlraith. SAT-Based Learning of Compact Binary Decision Diagrams for Classification. In 29th International Conference on Principles and Practice of Constraint Programming (CP 2023). Leibniz International Proceedings in Informatics (LIPIcs), Volume 280, pp. 33:1-33:19, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2023)


Decision trees are a popular classification model in machine learning due to their interpretability and performance. However, the number of splits in decision trees grow exponentially with their depth which can incur a higher computational cost, increase data fragmentation, hinder interpretability, and restrict their applicability to memory-constrained hardware. In constrast, binary decision diagrams (BDD) utilize the same split across each level, leading to a linear number of splits in total. Recent work has considered optimal binary decision diagrams (BDD) as compact and accurate classification models, but has only focused on binary datasets and has not explicitly optimized the compactness of the resulting diagrams. In this work, we present a SAT-based encoding for a multi-terminal variant of BDDs (MTBDDs) that incorporates a state-of-the-art direct encoding of numerical features. We then develop and evaluate different approaches to explicitly optimize the compactness of the diagrams. In one family of approaches, we learn a tree BDD first and model the size of the diagram the tree will be reduced to as a secondary objective, in a one-stage or two-stage optimization scheme. Alternatively, we directly learn diagrams that support multi-dimensional splits for improved expressiveness. Our experiments show that direct encoding of numerical features leads to better performance. Furthermore, we show that exact optimization of size leads to more compact solutions while maintaining higher accuracy. Finally, our experiments show that multi-dimensional splits are a viable approach to achieving higher expressiveness with a lower computational cost.

Subject Classification

ACM Subject Classification
  • Mathematics of computing → Combinatorial optimization
  • Computing methodologies → Machine learning
  • Binary Decision Diagram
  • Classification
  • Compactness
  • Numeric Data
  • MaxSAT


  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    PDF Downloads


  1. Gaël Aglin, Siegfried Nijssen, and Pierre Schaus. Learning optimal decision trees using caching branch-and-bound search. In AAAI Conference on Artificial Intelligence (AAAI), pages 3146-3153, 2020. Google Scholar
  2. Sheldon B. Akers. Binary decision diagrams. IEEE Transactions on computers, 27(06):509-516, 1978. Google Scholar
  3. Florent Avellaneda. Efficient inference of optimal decision trees. In AAAI Conference on Artificial Intelligence (AAAI), pages 3195-3202, 2020. Google Scholar
  4. Jeremias Berg, Emir Demirović, and Peter J Stuckey. Core-boosted linear search for incomplete MaxSAT. In International Conference on Integration of Constraint Programming, Artificial Intelligence, and Operations Research (CPAIOR), pages 39-56. Springer, 2019. Google Scholar
  5. Dimitris Bertsimas and Jack Dunn. Optimal classification trees. Machine Learning, 106(7):1039-1082, 2017. Google Scholar
  6. Allan Borodin, Alexander Razborov, and Roman Smolensky. On lower bounds for read-k-times branching programs. Computational Complexity, 3:1-18, 1993. Google Scholar
  7. Leo Breiman, Jerome H Friedman, Richard A Olshen, and Charles J Stone. Classification and regression trees. Wadsworth & Brooks/Cole Advanced Books & Software, 1984. Google Scholar
  8. Randal E Bryant. Graph-based algorithms for boolean function manipulation. Computers, IEEE Transactions on, 100(8):677-691, 1986. Google Scholar
  9. Gianpiero Cabodi, Paolo E Camurati, Alexey Ignatiev, Joao Marques-Silva, Marco Palena, and Paolo Pasini. Optimizing binary decision diagrams for interpretable machine learning classification. In 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE), pages 1122-1125. IEEE, 2021. Google Scholar
  10. Diogo V Carvalho, Eduardo M Pereira, and Jaime S Cardoso. Machine learning interpretability: A survey on methods and metrics. Electronics, 8(8):832, 2019. Google Scholar
  11. Edmund M Clarke, Masahiro Fujita, and Xudong Zhao. Multi-terminal binary decision diagrams and hybrid decision diagrams. Representations of discrete functions, pages 93-108, 1996. Google Scholar
  12. Adnan Darwiche. Sdd: A new canonical representation of propositional knowledge bases. In Twenty-Second International Joint Conference on Artificial Intelligence, 2011. Google Scholar
  13. Dheeru Dua and Casey Graff. UCI machine learning repository, 2017. URL:
  14. Alexandre M Florio, Pedro Martins, Maximilian Schiffer, Thiago Serra, and Thibaut Vidal. Optimal decision diagrams for classification. arXiv preprint arXiv:2205.14500, 2022. Google Scholar
  15. Oktay Günlük, Jayant Kalagnanam, Minhan Li, Matt Menickelly, and Katya Scheinberg. Optimal decision trees for categorical data via integer programming. Journal of Global Optimization, pages 1-28, 2021. Google Scholar
  16. Hao Hu, Marie-José Huguet, and Mohamed Siala. Optimizing binary decision diagrams with maxsat for classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36(4), pages 3767-3775, 2022. Google Scholar
  17. Hao Hu, Marie-José Huguet, and Mohamed Siala. Optimizing binary decision diagrams with maxsat for classification, 2022. URL:
  18. Hao Hu, Mohamed Siala, Emmanuel Hébrard, and Marie-José Huguet. Learning optimal decision trees with MaxSAT and its integration in AdaBoost. In International Joint Conference on Artificial Intelligence and Pacific Rim International Conference on Artificial Intelligence (IJCAI-PRICAI), 2020. Google Scholar
  19. Alexey Ignatiev and Joao Marques-Silva. SAT-based rigorous explanations for decision lists. In Theory and Applications of Satisfiability Testing-SAT 2021: 24th International Conference, Barcelona, Spain, July 5-9, 2021, Proceedings 24, pages 251-269. Springer, 2021. Google Scholar
  20. Alexey Ignatiev, Filipe Pereira, Nina Narodytska, and Joao Marques-Silva. A SAT-based approach to learn explainable decision sets. In Automated Reasoning: 9th International Joint Conference, IJCAR 2018, Held as Part of the Federated Logic Conference, FloC 2018, Oxford, UK, July 14-17, 2018, Proceedings 9, pages 627-645. Springer, 2018. Google Scholar
  21. Dmitry Ignatov and Andrey Ignatov. Decision stream: Cultivating deep decision trees. In 2017 ieee 29th international conference on tools with artificial intelligence (ictai), pages 905-912. IEEE, 2017. Google Scholar
  22. Stasys Jukna. A note on read-times branching programs. RAIRO-Theoretical Informatics and Applications, 29(1):75-83, 1995. Google Scholar
  23. Donald E Knuth. The Art of Computer Programming, Volume 4, Fascicle 1: Bitwise Tricks & Techniques; Binary Decision Diagrams. Addison-Wesley Professional, 2009. Google Scholar
  24. Ron Kohavi. Bottom-up induction of oblivious read-once decision graphs. In Machine Learning: ECML-94: European Conference on Machine Learning Catania, Italy, April 6-8, 1994 Proceedings 7, pages 154-169. Springer, 1994. Google Scholar
  25. Benjamin Letham, Cynthia Rudin, Tyler H McCormick, and David Madigan. Interpretable classifiers using rules and bayesian analysis: Building a better stroke prediction model, 2015. Google Scholar
  26. Breiman LI, Jerome Friedman, RA Olshen, and C.J. Stone. Classification and regression trees (CART). Biometrics, 40:358, September 1984. URL:
  27. Joao Marques-Silva and Alexey Ignatiev. Delivering trustworthy AI through formal XAI. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36(11), pages 12342-12350, 2022. Google Scholar
  28. Bernard ME Moret. Decision trees and diagrams. ACM Computing Surveys (CSUR), 14(4):593-623, 1982. Google Scholar
  29. J Ross Quinlan. Induction of decision trees. Machine learning, 1(1):81-106, 1986. Google Scholar
  30. J Ross Quinlan. C4. 5: programs for machine learning. Elsevier, 2014. Google Scholar
  31. Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. nat mach intell 1: 206-215. DOI: https://doi. org/10.1038/s42256-019-0048-x, 2019. Google Scholar
  32. Petr Savickỳ and Ingo Wegener. Efficient algorithms for the transformation between different types of binary decision diagrams. Acta Informatica, 34(4):245-256, 1997. Google Scholar
  33. Pouya Shati, Eldan Cohen, and Sheila McIlraith. SAT-based approach for learning optimal decision trees with non-binary features. In 27th International Conference on Principles and Practice of Constraint Programming (CP 2021). Schloss Dagstuhl-Leibniz-Zentrum für Informatik, 2021. Google Scholar
  34. Pouya Shati, Eldan Cohen, and Sheila A. McIlraith. SAT-based optimal classification trees for non-binary data. Constraints, July 2023. URL:
  35. Jamie Shotton, Toby Sharp, Pushmeet Kohli, Sebastian Nowozin, John Winn, and Antonio Criminisi. Decision jungles: Compact and rich models for classification. Advances in neural information processing systems, 26, 2013. Google Scholar
  36. Hazem Torfah, Shetal Shah, Supratik Chakraborty, S Akshay, and Sanjit A Seshia. Synthesizing pareto-optimal interpretations for black-box models. In 2021 Formal Methods in Computer Aided Design (FMCAD), pages 153-162. IEEE, 2021. Google Scholar
  37. Hélene Verhaeghe, Siegfried Nijssen, Gilles Pesant, Claude-Guy Quimper, and Pierre Schaus. Learning optimal decision trees using constraint programming. Constraints, 25(3):226-250, 2020. Google Scholar
  38. Yu Zhang, Peter Tiňo, Aleš Leonardis, and Ke Tang. A survey on neural network interpretability. IEEE Transactions on Emerging Topics in Computational Intelligence, 5(5):726-742, 2021. Google Scholar
Questions / Remarks / Feedback

Feedback for Dagstuhl Publishing

Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail