On Polynomial Time Constructions of Minimum Height Decision Tree

Authors Nader H. Bshouty, Waseem Makhoul

Thumbnail PDF


  • Filesize: 346 kB
  • 12 pages

Document Identifiers

Author Details

Nader H. Bshouty
  • Department of Computer Science, Technion, Haifa, Israel
Waseem Makhoul
  • Department of Computer Science, Technion, Haifa, Israel

Cite AsGet BibTex

Nader H. Bshouty and Waseem Makhoul. On Polynomial Time Constructions of Minimum Height Decision Tree. In 29th International Symposium on Algorithms and Computation (ISAAC 2018). Leibniz International Proceedings in Informatics (LIPIcs), Volume 123, pp. 34:1-34:12, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2018)


A decision tree T in B_m:={0,1}^m is a binary tree where each of its internal nodes is labeled with an integer in [m]={1,2,...,m}, each leaf is labeled with an assignment a in B_m and each internal node has two outgoing edges that are labeled with 0 and 1, respectively. Let A subset {0,1}^m. We say that T is a decision tree for A if (1) For every a in A there is one leaf of T that is labeled with a. (2) For every path from the root to a leaf with internal nodes labeled with i_1,i_2,...,i_k in[m], a leaf labeled with a in A and edges labeled with xi_{i_1},...,xi_{i_k}in {0,1}, a is the only element in A that satisfies a_{i_j}=xi_{i_j} for all j=1,...,k. Our goal is to write a polynomial time (in n:=|A| and m) algorithm that for an input A subseteq B_m outputs a decision tree for A of minimum depth. This problem has many applications that include, to name a few, computer vision, group testing, exact learning from membership queries and game theory. Arkin et al. and Moshkov [Esther M. Arkin et al., 1998; Mikhail Ju. Moshkov, 2004] gave a polynomial time (ln |A|)- approximation algorithm (for the depth). The result of Dinur and Steurer [Irit Dinur and David Steurer, 2014] for set cover implies that this problem cannot be approximated with ratio (1-o(1))* ln |A|, unless P=NP. Moshkov studied in [Mikhail Ju. Moshkov, 2004; Mikhail Ju. Moshkov, 1982; Mikhail Ju. Moshkov, 1982] the combinatorial measure of extended teaching dimension of A, ETD(A). He showed that ETD(A) is a lower bound for the depth of the decision tree for A and then gave an exponential time ETD(A)/log(ETD(A))-approximation algorithm and a polynomial time 2(ln 2)ETD(A)-approximation algorithm. In this paper we further study the ETD(A) measure and a new combinatorial measure, DEN(A), that we call the density of the set A. We show that DEN(A) <=ETD(A)+1. We then give two results. The first result is that the lower bound ETD(A) of Moshkov for the depth of the decision tree for A is greater than the bounds that are obtained by the classical technique used in the literature. The second result is a polynomial time (ln 2)DEN(A)-approximation (and therefore (ln 2)ETD(A)-approximation) algorithm for the depth of the decision tree of A. We then apply the above results to learning the class of disjunctions of predicates from membership queries [Nader H. Bshouty et al., 2017]. We show that the ETD of this class is bounded from above by the degree d of its Hasse diagram. We then show that Moshkov algorithm can be run in polynomial time and is (d/log d)-approximation algorithm. This gives optimal algorithms when the degree is constant. For example, learning axis parallel rays over constant dimension space.

Subject Classification

ACM Subject Classification
  • Mathematics of computing → Combinatorial optimization
  • Decision Tree
  • Minimal Depth
  • Approximation algorithms


  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    PDF Downloads


  1. Dana Angluin. Queries and Concept Learning. Machine Learning, 2(4):319-342, 1988. URL: http://dx.doi.org/10.1023/A:1022821128753.
  2. Martin Anthony, Graham R. Brightwell, David A. Cohen, and John Shawe-Taylor. On Exact Specification by Examples. In Proceedings of the Fifth Annual ACM Conference on Computational Learning Theory, COLT 1992, Pittsburgh, PA, USA, July 27-29, 1992., pages 311-318, 1992. URL: http://dx.doi.org/10.1145/130385.130420.
  3. Esther M. Arkin, Michael T. Goodrich, Joseph S. B. Mitchell, David M. Mount, Christine D. Piatko, and Steven Skiena. Point Probe Decision Trees for Geometric Concept Classes. In Algorithms and Data Structures, Third Workshop, WADS '93, Montréal, Canada, August 11-13, 1993, Proceedings, pages 95-106, 1993. URL: http://dx.doi.org/10.1007/3-540-57155-8_239.
  4. Esther M. Arkin, Henk Meijer, Joseph S. B. Mitchell, David Rappaport, and Steven Skiena. Decision trees for geometric models. Int. J. Comput. Geometry Appl., 8(3):343-364, 1998. URL: http://dx.doi.org/10.1142/S0218195998000175.
  5. Nader H. Bshouty, Dana Drachsler-Cohen, Martin T. Vechev, and Eran Yahav. Learning Disjunctions of Predicates. In Proceedings of the 30th Conference on Learning Theory, COLT 2017, Amsterdam, The Netherlands, 7-10 July 2017, pages 346-369, 2017. URL: http://proceedings.mlr.press/v65/bshouty17a.html.
  6. Nader H. Bshouty and Waseem Makhoul. On Polynomial time Constructions of Minimum Height Decision Tree. CoRR, abs/1802.00233, 2018. URL: http://arxiv.org/abs/1802.00233.
  7. Irit Dinur and David Steurer. Analytical approach to parallel repetition. In Symposium on Theory of Computing, STOC 2014, New York, NY, USA, May 31 - June 03, 2014, pages 624-633, 2014. URL: http://dx.doi.org/10.1145/2591796.2591884.
  8. M. R. Garey. Optimal Binary Identification Procedures. SIAM Journal on Applied Mathematics, 23(2):173-186, 1971. URL: http://epubs.siam.org/doi/abs/10.1137/0123019.
  9. Sally A. Goldman and Michael J. Kearns. On the Complexity of Teaching. J. Comput. Syst. Sci., 50(1):20-31, 1995. URL: http://dx.doi.org/10.1006/jcss.1995.1003.
  10. Tibor Hegedüs. Generalized Teaching Dimensions and the Query Complexity of Learning. In Proceedings of the Eigth Annual Conference on Computational Learning Theory, COLT 1995, Santa Cruz, California, USA, July 5-8, 1995, pages 108-117, 1995. URL: http://dx.doi.org/10.1145/225298.225311.
  11. Laurent Hyafil and Ronald L. Rivest. Constructing Optimal Binary Decision Trees is NP-Complete. Inf. Process. Lett., 5(1):15-17, 1976. URL: http://dx.doi.org/10.1016/0020-0190(76)90095-8.
  12. Eduardo Sany Laber and Loana Tito Nogueira. On the hardness of the minimum height decision tree problem. Discrete Applied Mathematics, 144(1-2):209-212, 2004. URL: http://dx.doi.org/10.1016/j.dam.2004.06.002.
  13. Mikhail Ju. Moshkov. On conditional tests. Problemy Kibernetiki. and Sov. Phys. Dokl., 27(7):528-530, 1982. Google Scholar
  14. Mikhail Ju. Moshkov. On conditional tests. Problems of Cybernetics, Nauka, Moscow, 40:131-170, 1982. Google Scholar
  15. Mikhail Ju. Moshkov. Greedy Algorithm of Decision Tree Construction for Real Data Tables. Transactions on Rough Sets I, Lecture Notes in Computer Science 3100, Springer-Verlag, Heidelberg., pages 161-168, 2004. URL: http://dx.doi.org/10.1007/978-3-540-27794-1_7.
  16. Ayumi Shinohara. Teachability in Computational Learning. New Generation Comput., 8(4):337-347, 1991. URL: http://dx.doi.org/10.1007/BF03037091.