Document Open Access Logo

Improved Sample Complexity Bounds for Branch-And-Cut

Authors Maria-Florina Balcan, Siddharth Prasad, Tuomas Sandholm, Ellen Vitercik

PDF
Thumbnail PDF

File

LIPIcs.CP.2022.3.pdf
  • Filesize: 1.08 MB
  • 19 pages

Document Identifiers

Author Details

Maria-Florina Balcan
  • Computer Science and Machine Learning Departments, Carnegie Mellon University, Pittsburgh, PA, USA
Siddharth Prasad
  • Computer Science Department, Carnegie Mellon University, Pittsburgh, PA, USA
Tuomas Sandholm
  • Computer Science Department, Carnegie Mellon University, Pittsburgh, PA, USA
  • Optimized Markets, Inc., Pittsburgh, PA, USA
  • Strategic Machine, Inc., Pittsburgh, PA, USA
  • Strategy Robot, Inc., Pittsburgh, PA, USA
Ellen Vitercik
  • Department of Electrical Engineering and Computer Sciences, University of California Berkeley, CA, USA

Funding

This material is based on work supported by the National Science Foundation under grants CCF-1733556, CCF-1910321, IIS-1901403, and SES-1919453, the ARO under award W911NF2010081, the Defense Advanced Research Projects Agency under cooperative agreement HR00112020003, a Simons Investigator Award, an AWS Machine Learning Research Award, an Amazon Research Award, a Bloomberg Research Grant, and a Microsoft Research Faculty Fellowship.

Cite AsGet BibTex

Maria-Florina Balcan, Siddharth Prasad, Tuomas Sandholm, and Ellen Vitercik. Improved Sample Complexity Bounds for Branch-And-Cut. In 28th International Conference on Principles and Practice of Constraint Programming (CP 2022). Leibniz International Proceedings in Informatics (LIPIcs), Volume 235, pp. 3:1-3:19, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2022)
https://doi.org/10.4230/LIPIcs.CP.2022.3

Abstract

The branch-and-cut algorithm for integer programming has a wide variety of tunable parameters that have a huge impact on its performance, but which are challenging to tune by hand. An increasingly popular approach is to use machine learning to configure these parameters based on a training set of integer programs from the application domain. We bound how large the training set should be to ensure that for any configuration, its average performance over the training set is close to its expected future performance. Our guarantees apply to parameters that control the most important aspects of branch-and-cut: node selection, branching constraint selection, and cut selection, and are sharper and more general than those from prior research.

Subject Classification

ACM Subject Classification
  • Theory of computation → Integer programming
  • Theory of computation → Sample complexity and generalization bounds
Keywords
  • Automated algorithm configuration
  • integer programming
  • machine learning theory
  • tree search
  • branch-and-bound
  • branch-and-cut
  • cutting planes
  • sample complexity
  • generalization guarantees
  • data-driven algorithm design

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

References

  1. Tobias Achterberg. Constraint Integer Programming. PhD thesis, Technische Universität Berlin, 2007. Google Scholar
  2. Alejandro Marcos Alvarez, Quentin Louveaux, and Louis Wehenkel. A machine learning-based approximation of strong branching. INFORMS Journal on Computing, 29(1):185-195, 2017. Google Scholar
  3. Martin Anthony and Peter Bartlett. Neural Network Learning: Theoretical Foundations. Cambridge University Press, 2009. Google Scholar
  4. Egon Balas, Sebastián Ceria, and Gérard Cornuéjols. Mixed 0-1 programming by lift-and-project in a branch-and-cut framework. Management Science, 42(9):1229-1246, 1996. Google Scholar
  5. Maria-Florina Balcan. Data-driven algorithm design. In Tim Roughgarden, editor, Beyond Worst Case Analysis of Algorithms. Cambridge University Press, 2020. Google Scholar
  6. Maria-Florina Balcan, Dan DeBlasio, Travis Dick, Carl Kingsford, Tuomas Sandholm, and Ellen Vitercik. How much data is sufficient to learn high-performing algorithms? Generalization guarantees for data-driven algorithm design. In Annual Symposium on Theory of Computing (STOC), 2021. Google Scholar
  7. Maria-Florina Balcan, Travis Dick, Tuomas Sandholm, and Ellen Vitercik. Learning to branch. In International Conference on Machine Learning (ICML), 2018. Google Scholar
  8. Maria-Florina Balcan, Vaishnavh Nagarajan, Ellen Vitercik, and Colin White. Learning-theoretic foundations of algorithm configuration for combinatorial partitioning problems. In Conference on Learning Theory (COLT), 2017. Google Scholar
  9. Maria-Florina Balcan, Siddharth Prasad, Tuomas Sandholm, and Ellen Vitercik. Sample complexity of tree search configuration: Cutting planes and beyond. In Annual Conference on Neural Information Processing Systems (NeurIPS), 2021. Google Scholar
  10. Antonia Chmiela, Elias B Khalil, Ambros Gleixner, Andrea Lodi, and Sebastian Pokutta. Learning to schedule heuristics in branch-and-bound. In Annual Conference on Neural Information Processing Systems (NeurIPS), 2021. Google Scholar
  11. Vasek Chvátal. Hard knapsack problems. Operations Research, 28(6):1402-1411, 1980. Google Scholar
  12. Michele Conforti, Gérard Cornuéjols, Giacomo Zambelli, et al. Integer programming, volume 271. Springer, 2014. Google Scholar
  13. Giovanni Di Liberto, Serdar Kadioglu, Kevin Leo, and Yuri Malitsky. Dash: Dynamic approach for switching heuristics. European Journal of Operational Research, 248(3):943-953, 2016. Google Scholar
  14. Matteo Fischetti and Andrea Lodi. Local branching. Mathematical Programming, 98:23-47, 2002. Google Scholar
  15. Gerald Gamrath, Daniel Anderson, Ksenia Bestuzheva, Wei-Kun Chen, Leon Eifler, Maxime Gasse, Patrick Gemander, Ambros Gleixner, Leona Gottwald, Katrin Halbig, Gregor Hendel, Christopher Hojny, Thorsten Koch, Pierre Le Bodic, Stephen J. Maher, Frederic Matter, Matthias Miltenberger, Erik Mühmer, Benjamin Müller, Marc E. Pfetsch, Franziska Schlösser, Felipe Serrano, Yuji Shinano, Christine Tawfik, Stefan Vigerske, Fabian Wegscheider, Dieter Weninger, and Jakob Witzig. The SCIP Optimization Suite 7.0. Technical report, Optimization Online, March 2020. URL: http://www.optimization-online.org/DB_HTML/2020/03/7705.html.
  16. Maxime Gasse, Didier Chételat, Nicola Ferroni, Laurent Charlin, and Andrea Lodi. Exact combinatorial optimization with graph convolutional neural networks. In Annual Conference on Neural Information Processing Systems (NeurIPS), pages 15554-15566, 2019. Google Scholar
  17. Andrew Gilpin and Tuomas Sandholm. Information-theoretic approaches to branching in search. Discrete Optimization, 8(2):147-159, 2011. Early version in IJCAI-07. Google Scholar
  18. Ralph E. Gomory. Outline of an algorithm for integer solutions to linear programs. Bulletin of the American Mathematical Society, 64(5):275-278, 1958. Google Scholar
  19. Prateek Gupta, Maxime Gasse, Elias B Khalil, M Pawan Kumar, Andrea Lodi, and Yoshua Bengio. Hybrid models for learning to branch. In Annual Conference on Neural Information Processing Systems (NeurIPS), 2020. Google Scholar
  20. Rishi Gupta and Tim Roughgarden. A PAC approach to application-specific algorithm selection. SIAM Journal on Computing, 46(3):992-1017, 2017. Google Scholar
  21. He He, Hal Daume III, and Jason M Eisner. Learning to search in branch and bound algorithms. In Annual Conference on Neural Information Processing Systems (NeurIPS), 2014. Google Scholar
  22. Eric Horvitz, Yongshao Ruan, Carla Gomez, Henry Kautz, Bart Selman, and Max Chickering. A Bayesian approach to tackling hard computational problems. In Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI), 2001. Google Scholar
  23. Zeren Huang, Kerong Wang, Furui Liu, Hui-ling Zhen, Weinan Zhang, Mingxuan Yuan, Jianye Hao, Yong Yu, and Jun Wang. Learning to select cuts for efficient mixed-integer programming. arXiv preprint, 2021. URL: http://arxiv.org/abs/2105.13645.
  24. Frank Hutter, Holger Hoos, and Kevin Leyton-Brown. Sequential model-based optimization for general algorithm configuration. In International Conference on Learning and Intelligent Optimization (LION), pages 507-523, 2011. Google Scholar
  25. Frank Hutter, Holger H Hoos, Kevin Leyton-Brown, and Thomas Stützle. ParamILS: An automatic algorithm configuration framework. Journal of Artificial Intelligence Research, 36(1):267-306, 2009. Google Scholar
  26. Robert G Jeroslow. Trivial integer programs unsolvable by branch-and-bound. Mathematical Programming, 6(1):105-109, 1974. Google Scholar
  27. Serdar Kadioglu, Yuri Malitsky, Meinolf Sellmann, and Kevin Tierney. ISAC - instance-specific algorithm configuration. In European Conference on Artificial Intelligence (ECAI), 2010. Google Scholar
  28. Miroslav Karamanov and Gérard Cornuéjols. Branching on general disjunctions. Mathematical Programming, 128(1-2):403-436, 2011. Google Scholar
  29. Elias Khalil, Pierre Le Bodic, Le Song, George Nemhauser, and Bistra Dilkina. Learning to branch in mixed integer programming. In AAAI Conference on Artificial Intelligence, 2016. Google Scholar
  30. Elias Khalil, Bistra Dilkina, George Nemhauser, Shabbir Ahmed, and Yufen Shao. Learning to run heuristics in tree search. In International Joint Conference on Artificial Intelligence (IJCAI), 2017. Google Scholar
  31. Robert Kleinberg, Kevin Leyton-Brown, and Brendan Lucier. Efficiency through procrastination: Approximately optimal algorithm configuration with runtime guarantees. In International Joint Conference on Artificial Intelligence (IJCAI), 2017. Google Scholar
  32. Kevin Leyton-Brown, Eugene Nudelman, and Yoav Shoham. Empirical hardness models: Methodology and a case study on combinatorial auctions. Journal of the ACM, 56(4):1-52, 2009. Google Scholar
  33. Ashutosh Mahajan and Theodore K Ralphs. Experiments with branching using general disjunctions. In Operations Research and Cyber-Infrastructure, pages 101-118. Springer, 2009. Google Scholar
  34. Nimrod Megiddo. Combinatorial optimization with rational objective functions. Mathematics of Operations Research, pages 414-424, 1979. Google Scholar
  35. Jonathan H. Owen and Sanjay Mehrotra. Experimental results on using general disjunctions in branch-and-bound for general-integer linear programs. Computational Optimization and Applications, 20(2):159-170, November 2001. Google Scholar
  36. Ashish Sabharwal, Horst Samulowitz, and Chandra Reddy. Guiding combinatorial optimization with UCT. In International Conference on AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems. Springer, 2012. Google Scholar
  37. Tuomas Sandholm. Very-large-scale generalized combinatorial multi-attribute auctions: Lessons from conducting $60 billion of sourcing. In Zvika Neeman, Alvin Roth, and Nir Vulkan, editors, Handbook of Market Design. Oxford University Press, 2013. Google Scholar
  38. Jialin Song, Ravi Lanka, Yisong Yue, and Bistra Dilkina. A general large neighborhood search framework for solving integer programs. In Annual Conference on Neural Information Processing Systems (NeurIPS), 2020. Google Scholar
  39. Yunhao Tang, Shipra Agrawal, and Yuri Faenza. Reinforcement learning for integer programming: Learning to cut. International Conference on Machine Learning (ICML), 2020. Google Scholar
  40. Gellért Weisz, András György, and Csaba Szepesvári. LeapsAndBounds: method for approximately optimal algorithm configuration. In International Conference on Machine Learning (ICML), 2018. Google Scholar
  41. Franz Wesselmann and Uwe Suhl. Implementing cutting plane management and selection techniques. Technical report, University of Paderborn, 2012. Google Scholar
  42. Lin Xu, Frank Hutter, Holger H Hoos, and Kevin Leyton-Brown. Satzilla: portfolio-based algorithm selection for SAT. Journal of Artificial Intelligence Research, 32(1):565-606, 2008. Google Scholar
  43. Lin Xu, Frank Hutter, Holger H Hoos, and Kevin Leyton-Brown. Hydra-MIP: Automated algorithm configuration and selection for mixed integer programming. In RCRA workshop on Experimental Evaluation of Algorithms for Solving Problems with Combinatorial Explosion at the International Joint Conference on Artificial Intelligence (IJCAI), 2011. Google Scholar
  44. Yu Yang, Natashia Boland, Bistra Dilkina, and Martin Savelsbergh. Learning generalized strong branching for set covering, set packing, and 0-1 knapsack problems. Technical report, Technical Report, 2020., 2020. Google Scholar
  45. Yu Yang, Natashia Boland, and Martin Savelsbergh. Multivariable branching: A 0-1 knapsack problem case study. INFORMS Journal on Computing, 2021. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH Imprint Privacy Contact