Document Open Access Logo

Targeted Branching for the Maximum Independent Set Problem Using Graph Neural Networks

Authors Kenneth Langedal , Demian Hespe , Peter Sanders

PDF
Thumbnail PDF

File

LIPIcs.SEA.2024.20.pdf
  • Filesize: 1.03 MB
  • 21 pages

Document Identifiers

Author Details

Kenneth Langedal
  • University of Bergen, Department of Informatics, Norway
Demian Hespe
  • Karlsruhe Institute of Technology, Institute for Theoretical Informatics, Germany
Peter Sanders
  • Karlsruhe Institute of Technology, Institute for Theoretical Informatics, Germany

Funding

  • Langedal, Kenneth: Supported by the Research Council of Norway under contract 303404 and NORA National Research School for AI, project number 331723
  • Sanders, Peter: European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement No. 882500)

Acknowledgements

We thank Fredrik Manne for all the help and feedback he provided throughout the writing process.

Cite AsGet BibTex

Kenneth Langedal, Demian Hespe, and Peter Sanders. Targeted Branching for the Maximum Independent Set Problem Using Graph Neural Networks. In 22nd International Symposium on Experimental Algorithms (SEA 2024). Leibniz International Proceedings in Informatics (LIPIcs), Volume 301, pp. 20:1-20:21, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2024)
https://doi.org/10.4230/LIPIcs.SEA.2024.20

Abstract

Identifying a maximum independent set is a fundamental NP-hard problem. This problem has several real-world applications and requires finding the largest possible set of vertices not adjacent to each other in an undirected graph. Over the past few years, branch-and-bound and branch-and-reduce algorithms have emerged as some of the most effective methods for solving the problem exactly. Specifically, the branch-and-reduce approach, which combines branch-and-bound principles with reduction rules, has proven particularly successful in tackling previously unmanageable real-world instances. This progress was largely made possible by the development of more effective reduction rules. Nevertheless, other key components that can impact the efficiency of these algorithms have not received the same level of interest. Among these is the branching strategy, which determines which vertex to branch on next. Until recently, the most widely used strategy was to choose the vertex of the highest degree. In this work, we present a graph neural network approach for selecting the next branching vertex. The intricate nature of current branch-and-bound solvers makes supervised and reinforcement learning difficult. Therefore, we use a population-based genetic algorithm to evolve the model’s parameters instead. Our proposed approach results in a speedup on 73% of the benchmark instances with a median speedup of 24%.

Subject Classification

ACM Subject Classification
  • Mathematics of computing → Graph algorithms
  • Computing methodologies → Neural networks
Keywords
  • Graphs
  • Independent Set
  • Vertex Cover
  • Graph Neural Networks
  • Branch-and-Reduce

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Supplementary Materials

References

  1. Takuya Akiba and Yoichi Iwata. Branch-and-reduce exponential/FPT algorithms in practice: A case study of vertex cover. Theoretical Computer Science, 609:211-225, 2016. URL: https://doi.org/10.1016/j.tcs.2015.09.023.
  2. Maram Alsahafy and Lijun Chang. Computing maximum independent sets over large sparse graphs. In International Conference on Web Information Systems Engineering, pages 711-727. Springer, 2020. Google Scholar
  3. Sergiy Butenko and Wilbert E. Wilhelm. Clique-detection models in computational biochemistry and genomics. Eur. J. Oper. Res., 173(1):1-17, 2006. URL: https://doi.org/10.1016/j.ejor.2005.05.026.
  4. Randy Carraghan and Panos M. Pardalos. An exact algorithm for the maximum clique problem. Oper. Res. Lett., 9(6):375-382, 1990. URL: https://doi.org/10.1016/0167-6377(90)90057-C.
  5. Lijun Chang, Wei Li, and Wenjie Zhang. Computing A near-maximum independent set in linear time by reducing-peeling. In Proceedings of the 2017 ACM International Conference on Management of Data, pages 1181-1196. ACM, 2017. URL: https://doi.org/10.1145/3035918.3035939.
  6. Jianer Chen, Iyad A. Kanj, and Ge Xia. Improved upper bounds for vertex cover. Theoretical Computer Science, 411(40-42):3736-3756, 2010. URL: https://doi.org/10.1016/j.tcs.2010.06.026.
  7. Tammy M. K. Cheng, Yu-En Lu, Michele Vendruscolo, Pietro Liò, and Tom L. Blundell. Prediction by graph theoretic measures of structural effects in proteins arising from non-synonymous single nucleotide polymorphisms. PLoS Computational Biology, 4(7), 2008. URL: https://doi.org/10.1371/journal.pcbi.1000135.
  8. Jakob Dahlum, Sebastian Lamm, Peter Sanders, Christian Schulz, Darren Strash, and Renato F Werneck. Accelerating local search for the maximum independent set problem. In International symposium on experimental algorithms, pages 118-133. Springer, 2016. Google Scholar
  9. Camil Demetrescu, Andrew V Goldberg, and David S Johnson. The shortest path problem: Ninth DIMACS implementation challenge, volume 74. American Mathematical Soc., 2009. Google Scholar
  10. Iddo Drori, Anant Kharkar, William R Sickinger, Brandon Kates, Qiang Ma, Suwen Ge, Eden Dolev, Brenda Dietrich, David P Williamson, and Madeleine Udell. Learning to solve combinatorial optimization problems on real-world graphs in linear time. In 2020 19th IEEE International Conference on Machine Learning and Applications (ICMLA), pages 19-24. IEEE, 2020. Google Scholar
  11. M Ayaz Dzulfikar, Johannes K Fichte, and Markus Hecher. The pace 2019 parameterized algorithms and computational experiments challenge: the fourth iteration. In 14th International Symposium on Parameterized and Exact Computation (IPEC 2019). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2019. Google Scholar
  12. M. Ayaz Dzulfikar, Johannes K. Fichte, and Markus Hecher. The PACE 2019 Parameterized Algorithms and Computational Experiments Challenge: The Fourth Iteration (Invited Paper). In 14th International Symposium on Parameterized and Exact Computation (IPEC 2019), volume 148 of Leibniz International Proceedings in Informatics (LIPIcs), pages 25:1-25:23. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2019. URL: https://doi.org/10.4230/LIPIcs.IPEC.2019.25.
  13. Tarek Faycal and Claudio Zito. Direct mutation and crossover in genetic algorithms applied to reinforcement learning tasks. arXiv preprint arXiv:2201.04815, 2022. Google Scholar
  14. Fedor V. Fomin, Fabrizio Grandoni, and Dieter Kratsch. A measure & conquer approach for the analysis of exact algorithms. J. ACM, 56(5):25:1-25:32, 2009. URL: https://doi.org/10.1145/1552285.1552286.
  15. M. R. Garey, D. S. Johnson, and L. Stockmeyer. Some simplified NP-complete problems. In Proceedings of the 6th ACM Symposium on Theory of Computing, STOC '74, pages 47-63. ACM, 1974. Google Scholar
  16. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016. URL: http://www.deeplearningbook.org.
  17. William L. Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs, 2018. URL: https://arxiv.org/abs/1706.02216.
  18. Demian Hespe, Sebastian Lamm, and Christian Schorr. Targeted branching for the maximum independent set problem. In 19th International Symposium on Experimental Algorithms, SEA 2021, June 7-9, 2021, Nice, France, volume 190 of LIPIcs, pages 17:1-17:21. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2021. URL: https://doi.org/10.4230/LIPIcs.SEA.2021.17.
  19. Demian Hespe, Sebastian Lamm, Christian Schulz, and Darren Strash. WeGotYouCovered: The winning solver from the PACE 2019 challenge, vertex cover track. In Proceedings of the SIAM Workshop on Combinatorial Scientific Computing, pages 1-11. SIAM, 2020. URL: https://doi.org/10.1137/1.9781611976229.1.
  20. Demian Hespe, Christian Schulz, and Darren Strash. Scalable kernelization for maximum independent sets. Journal of Experimental Algorithmics (JEA), 24(1):1-22, 2019. Google Scholar
  21. David S Johnson. Cliques, coloring, and satisfiability: second dimacs implementation challenge. DIMACS series in discrete mathematics and theoretical computer science, 26:11-13, 1993. Google Scholar
  22. Chaitanya K Joshi, Thomas Laurent, and Xavier Bresson. An efficient graph convolutional network technique for the travelling salesman problem. arXiv preprint arXiv:1906.01227, 2019. Google Scholar
  23. Elias Khalil, Hanjun Dai, Yuyu Zhang, Bistra Dilkina, and Le Song. Learning combinatorial optimization algorithms over graphs. Advances in neural information processing systems, 30, 2017. Google Scholar
  24. Tim Kieritz, Dennis Luxen, Peter Sanders, and Christian Vetter. Distributed time-dependent contraction hierarchies. In Experimental Algorithms, 9th International Symposium, volume 6049, pages 83-93. Springer, 2010. URL: https://doi.org/10.1007/978-3-642-13193-6_8.
  25. Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016. Google Scholar
  26. Kenneth Langedal, Johannes Langguth, Fredrik Manne, and Daniel Thilo Schroeder. Efficient minimum weight vertex cover heuristics using graph neural networks. In 20th International Symposium on Experimental Algorithms (SEA 2022). Schloss Dagstuhl-Leibniz-Zentrum für Informatik, 2022. Google Scholar
  27. Henrique Lemos, Marcelo Prates, Pedro Avelar, and Luis Lamb. Graph colouring meets deep learning: Effective graph neural network models for combinatorial problems. In 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI), pages 879-885. IEEE, 2019. Google Scholar
  28. Jure Leskovec and Andrej Krevl. SNAP Datasets: Stanford large network dataset collection. http://snap.stanford.edu/data, June 2014.
  29. Chu-Min Li, Zhiwen Fang, and Ke Xu. Combining maxsat reasoning and incremental upper bound for the maximum clique problem. In 25th IEEE International Conference on Tools with Artificial Intelligence, pages 939-946. IEEE Computer Society, 2013. URL: https://doi.org/10.1109/ICTAI.2013.143.
  30. Chu-Min Li, Hua Jiang, and Felip Manyà. On minimization of the number of branches in branch-and-bound algorithms for the maximum clique problem. Comput. Oper. Res., 84:1-15, 2017. URL: https://doi.org/10.1016/j.cor.2017.02.017.
  31. Chu Min Li and Zhe Quan. An efficient branch-and-bound algorithm based on maxsat for the maximum clique problem. In Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence. AAAI Press, 2010. URL: http://www.aaai.org/ocs/index.php/AAAI/AAAI10/paper/view/1611.
  32. Zhuwen Li, Qifeng Chen, and Vladlen Koltun. Combinatorial optimization with graph convolutional networks and guided tree search. Advances in neural information processing systems, 31, 2018. Google Scholar
  33. Melanie Mitchell. An introduction to genetic algorithms. MIT press, 1998. Google Scholar
  34. Michael Mitzenmacher and Sergei Vassilvitskii. Algorithms with predictions. Communications of the ACM, 65(7):33-35, 2022. Google Scholar
  35. Decebal Constantin Mocanu, Elena Mocanu, Peter Stone, Phuong H Nguyen, Madeleine Gibescu, and Antonio Liotta. Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science. Nature communications, 9(1):2383, 2018. Google Scholar
  36. David J Montana, Lawrence Davis, et al. Training feedforward neural networks using genetic algorithms. In IJCAI, volume 89, pages 762-767, 1989. Google Scholar
  37. Christopher Morris, Martin Ritzert, Matthias Fey, William L Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe. Weisfeiler and leman go neural: Higher-order graph neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 4602-4609, 2019. Google Scholar
  38. Vinod Nair, Sergey Bartunov, Felix Gimeno, Ingrid Von Glehn, Pawel Lichocki, Ivan Lobov, Brendan O'Donoghue, Nicolas Sonnerat, Christian Tjandraatmadja, Pengming Wang, et al. Solving mixed integer programs using neural networks. arXiv preprint arXiv:2012.13349, 2020. Google Scholar
  39. Mohammadreza Nazari, Afshin Oroojlooy, Lawrence Snyder, and Martin Takác. Reinforcement learning for solving the vehicle routing problem. Advances in neural information processing systems, 31, 2018. Google Scholar
  40. Rick Plachetta and Alexander van der Grinten. SAT-and-reduce for vertex cover: Accelerating branch-and-reduce by SAT solving. In Proceedings of the Symposium on Algorithm Engineering and Experiments, ALENEX 2021, Virtual Conference, January 10-11, 2021, pages 169-180. SIAM, 2021. URL: https://doi.org/10.1137/1.9781611976472.13.
  41. Deepak Puthal, Surya Nepal, Cécile Paris, Rajiv Ranjan, and Jinjun Chen. Efficient algorithms for social network coverage and reach. In IEEE International Congress on Big Data, pages 467-474. IEEE Computer Society, 2015. URL: https://doi.org/10.1109/BigDataCongress.2015.75.
  42. Ryan A. Rossi and Nesreen K. Ahmed. The network data repository with interactive graph analytics and visualization. In AAAI, 2015. URL: https://networkrepository.com.
  43. Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, and Ilya Sutskever. Evolution strategies as a scalable alternative to reinforcement learning. arXiv preprint arXiv:1703.03864, 2017. Google Scholar
  44. Pedro V. Sander, Diego Nehab, Eden Chlamtac, and Hugues Hoppe. Efficient traversal of mesh edges using adjacency primitives. ACM Trans. Graph., 27(5):144, 2008. URL: https://doi.org/10.1145/1409060.1409097.
  45. Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph geural network model. IEEE Transactions on Neural Networks, 20(1):61-80, 2008. Google Scholar
  46. Martin JA Schuetz, J Kyle Brubaker, and Helmut G Katzgraber. Combinatorial optimization with physics-inspired graph neural networks. Nature Machine Intelligence, 4(4):367-377, 2022. Google Scholar
  47. Pablo San Segundo and Cristóbal Tapia. Relaxed approximate coloring in exact maximum clique search. Comput. Oper. Res., 44:185-192, 2014. URL: https://doi.org/10.1016/j.cor.2013.10.018.
  48. David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mastering the game of go with deep neural networks and tree search. nature, 529(7587):484-489, 2016. Google Scholar
  49. Darren Strash. On the power of simple reductions for the maximum independent set problem. In Computing and Combinatorics - 22nd International Conference, COCOON 2016, Ho Chi Minh City, Vietnam, August 2-4, 2016, Proceedings, volume 9797 of Lecture Notes in Computer Science, pages 345-356. Springer, 2016. URL: https://doi.org/10.1007/978-3-319-42634-1_28.
  50. Felipe Petroski Such, Vashisht Madhavan, Edoardo Conti, Joel Lehman, Kenneth O Stanley, and Jeff Clune. Deep neuroevolution: Genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning. arXiv preprint arXiv:1712.06567, 2017. Google Scholar
  51. Etsuji Tomita, Yoichi Sutani, Takanori Higashi, and Mitsuo Wakatsuki. A simple and faster branch-and-bound algorithm for finding a maximum clique with computational experiments. IEICE Trans. Inf. Syst., 96-D(6):1286-1298, 2013. URL: https://doi.org/10.1587/transinf.E96.D.1286.
  52. Hanchen Wang, Ying Zhang, Lu Qin, Wei Wang, Wenjie Zhang, and Xuemin Lin. Reinforcement learning based query vertex ordering model for subgraph matching. In 2022 IEEE 38th International Conference on Data Engineering (ICDE), pages 245-258. IEEE, 2022. Google Scholar
  53. Luzhi Wang, Chu-Min Li, Junping Zhou, Bo Jin, and Minghao Yin. An exact algorithm for minimum weight vertex cover problem in large graphs. arXiv preprint arXiv:1903.05948, 2019. Google Scholar
  54. Mingyu Xiao and Hiroshi Nagamochi. Exact algorithms for maximum independent set. Inf. Comput., 255:126-146, 2017. URL: https://doi.org/10.1016/j.ic.2017.06.001.
  55. Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks?, 2019. URL: https://arxiv.org/abs/1810.00826.
© 2023-2024 Schloss Dagstuhl – LZI GmbH Imprint Privacy Contact