Document Open Access Logo

FREIGHT: Fast Streaming Hypergraph Partitioning

Authors Kamal Eyubov , Marcelo Fonseca Faraj , Christian Schulz

PDF
Thumbnail PDF

File

LIPIcs.SEA.2023.15.pdf
  • Filesize: 1.21 MB
  • 16 pages

Document Identifiers

Author Details

Kamal Eyubov
  • Universität Heidelberg, Germany
Marcelo Fonseca Faraj
  • Universität Heidelberg, Germany
Christian Schulz
  • Universität Heidelberg, Germany

Funding

We acknowledge support by DFG grant SCHU 2567/5-1.

Cite As Get BibTex

Kamal Eyubov, Marcelo Fonseca Faraj, and Christian Schulz. FREIGHT: Fast Streaming Hypergraph Partitioning. In 21st International Symposium on Experimental Algorithms (SEA 2023). Leibniz International Proceedings in Informatics (LIPIcs), Volume 265, pp. 15:1-15:16, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2023) https://doi.org/10.4230/LIPIcs.SEA.2023.15

Abstract

Partitioning the vertices of a (hyper)graph into k roughly balanced blocks such that few (hyper)edges run between blocks is a key problem for large-scale distributed processing. A current trend for partitioning huge (hyper)graphs using low computational resources are streaming algorithms. In this work, we propose FREIGHT: a Fast stREamInG Hypergraph parTitioning algorithm which is an adaptation of the widely-known graph-based algorithm Fennel. By using an efficient data structure, we make the overall running of FREIGHT linearly dependent on the pin-count of the hypergraph and the memory consumption linearly dependent on the numbers of nets and blocks. The results of our extensive experimentation showcase the promising performance of FREIGHT as a highly efficient and effective solution for streaming hypergraph partitioning. Our algorithm demonstrates competitive running time with the Hashing algorithm, with a difference of a maximum factor of four observed on three fourths of the instances. Significantly, our findings highlight the superiority of FREIGHT over all existing (buffered) streaming algorithms and even the in-memory algorithm HYPE, with respect to both cut-net and connectivity measures. This indicates that our proposed algorithm is a promising hypergraph partitioning tool to tackle the challenge posed by large-scale and dynamic data processing.

Subject Classification

ACM Subject Classification
  • Theory of computation → Streaming, sublinear and near linear time algorithms
  • Theory of computation → Graph algorithms analysis
Keywords
  • Hypergraph partitioning
  • graph partitioning
  • edge partitioning
  • streaming

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Related Versions

References

  1. Zainab Abbas, Vasiliki Kalavri, Paris Carbone, and Vladimir Vlassov. Streaming graph partitioning: An experimental study. Proc. VLDB Endow., 11(11):1590-1603, 2018. URL: https://doi.org/10.14778/3236187.3236208.
  2. Yaroslav Akhremtsev, Peter Sanders, and Christian Schulz. High-quality shared-memory graph partitioning. In Marco Aldinucci, Luca Padovani, and Massimo Torquati, editors, Euro-Par 2018: Parallel Processing - 24th International Conference on Parallel and Distributed Computing, Turin, Italy, August 27-31, 2018, Proceedings, volume 11014 of Lecture Notes in Computer Science, pages 659-671. Springer, 2018. URL: https://doi.org/10.1007/978-3-319-96983-1_47.
  3. Dan Alistarh, Jennifer Iglesias, and Milan Vojnovic. Streaming min-max hypergraph partitioning. In Advances in Neural Information Processing Systems, pages 1900-1908, 2015. URL: https://doi.org/10.5555/2969442.2969452.
  4. Charles J. Alpert. The ISPD98 circuit benchmark suite. In Majid Sarrafzadeh, editor, Proceedings of the 1998 International Symposium on Physical Design, ISPD 1998, Monterey, CA, USA, April 6-8, 1998, pages 80-85. ACM, 1998. URL: https://doi.org/10.1145/274535.274546.
  5. Amel Awadelkarim and Johan Ugander. Prioritized restreaming algorithms for balanced graph partitioning. In Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Aditya Prakash, editors, KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, pages 1877-1887. ACM, 2020. URL: https://doi.org/10.1145/3394486.3403239.
  6. Anton Belov, Daiel Diepold, Marijn Heule, and Matti Järvisalo. The sat competition 2014. http://www.satcompetition.org/2014/, 2014.
  7. Ulrik Brandes, Daniel Delling, Marco Gaertler, Robert Gorke, Martin Hoefer, Zoran Nikoloski, and Dorothea Wagner. On modularity clustering. IEEE transactions on knowledge and data engineering, 20(2):172-188, 2007. URL: https://doi.org/10.1109/TKDE.2007.190689.
  8. Thang Nguyen Bui and Curt Jones. Finding good approximate vertex and edge partitions is np-hard. Inf. Process. Lett., 42(3):153-159, 1992. URL: https://doi.org/10.1016/0020-0190(92)90140-Q.
  9. Aydın Buluç, Henning Meyerhenke, Ilya Safro, Peter Sanders, and Christian Schulz. Recent Advances in Graph Partitioning, pages 117-158. Springer International Publishing, Cham, 2016. URL: https://doi.org/10.1007/978-3-319-49487-6_4.
  10. Ümit V. Çatalyürek and Cevdet Aykanat. Patoh (partitioning tool for hypergraphs). In David A. Padua, editor, Encyclopedia of Parallel Computing, pages 1479-1487. Springer, 2011. URL: https://doi.org/10.1007/978-0-387-09766-4_93.
  11. Ümit V. Çatalyürek, Karen D. Devine, Marcelo Fonseca Faraj, Lars Gottesbüren, Tobias Heuer, Henning Meyerhenke, Peter Sanders, Sebastian Schlag, Christian Schulz, Daniel Seemaier, and Dorothea Wagner. More recent advances in (hyper)graph partitioning. ACM Computing Surveys, 2023. URL: https://doi.org/doi.org/10.1145/3571808.
  12. Timothy A. Davis and Yifan Hu. The university of florida sparse matrix collection. ACM Trans. Math. Softw., 38(1):1:1-1:25, 2011. URL: https://doi.org/10.1145/2049662.2049663.
  13. Marcelo Fonseca Faraj and Christian Schulz. Buffered streaming graph partitioning. ACM J. Exp. Algorithmics, 27, October 2022. URL: https://doi.org/10.1145/3546911.
  14. Marcelo Fonseca Faraj and Christian Schulz. Recursive multi-section on the fly: Shared-memory streaming algorithms for hierarchical graph partitioning and process mapping. In 2022 IEEE International Conference on Cluster Computing (CLUSTER), pages 473-483, 2022. URL: https://doi.org/10.1109/CLUSTER51413.2022.00057.
  15. Marcelo Fonseca Faraj, Alexander van der Grinten, Henning Meyerhenke, Jesper Larsson Träff, and Christian Schulz. High-quality hierarchical process mapping. In 18th International Symposium on Experimental Algorithms, SEA, volume 160 of LIPIcs, pages 4:1-4:15. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2020. URL: https://doi.org/10.4230/LIPIcs.SEA.2020.4.
  16. Michael R. Garey, David S. Johnson, and Larry J. Stockmeyer. Some simplified np-complete problems. In Proceedings of the 6th Annual ACM Symposium on Theory of Computing, April 30 - May 2, 1974, Seattle, Washington, USA, pages 47-63. ACM, 1974. URL: https://doi.org/10.1145/800119.803884.
  17. Lars Gottesbüren, Tobias Heuer, Peter Sanders, and Sebastian Schlag. Scalable Shared-Memory Hypergraph Partitioning. In Proceedings of the Symposium on Algorithm Engineering and Experiments ALENEX, pages 16-30, 2021. URL: https://doi.org/10.1137/1.9781611976472.2.
  18. Lars Gottesbüren, Tobias Heuer, Peter Sanders, Christian Schulz, and Daniel Seemaier. Deep multilevel graph partitioning. In 29th Annual European Symposium on Algorithms, ESA, volume 204 of LIPIcs, pages 48:1-48:17. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2021. URL: https://doi.org/10.4230/LIPIcs.ESA.2021.48.
  19. Loc Hoang, Roshan Dathathri, Gurbinder Gill, and Keshav Pingali. Cusp: A customizable streaming edge partitioner for distributed graph analytics. In 2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS), pages 439-450. IEEE, 2019. URL: https://doi.org/10.1109/IPDPS.2019.00054.
  20. Nazanin Jafari, Oguz Selvitopi, and Cevdet Aykanat. Fast shared-memory streaming multilevel graph partitioning. Journal of Parallel and Distributed Computing, 147:140-151, 2021. URL: https://doi.org/10.1016/j.jpdc.2020.09.004.
  21. George Karypis and Vipin Kumar. Parallel multilevel k-way partitioning scheme for irregular graphs. In Proceedings of the ACM/IEEE Conference on Supercomputing, page 35. IEEE Computer Society, 1996. URL: https://doi.org/10.1109/SC.1996.32.
  22. George Karypis and Vipin Kumar. Multilevel k-way hypergraph partitioning. In Mary Jane Irwin, editor, Proceedings of the 36th Conference on Design Automation, pages 343-348. ACM Press, 1999. URL: https://doi.org/10.1145/309847.309954.
  23. Renaud Lambiotte, Martin Rosvall, and Ingo Scholtes. From networks to optimal higher-order models of complex systems. Nature physics, 15(4):313-320, 2019. URL: https://doi.org/10.1038/s41567-019-0459-y.
  24. Christian Mayer, Ruben Mayer, Sukanya Bhowmik, Lukas Epple, and Kurt Rothermel. HYPE: massive hypergraph partitioning with neighborhood expansion. In IEEE International Conference on Big Data (IEEE BigData), pages 458-467. IEEE, 2018. URL: https://doi.org/10.1109/BigData.2018.8621968.
  25. Christian Mayer, Ruben Mayer, Muhammad Adnan Tariq, Heiko Geppert, Larissa Laich, Lukas Rieger, and Kurt Rothermel. Adwise: Adaptive window-based streaming edge partitioning for high-speed graph processing. In 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS), pages 685-695. IEEE, 2018. URL: https://doi.org/10.1109/ICDCS.2018.00072.
  26. Joel Nishimura and Johan Ugander. Restreaming graph partitioning: simple versatile algorithms for advanced balancing. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1106-1114, 2013. URL: https://doi.org/10.1145/2487575.2487696.
  27. Md Anwarul Kaium Patwary, Saurabh Kumar Garg, and Byeong Kang. Window-based streaming graph partitioning algorithm. In Proceedings of the Australasian Computer Science Week Multiconference, ACSW, pages 51:1-51:10. ACM, 2019. URL: https://doi.org/10.1145/3290688.3290711.
  28. François Pellegrini and Jean Roman. Experimental analysis of the dual recursive bipartitioning algorithm for static mapping. Technical report, TR 1038-96, LaBRI, 1996. URL: https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=94b913363b57e019b8a32529b076a8d4181587ac.
  29. Maria Predari, Charilaos Tzovas, Christian Schulz, and Henning Meyerhenke. An mpi-based algorithm for mapping complex networks onto hierarchical architectures. In Euro-Par 2021: Parallel Processing - 27th International Conference on Parallel and Distributed Computing, volume 12820 of LNCS, pages 167-182. Springer, 2021. URL: https://doi.org/10.1007/978-3-030-85665-6_11.
  30. Peter Sanders and Christian Schulz. Think locally, act globally: Highly balanced graph partitioning. In Experimental Algorithms, 12th International Symposium, SEA, volume 7933 of LNCS, pages 164-175. Springer, 2013. URL: https://doi.org/10.1007/978-3-642-38527-8_16.
  31. Sebastian Schlag, Vitali Henne, Tobias Heuer, Henning Meyerhenke, Peter Sanders, and Christian Schulz. k-way hypergraph partitioning via n-level recursive bisection. In Proceedings of the Eighteenth Workshop on Algorithm Engineering and Experiments, ALENEX, pages 53-67. SIAM, 2016. URL: https://doi.org/10.1137/1.9781611974317.5.
  32. Sebastian Schlag, Tobias Heuer, Lars Gottesbüren, Yaroslav Akhremtsev, Christian Schulz, and Peter Sanders. High-quality hypergraph partitioning. ACM Journal of Experimental Algorithms (JEA), 2022. URL: https://doi.org/10.1145/3529090.
  33. Christian Schulz and Darren Strash. Graph partitioning: Formulations and applications to big data. In Encyclopedia of Big Data Technologies. Springer, 2019. URL: https://doi.org/10.1007/978-3-319-63962-8_312-2.
  34. Isabelle Stanton and Gabriel Kliot. Streaming graph partitioning for large distributed graphs. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1222-1230, 2012. URL: https://doi.org/10.1145/2339530.2339722.
  35. Fatih Taşyaran, Berkay Demireller, Kamer Kaya, and Bora Uçar. Streaming Hypergraph Partitioning Algorithms on Limited Memory Environments. In HPCS 2020 - International Conference on High Performance Computing & Simulation, pages 1-8. IEEE, 2021. URL: https://hal.archives-ouvertes.fr/hal-03182122.
  36. Charalampos Tsourakakis, Christos Gkantsidis, Bozidar Radunovic, and Milan Vojnovic. Fennel: Streaming graph partitioning for massive scale graphs. In Proceedings of the 7th ACM international conference on Web search and data mining, pages 333-342, 2014. URL: https://doi.org/10.1145/2556195.2556213.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact