Document Open Access Logo

A Fast Data Structure for Dynamic Graphs Based on Hash-Indexed Adjacency Blocks

Authors Alexander van der Grinten, Maria Predari, Florian Willich

PDF
Thumbnail PDF

File

LIPIcs.SEA.2022.11.pdf
  • Filesize: 0.87 MB
  • 18 pages

Document Identifiers

Author Details

Alexander van der Grinten
  • Humboldt-Universität zu Berlin, Germany
Maria Predari
  • Humboldt-Universität zu Berlin, Germany
Florian Willich
  • Humboldt-Universität zu Berlin, Germany

Funding

  • van der Grinten, Alexander: The author was supported by German Research Foundation (DFG) grant GR 5745/1-1 (DyANE).
  • Predari, Maria: The author was supported by German Research Foundation (DFG) DFG grant ME 3619/4-1 (ALMACOM).

Acknowledgements

The authors would like to thank Duy Le Thanh for his help in setting up some competitors.

Cite As Get BibTex

Alexander van der Grinten, Maria Predari, and Florian Willich. A Fast Data Structure for Dynamic Graphs Based on Hash-Indexed Adjacency Blocks. In 20th International Symposium on Experimental Algorithms (SEA 2022). Leibniz International Proceedings in Informatics (LIPIcs), Volume 233, pp. 11:1-11:18, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2022) https://doi.org/10.4230/LIPIcs.SEA.2022.11

Abstract

Several dynamic graph data structures have been proposed in literature. Yet, these data structures either offer limited support for arbitrary graph algorithms or they are designed as part of specific frameworks (e.g., for GPUs or specialized hardware). Such frameworks are difficult to adopt to arbitrary graph computations and lead practitioners to fall back to less sophisticated solutions when dealing with dynamic graphs. In this work, we propose a new "dynamic hashed blocks" (DHB) data structure for sparse dynamic graphs and matrices on general-purpose CPU architectures. DHB combines an efficient block-based memory layout to store incident edges with an additional per-vertex hash index for high degree vertices. This hash index allows us to quickly insert edges without introducing duplicates, while the block-based memory layout retains advantageous cache locality properties of traditional adjacency arrays.
Experiments show that DHB outperforms competing dynamic graph structures for edge insertions, updates, deletions, and traversal operations. Compared to static CSR layouts, DHB exhibits only a small overhead in traversal performance. DHB’s interface is similar to general-purpose abstract graph data types and can be easily used as a drop-in replacement for traditional adjacency arrays. To demonstrate that, we modify the well-known NetworKit framework to use DHB instead of its own dynamic graph representation. Experiments show that this modification only slightly penalizes the performance of graph algorithms while considerably boosting update rates.

Subject Classification

ACM Subject Classification
  • Theory of computation → Dynamic graph algorithms
Keywords
  • dynamic graph data structures
  • sparse matrix layout
  • dynamic algorithms
  • parallel algorithms
  • graph analysis

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Supplementary Materials

References

  1. Khaled Ammar. Techniques and systems for large dynamic graphs. In Eduard C. Dragut and Heng Tao Shen, editors, Proceedings of the SIGMOD 2016 PhD Symposium, San Francisco, California, USA, June 26, 2016, pages 7-11. ACM, 2016. Google Scholar
  2. Eugenio Angriman, Alexander van der Grinten, Moritz von Looz, Henning Meyerhenke, Martin Nöllenburg, Maria Predari, and Charilaos Tzovas. Guidelines for experimental algorithmics: A case study in network analysis. Algorithms, 12(7):127, 2019. Google Scholar
  3. Michael A. Bender and Haodong Hu. An adaptive packed-memory array. In Proceedings of the Twenty-Fifth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, PODS '06, pages 20-29, New York, NY, USA, 2006. Association for Computing Machinery. URL: https://doi.org/10.1145/1142351.1142355.
  4. Maciej Besta, Marc Fischer, Vasiliki Kalavri, Michael Kapralov, and Torsten Hoefler. Practice of streaming processing of dynamic graphs: concepts, models, and systems, 2021. URL: https://open.bu.edu/handle/2144/42895.
  5. Federico Busato, Oded Green, Nicola Bombieri, and David A. Bader. Hornet: An efficient data structure for dynamic sparse graphs and matrices on gpus. In The 22nd Annual IEEE High Performance Extreme Computing Conference, HPEC 2018, Waltham, MA, USA, September 25-27, 2018, pages 1-7, Los Alamitos, CA, 2018. IEEE Computer Society. URL: https://doi.org/10.1109/HPEC.2018.8547541.
  6. Intel Corporation. Intel cilk plus language specification, 2010. URL: http://software.intel.com/sites/products/cilkplus/cilk_plus_language_specification.pdf.
  7. Laxman Dhulipala, Guy E. Blelloch, and Julian Shun. Low-latency graph streaming using compressed purely-functional trees. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2019, pages 918-934, New York, NY, USA, 2019. Association for Computing Machinery. URL: https://doi.org/10.1145/3314221.3314598.
  8. David Ediger, Robert McColl, Jason E. Riedy, and A. David Bader. Stinger: High performance data structure for streaming graphs. HPEC, pages 1-5, 2012. Google Scholar
  9. Jianhua Gao, Weixing Ji, Zhaonian Tan, and Yueyan Zhao. A systematic survey of general sparse matrix-matrix multiplication. CoRR, abs/2002.11273, 2020. URL: http://arxiv.org/abs/2002.11273.
  10. Fred G. Gustavson. Two fast algorithms for sparse matrices: Multiplication and permuted transposition. ACM Trans. Math. Softw., 4(3):250-269, September 1978. URL: https://doi.org/10.1145/355791.355796.
  11. Jeremy Kepner, Peter Aaltonen, A. David Bader, Aydin Buluç, Franz Franchetti, R. John Gilbert, Dylan Hutchison, Manoj Kumar, Andrew Lumsdaine, Henning Meyerhenke, Scott McMillan, E. José Moreira, D. John Owens, Carl Yang, Marcin Zalewski, and G. Timothy Mattson. Mathematical foundations of the graphblas. HPEC, pages 1-9, 2016. Google Scholar
  12. Jérôme Kunegis. Konect: The koblenz network collection. In Proceedings of the 22nd International Conference on World Wide Web, WWW '13 Companion, pages 1343-1350, New York, NY, USA, 2013. Association for Computing Machinery. URL: https://doi.org/10.1145/2487788.2488173.
  13. Daan Leijen, Benjamin Zorn, and Leonardo de Moura. Mimalloc: Free list sharding in action. In APLAS, volume 11893 of Lecture Notes in Computer Science, pages 244-265. Springer, 2019. Google Scholar
  14. Charles E. Leiserson. Cilk, pages 273-288. Springer US, Boston, MA, 2011. URL: https://doi.org/10.1007/978-0-387-09766-4_289.
  15. J. Leskovec. Stanford Network Analysis Package (SNAP). URL: http://snap.stanford.edu/index.html.
  16. Chun Liu, Shuhang Zhang, Hangbin Wu, and Qiang Fu. A dynamic spatiotemporal analysis model for traffic incident influence prediction on urban road networks. ISPRS International Journal of Geo-Information, 6(11), 2017. URL: https://www.mdpi.com/2220-9964/6/11/362.
  17. Yucheng Low, Joseph Gonzalez, Aapo Kyrola, Danny Bickson, Carlos Guestrin, and Joseph Hellerstein. Graphlab: A new framework for parallel machine learning. In Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence, UAI'10, pages 340-349, Arlington, Virginia, USA, 2010. AUAI Press. Google Scholar
  18. Fredrik Manne and Mahantesh Halappanavar. New effective multithreaded matching algorithms. In IPDPS, pages 519-528. IEEE Computer Society, 2014. Google Scholar
  19. Mugilan Mariappan and Keval Vora. Graphbolt: Dependency-driven synchronous processing of streaming graphs. In Proceedings of the Fourteenth EuroSys Conference 2019, EuroSys '19, New York, NY, USA, 2019. Association for Computing Machinery. Google Scholar
  20. Prashant Pandey, Brian Wheatman, Helen Xu, and Aydin Buluc. Terrace: A hierarchical graph container for skewed dynamic graphs. In Proceedings of the 2021 International Conference on Management of Data, SIGMOD/PODS '21, pages 1372-1385, New York, NY, USA, 2021. Association for Computing Machinery. Google Scholar
  21. J. M. Robson. Worst case fragmentation of first fit and best fit storage allocation strategies. Comput. J., 20(3):242-244, 1977. Google Scholar
  22. Ryan A. Rossi and Nesreen K. Ahmed. The network data repository with interactive graph analytics and visualization. In AAAI, 2015. URL: http://networkrepository.com.
  23. Tao B. Schardl, William S. Moses, and Charles E. Leiserson. Tapir: Embedding fork-join parallelism into llvm’s intermediate representation. In Proceedings of the 22nd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP '17, pages 249-265, New York, NY, USA, 2017. Association for Computing Machinery. URL: https://doi.org/10.1145/3018743.3018758.
  24. Dipanjan Sengupta, Narayanan Sundaram, Xia Zhu, Theodore L. Willke, Jeffrey Young, Matthew Wolf, and Karsten Schwan. Graphin: An online high performance incremental graph processing framework. In Proceedings of the 22nd International Conference on Euro-Par 2016: Parallel Processing - Volume 9833, pages 319-333, 2016. Google Scholar
  25. Mo Sha, Yuchen Li, Bingsheng He, and Kian-Lee Tan. Accelerating dynamic graph analytics on gpus. Proc. VLDB Endow., 11(1):107-120, September 2017. Google Scholar
  26. Julian Shun and E. Guy Blelloch. Ligra: a lightweight graph processing framework for shared memory. PPOPP, pages 135-146, 2013. Google Scholar
  27. Christian L. Staudt, Aleksejs Sazonovs, and Henning Meyerhenke. Networkit: A tool suite for large-scale complex network analysis. Network Science, 4(4):508-530, 2016. URL: https://doi.org/10.1017/nws.2016.20.
  28. Brian Wheatman and Helen Xu. Packed compressed sparse row: A dynamic graph representation. In 2018 IEEE High Performance Extreme Computing Conference, HPEC 2018, Waltham, MA, USA, September 25-27, 2018, pages 1-7, September 2018. URL: https://doi.org/10.1109/HPEC.2018.8547566.
  29. Brian Wheatman and Helen Xu. A parallel packed memory array to store dynamic graphs. In Martin Farach-Colton and Sabine Storandt, editors, Proceedings of the Symposium on Algorithm Engineering and Experiments, ALENEX 2021, Virtual Conference, January 10-11, 2021, pages 31-45. SIAM, 2021. Google Scholar
  30. Martin Winter, Daniel Mlakar, Rhaleb Zayer, Hans-Peter Seidel, and Markus Steinberger. Faimgraph: High performance management of fully-dynamic graphs under tight memory constraints on the gpu. In Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis, SC '18. IEEE Press, 2018. Google Scholar
  31. Martin Winter, Rhaleb Zayer, and Markus Steinberger. Autonomous, independent management of dynamic graphs on gpus. In 2017 IEEE High Performance Extreme Computing Conference (HPEC), pages 1-8, 2017. URL: https://doi.org/10.1109/HPEC.2017.8091058.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact