Document Open Access Logo

Foundations of Graph Neural Networks (A Logician’s View) (Invited Paper)

Author Egor V. Kostylev

PDF

File

Thumbnail PDF
PDF
OASIcs.RW.2024-2025.3.pdf
  • Filesize: 0.65 MB
  • 19 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Theory of computation → Query learning
  • Theory of computation → Modal and temporal logics
  • Information systems → Query languages
Keywords
  • Graph Neural Networks
  • Expressivity
  • Logic

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

Graph Neural Networks (GNNs) are a family of neural architectures that are naturally suited to learning functions on graphs. They are now used in a wide range of applications. It has been observed that GNNs share many similarities with classical computer science (CS) formalisms, such as the Weisfeiler-Leman graph isomorphism test, bisimulation, and logic. Most notably, both GNNs and these formalisms deal with functions on graphs and graph-like structures. This observation opens up an opportunity to compare GNN architectures with these formalisms in terms of different kinds of expressibility, thus positioning these architectures within the well-established landscape of theoretical CS. This, in turn, helps us better understand the fundamental capabilities and limitations of various GNN architectures, enabling more informed choices about which architecture to use - if any at all. In these lecture notes, I give an introduction to the state-of-the-art foundations of GNNs - specifically, our current understanding of their expressibility in terms of the classical formalisms, considering several notions of expressive power.

Cite As Get BibTex

Egor V. Kostylev. Foundations of Graph Neural Networks (A Logician’s View) (Invited Paper). In Joint Proceedings of the 20th and 21st Reasoning Web Summer Schools (RW 2024 & RW 2025). Open Access Series in Informatics (OASIcs), Volume 138, pp. 3:1-3:19, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/OASIcs.RW.2024/2025.3

Author Details

Egor V. Kostylev
  • University of Oslo, Norway

Funding

The Research Council of Norway through its Centres of Excellence scheme, Integreat - Norwegian Centre for knowledge-driven machine learning, project 332645.

References

  1. Ralph Abboud, İsmail İlkan Ceylan, Martin Grohe, and Thomas Lukasiewicz. The surprising power of graph neural networks with random node initialization. In International Joint Conference on Artificial Intelligence, volume 30, pages 2112-2118, 2021. URL: https://doi.org/10.24963/IJCAI.2021/291.
  2. Veeti Ahvonen, Damian Heiman, Antti Kuusisto, and Carsten Lutz. Logical characterizations of recurrent graph neural networks with reals and floats. In Advances in Neural Information Processing Systems, 2024. Google Scholar
  3. Pablo Barceló, Egor V. Kostylev, Mikaël Monet, Jorge Pérez, Juan Reutter, and Juan-Pablo Silva. The logical expressiveness of graph neural networks. In International Conference on Learning Representations, volume 8, 2020. Google Scholar
  4. Michael Benedikt, Chia-Hsuan Lu, Boris Motik, and Tony Tan. Decidability of graph neural networks via logical characterizations. In International Colloquium on Automata, Languages, and Programming, volume 297 of LIPIcs, pages 127:1-127:20, 2024. URL: https://doi.org/10.4230/LIPICS.ICALP.2024.127.
  5. Alex Fout, Jonathon Byrd, Basir Shariat, and Asa Ben-Hur. Protein interface prediction using graph convolutional networks. In Advances in Neural Information Processing Systems, volume 30, 2017. Google Scholar
  6. Floris Geerts and Juan Reutter. Expressiveness and approximation properties of graph neural networks. In International Conference on Learning Representations, volume 10, 2022. Google Scholar
  7. Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry. In International Conference on Machine Learning, pages 1263-1272, 2017. URL: http://proceedings.mlr.press/v70/gilmer17a.html.
  8. Martin Grohe. The descriptive complexity of graph neural networks. In ACM/IEEE Symposium on Logic in Computer Science, volume 38, pages 1-14. ACM, 2023. URL: https://doi.org/10.1109/LICS56636.2023.10175735.
  9. William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems 30, pages 1024-1034, 2017. URL: https://proceedings.neurips.cc/paper/2017/hash/5dd9db5e033da9c6fb5ba83c7a7ebea9-Abstract.html.
  10. Andreas Loukas. What graph neural networks cannot learn: Depth vs width. In International Conference on Learning Representations, volume 8, 2020. Google Scholar
  11. Christopher Morris, Martin Ritzert, Matthias Fey, William L. Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe. Weisfeiler and leman go neural: Higher-order graph neural networks. In AAAI Conference on Artificial Intelligence, volume 33, pages 4602-4609, 2019. URL: https://doi.org/10.1609/AAAI.V33I01.33014602.
  12. Martin Otto. Graded modal logic and counting bisimulation. arXiv preprint arXiv:1910.00039, 2019. URL: https://arxiv.org/abs/1910.00039.
  13. Maximilian Pflüger, David Tena Cucala, and Egor V. Kostylev. Recurrent graph neural networks and their connections to bisimulation and logic. In AAAI Conference on Artificial Intelligence, volume 38, 2024. Google Scholar
  14. Patrick Reiser, Marlen Neubert, André Eberhard, Luca Torresi, Chen Zhou, Chen Shao, Houssam Metni, Clint van Hoesel, Henrik Schopmans, Timo Sommer, et al. Graph neural networks for materials science and chemistry. Communications Materials, 3(1):93, 2022. Google Scholar
  15. Eran Rosenbluth, Jan Tönshoff, and Martin Grohe. Some might say all you need is sum. In International Joint Conference on Artificial Intelligence, volume 32, pages 4172-4179. International Joint Conferences on Artificial Intelligence Organization, 2023. URL: https://doi.org/10.24963/IJCAI.2023/464.
  16. Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE Transactions on Neural Networks, pages 61-80, 2008. Google Scholar
  17. Michael Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In European Semantic Web Conference, volume 15, pages 593-607. Springer, 2018. Google Scholar
  18. David Tena Cucala, Bernardo Cuenca Grau, Boris Motik, and Egor V. Kostylev. On the correspondence between monotonic max-sum gnns and datalog. In International Conference on Principles of Knowledge Representation and Reasoning, volume 20, 2023. Google Scholar
  19. Boris Weisfeiler and Andrei Leman. A reduction of a graph to a canonical form and an algebra arising during this reduction. Nauchno-Technicheskaya Informatsia, 2(9):12-16, 1968. Translated from Russian. Google Scholar
  20. Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In International Conference on Learning Representations, volume 7, 2019. Google Scholar
  21. Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L. Hamilton, and Jure Leskovec. Graph convolutional neural networks for web-scale recommender systems. In ACM/SIGKDD International Conference on Knowledge Discovery & Data Mining, volume 24, pages 974-983. ACM, 2018. URL: https://doi.org/10.1145/3219819.3219890.
  22. Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabás Póczos, Ruslan Salakhutdinov, and Alexander J. Smola. Deep sets. In Advances in Neural Information Processing Systems, pages 3391-3401, 2017. URL: https://proceedings.neurips.cc/paper/2017/hash/f22e4747da1aa27e363d86d40ff442fe-Abstract.html.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact