Document Open Access Logo

Descriptive Complexity for Neural Networks via Boolean Networks

Authors Veeti Ahvonen , Damian Heiman , Antti Kuusisto

PDF
Thumbnail PDF

File

LIPIcs.CSL.2024.9.pdf
  • Filesize: 0.8 MB
  • 22 pages

Document Identifiers

Author Details

Veeti Ahvonen
  • Tampere University, Finland
Damian Heiman
  • Tampere University, Finland
Antti Kuusisto
  • Tampere University, Finland

Funding

Antti Kuusisto was supported by the Academy of Finland project Theory of computational logics, grant numbers 352419, 352420, 353027, 324435, 328987. Damian Heiman was supported by the same project, grant number 353027. Antti Kuusisto was also supported by the Academy of Finland consortium project Explaining AI via Logic (XAILOG), grant number 345612. Veeti Ahvonen was supported by the Vilho, Yrjö and Kalle Väisälä Foundation of the Finnish Academy of Science and Letters.

Cite AsGet BibTex

Veeti Ahvonen, Damian Heiman, and Antti Kuusisto. Descriptive Complexity for Neural Networks via Boolean Networks. In 32nd EACSL Annual Conference on Computer Science Logic (CSL 2024). Leibniz International Proceedings in Informatics (LIPIcs), Volume 288, pp. 9:1-9:22, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2024)
https://doi.org/10.4230/LIPIcs.CSL.2024.9

Abstract

We investigate the descriptive complexity of a class of neural networks with unrestricted topologies and piecewise polynomial activation functions. We consider the general scenario where the running time is unlimited and floating-point numbers are used for simulating reals. We characterize these neural networks with a rule-based logic for Boolean networks. In particular, we show that the sizes of the neural networks and the corresponding Boolean rule formulae are polynomially related. In fact, in the direction from Boolean rules to neural networks, the blow-up is only linear. We also analyze the delays in running times due to the translations. In the translation from neural networks to Boolean rules, the time delay is polylogarithmic in the neural network size and linear in time. In the converse translation, the time delay is linear in both factors. We also obtain translations between the rule-based logic for Boolean networks, the diamond-free fragment of modal substitution calculus and a class of recursive Boolean circuits where the number of input and output gates match.

Subject Classification

ACM Subject Classification
  • Computing methodologies → Neural networks
  • Theory of computation → Finite Model Theory
  • Mathematics of computing → Numerical analysis
  • Computer systems organization → Parallel architectures
Keywords
  • Descriptive complexity
  • neural networks
  • Boolean networks
  • floating-point arithmetic
  • logic

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Related Versions

References

  1. Veeti Ahvonen, Damian Heiman, Lauri Hella, and Antti Kuusisto. Descriptive complexity for distributed computing with circuits. In Jérôme Leroux, Sylvain Lombardy, and David Peleg, editors, 48th International Symposium on Mathematical Foundations of Computer Science, MFCS 2023, August 28 to September 1, 2023, Bordeaux, France, volume 272 of LIPIcs, pages 9:1-9:15. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2023. URL: https://doi.org/10.4230/LIPICS.MFCS.2023.9.
  2. Veeti Ahvonen, Damian Heiman, and Antti Kuusisto. Descriptive complexity for neural networks via boolean networks. CoRR, abs/2308.06277, 2023. URL: https://doi.org/10.48550/arXiv.2308.06277.
  3. Pablo Barceló, Egor V. Kostylev, Mikaël Monet, Jorge Pérez, Juan L. Reutter, and Juan Pablo Silva. The logical expressiveness of graph neural networks. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. Google Scholar
  4. Maria Luisa Bonet and Samuel R. Buss. Size-depth tradeoffs for boolean formulae. Information Processing Letters, 49(3):151-155, 1994. URL: https://doi.org/10.1016/0020-0190(94)90093-0.
  5. Daizhan Cheng and Hongsheng Qi. A linear representation of dynamics of boolean networks. IEEE Transactions on Automatic Control, 55(10):2251-2258, 2010. Google Scholar
  6. Martin Grohe. The logic of graph neural networks. In 36th Annual ACM/IEEE Symposium on Logic in Computer Science, LICS 2021, Rome, Italy, June 29 - July 2, 2021, pages 1-17. IEEE, 2021. Google Scholar
  7. Martin Grohe. The descriptive complexity of graph neural networks, 2023. URL: https://arxiv.org/abs/2303.04613.
  8. Lauri Hella, Matti Järvisalo, Antti Kuusisto, Juhana Laurinharju, Tuomo Lempiäinen, Kerkko Luosto, Jukka Suomela, and Jonni Virtema. Weak models of distributed computing, with connections to modal logic. In Proceedings of the 2012 ACM Symposium on Principles of distributed computing, pages 185-194, 2012. Google Scholar
  9. Lauri Hella, Matti Järvisalo, Antti Kuusisto, Juhana Laurinharju, Tuomo Lempiäinen, Kerkko Luosto, Jukka Suomela, and Jonni Virtema. Weak models of distributed computing, with connections to modal logic. Distributed Comput., 28(1):31-53, 2015. Google Scholar
  10. Stuart Kauffman. Homeostasis and differentiation in random genetic control networks. Nature, 224(5215):177-178, 1969. Google Scholar
  11. Antti Kuusisto. Modal Logic and Distributed Message Passing Automata. In Computer Science Logic 2013 (CSL 2013), volume 23 of Leibniz International Proceedings in Informatics (LIPIcs), pages 452-468, 2013. Google Scholar
  12. Leonid Libkin. Elements of Finite Model Theory. Texts in Theoretical Computer Science. An EATCS Series. Springer, 2004. Google Scholar
  13. Fabian Reiter. Asynchronous distributed automata: A characterization of the modal mu-fragment. In Ioannis Chatzigiannakis, Piotr Indyk, Fabian Kuhn, and Anca Muscholl, editors, 44th International Colloquium on Automata, Languages, and Programming, ICALP 2017, July 10-14, 2017, Warsaw, Poland, volume 80 of LIPIcs, pages 100:1-100:14, 2017. Google Scholar
  14. Julian D Schwab, Silke D Kühlwein, Nensi Ikonomi, Michael Kühl, and Hans A Kestler. Concepts in boolean network modeling: What do they all mean? Computational and structural biotechnology journal, 18:571-582, 2020. Google Scholar
  15. Massimiliano Zanin and Alexander N Pisarchik. Boolean networks for cryptography and secure communication. Nonlinear Science Letters B: Chaos, Fractal and Synchronization. Vol, 1(1):27-34, 2011. Google Scholar
  16. Ranran Zhang, Mithun Vinod Shah, Jun Yang, Susan B Nyland, Xin Liu, Jong K Yun, Réka Albert, and Thomas P Loughran Jr. Network model of survival signaling in large granular lymphocyte leukemia. Proceedings of the National Academy of Sciences, 105(42):16308-16313, 2008. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH Imprint Privacy Contact