Description Complexity of Unary Structures in First-Order Logic with Links to Entropy

Authors Reijo Jaakkola , Antti Kuusisto , Miikka Vilander



PDF
Thumbnail PDF

File

LIPIcs.CSL.2025.17.pdf
  • Filesize: 0.76 MB
  • 20 pages

Document Identifiers

Author Details

Reijo Jaakkola
  • Mathematics Research Centre, Tampere University, Finland
Antti Kuusisto
  • Mathematics Research Centre, Tampere University, Finland
Miikka Vilander
  • Mathematics Research Centre, Tampere University, Finland

Cite As Get BibTex

Reijo Jaakkola, Antti Kuusisto, and Miikka Vilander. Description Complexity of Unary Structures in First-Order Logic with Links to Entropy. In 33rd EACSL Annual Conference on Computer Science Logic (CSL 2025). Leibniz International Proceedings in Informatics (LIPIcs), Volume 326, pp. 17:1-17:20, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/LIPIcs.CSL.2025.17

Abstract

The description complexity of a model is the length of the shortest formula that defines the model. We study the description complexity of unary structures in first-order logic FO, also drawing links to semantic complexity in the form of entropy. The class of unary structures provides, e.g., a simple way to represent tabular Boolean data sets as relational structures. We define structures with FO-formulas that are strictly linear in the size of the model as opposed to using the naive quadratic ones, and we use arguments based on formula size games to obtain related lower bounds for description complexity. For a typical structure the upper and lower bounds in fact match up to a sublinear term, leading to a precise asymptotic result on the expected description complexity of a randomly selected structure. We then give bounds on the relationship between Shannon entropy and description complexity. We extend this relationship also to Boltzmann entropy by establishing an asymptotic match between the two entropies. Despite the simplicity of unary structures, our arguments require the use of formula size games, Stirling’s approximation and Chernoff bounds.

Subject Classification

ACM Subject Classification
  • Theory of computation → Finite Model Theory
  • Mathematics of computing → Information theory
Keywords
  • formula size
  • finite model theory
  • formula size games
  • entropy
  • randomness

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads

References

  1. Micah Adler and Neil Immerman. An n! lower bound on formula size. ACM Trans. Comput. Log., 4(3):296-314, 2003. URL: https://doi.org/10.1145/772062.772064.
  2. Philippe Balbiani, David Fernández-Duque, Andreas Herzig, and Petar Iliev. Frame-validity games and lower bounds on the complexity of modal axioms. Log. J. IGPL, 30(1):155-185, 2022. URL: https://doi.org/10.1093/jigpal/jzaa068.
  3. Pablo Barceló, Mikaël Monet, Jorge Pérez, and Bernardo Subercaseaux. Model interpretability through the lens of computational complexity. In Hugo Larochelle, Marc'Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL: https://proceedings.neurips.cc/paper/2020/hash/b1adda14824f50ef24ff1c05bb66faf3-Abstract.html.
  4. Marco Carmosino, Ronald Fagin, Neil Immerman, Phokion Kolaitis, Jonathan Lenchner, and Rik Sengupta. On the number of quantifiers needed to define boolean functions, 2024. Google Scholar
  5. Marco Carmosino, Ronald Fagin, Neil Immerman, Phokion G. Kolaitis, Jonathan Lenchner, and Rik Sengupta. A finer analysis of multi-structural games and beyond. CoRR, abs/2301.13329, 2023. URL: https://doi.org/10.48550/arXiv.2301.13329.
  6. Marco Leandro Carmosino, Ronald Fagin, Neil Immerman, Ph. G. Kolaitis, Jonathan Lenchner, Rik Sengupta, and Ryan Williams. Parallel play saves quantifiers. ArXiv, 2024. Google Scholar
  7. Ronald Fagin, Jonathan Lenchner, Kenneth W. Regan, and Nikhil Vyas. Multi-structural games and number of quantifiers. In 2021 36th Annual ACM/IEEE Symposium on Logic in Computer Science (LICS), pages 1-13, 2021. URL: https://doi.org/10.1109/LICS52264.2021.9470756.
  8. Ronald Fagin, Jonathan Lenchner, Nikhil Vyas, and Ryan Williams. On the Number of Quantifiers as a Complexity Measure. In Stefan Szeider, Robert Ganian, and Alexandra Silva, editors, 47th International Symposium on Mathematical Foundations of Computer Science (MFCS 2022), volume 241 of Leibniz International Proceedings in Informatics (LIPIcs), pages 48:1-48:14, Dagstuhl, Germany, 2022. Schloss Dagstuhl - Leibniz-Zentrum für Informatik. URL: https://doi.org/10.4230/LIPIcs.MFCS.2022.48.
  9. Peter Grünwald and Paul M. B. Vitányi. Shannon information and Kolmogorov complexity. CoRR, cs.IT/0410002, 2004. URL: http://arxiv.org/abs/cs.IT/0410002, URL: https://doi.org/10.48550/arXiv.cs/0410002.
  10. Lauri Hella and Jouko Väänänen. The size of a formula as a measure of complexity. In Åsa Hirvonen, Juha Kontinen, Roman Kossak, and Andrés Villaveces, editors, Logic Without Borders - Essays on Set Theory, Model Theory, Philosophical Logic and Philosophy of Mathematics, volume 5 of Ontos Mathematical Logic, pages 193-214. De Gruyter, 2015. URL: https://doi.org/10.1515/9781614516873.193.
  11. Lauri Hella and Miikka Vilander. Formula size games for modal logic and μ-calculus. J. Log. Comput., 29(8):1311-1344, 2019. URL: https://doi.org/10.1093/logcom/exz025.
  12. Reijo Jaakkola, Tomi Janhunen, Antti Kuusisto, Masood Feyzbakhsh Rankooh, and Miikka Vilander. Explainability via short formulas: the case of propositional logic with implementation. In Joint Proceedings of (HYDRA 2022) and the RCRA Workshop on Experimental Evaluation of Algorithms for Solving Problems with Combinatorial Explosion, volume 3281 of CEUR Workshop Proceedings, pages 64-77, 2022. URL: https://ceur-ws.org/Vol-3281/paper6.pdf.
  13. Reijo Jaakkola, Tomi Janhunen, Antti Kuusisto, Masood Feyzbakhsh Rankooh, and Miikka Vilander. Short boolean formulas as explanations in practice. In Sarah Alice Gaggl, Maria Vanina Martinez, and Magdalena Ortiz, editors, Logics in Artificial Intelligence - 18th European Conference, JELIA 2023, Dresden, Germany, September 20-22, 2023, Proceedings, volume 14281 of Lecture Notes in Computer Science, pages 90-105. Springer, 2023. URL: https://doi.org/10.1007/978-3-031-43619-2_7.
  14. Reijo Jaakkola, Antti Kuusisto, and Miikka Vilander. Relating description complexity to entropy. In Petra Berenbrink, Patricia Bouyer, Anuj Dawar, and Mamadou Moustapha Kanté, editors, 40th International Symposium on Theoretical Aspects of Computer Science, STACS 2023, March 7-9, 2023, Hamburg, Germany, volume 254 of LIPIcs, pages 38:1-38:18. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2023. URL: https://doi.org/10.4230/LIPIcs.STACS.2023.38.
  15. Andrey Kolmogorov. The theory of transmission of information. In Selected Works of A. N. Kolmogorov: Volume III: Information Theory and the Theory of Algorithms, pages 6-32. Springer Netherlands, 1993. URL: https://doi.org/10.1007/978-94-017-2973-4_3.
  16. Sik K. Leung-Yan-Cheong and Thomas M. Cover. Some equivalences between Shannon entropy and Kolmogorov complexity. IEEE Trans. Inf. Theory, 24(3):331-338, 1978. URL: https://doi.org/10.1109/TIT.1978.1055891.
  17. Ming Li and Paul M. B. Vitányi. An Introduction to Kolmogorov Complexity and Its Applications, 4th Edition. Texts in Computer Science. Springer, 2019. URL: https://doi.org/10.1007/978-3-030-11298-1.
  18. João Marques-Silva, Thomas Gerspacher, Martin C. Cooper, Alexey Ignatiev, and Nina Narodytska. Explanations for monotonic classifiers. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 7469-7479. PMLR, 2021. URL: http://proceedings.mlr.press/v139/marques-silva21a.html.
  19. Michael Mitzenmacher and Eli Upfal. Probability and Computing: Randomized Algorithms and Probabilistic Analysis. Cambridge University Press, 2005. URL: https://doi.org/10.1017/CBO9780511813603.
  20. Alexander A. Razborov. Applications of matrix methods to the theory of lower bounds in computational complexity. Comb., 10(1):81-93, 1990. URL: https://doi.org/10.1007/BF02122698.
  21. Herbert Robbins. A remark on stirling’s formula. The American Mathematical Monthly, 62(1):26-29, 1955. URL: https://doi.org/10.2307/2308012.
  22. Advait Sarkar. Is explainable AI a race against model complexity? In Workshop on Transparency and Explanations in Smart Systems (TeXSS), in conjunction with ACM Intelligent User Interfaces (IUI 2022), volume 3124 of CEUR Workshop Proceedings, pages 192-199, 2022. Google Scholar
  23. Andreia Teixeira, Armando Matos, Andre Souto, and Luis Filipe Coelho Antunes. Entropy measures vs. Kolmogorov complexity. Entropy, 13(3):595-611, 2011. URL: https://doi.org/10.3390/e13030595.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail