LIPIcs.MFCS.2023.24.pdf
- Filesize: 0.74 MB
- 15 pages
Document
The Compositional Structure of Bayesian Inference
Authors
-
Part of:
Volume:
48th International Symposium on Mathematical Foundations of Computer Science (MFCS 2023)
Series: Leibniz International Proceedings in Informatics (LIPIcs)
Conference: Mathematical Foundations of Computer Science (MFCS) - License:
Creative Commons Attribution 4.0 International license
- Publication Date: 2023-08-21
Cite AsGet BibTex
Dylan Braithwaite, Jules Hedges, and Toby St Clere Smithe. The Compositional Structure of Bayesian Inference. In 48th International Symposium on Mathematical Foundations of Computer Science (MFCS 2023). Leibniz International Proceedings in Informatics (LIPIcs), Volume 272, pp. 24:1-24:15, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2023)
https://doi.org/10.4230/LIPIcs.MFCS.2023.24
https://doi.org/10.4230/LIPIcs.MFCS.2023.24
Abstract
Bayes' rule tells us how to invert a causal process in order to update our beliefs in light of new evidence. If the process is believed to have a complex compositional structure, we may observe that the inversion of the whole can be computed piecewise in terms of the component processes. We study the structure of this compositional rule, noting that it relates to the lens pattern in functional programming. Working in a suitably general axiomatic presentation of a category of Markov kernels, we see how we can think of Bayesian inversion as a particular instance of a state-dependent morphism in a fibred category. We discuss the compositional nature of this, formulated as a functor on the underlying category and explore how this can used for a more type-driven approach to statistical inference.
Subject Classification
ACM Subject Classification
- Theory of computation → Categorical semantics
- Mathematics of computing → Probabilistic representations
- Mathematics of computing → Bayesian computation
Keywords
- monoidal categories
- probabilistic programming
- Bayesian inference
Metrics
- Access Statistics
-
Total Accesses (updated on a weekly basis)
0PDF Downloads
References
- Andre M. Bastos, W. Martin Usrey, Rick A. Adams, George R. Mangun, Pascal Fries, and Karl J. Friston. Canonical microcircuits for predictive coding. Neuron, 2012. URL: https://doi.org/10.1016/j.neuron.2012.10.038.
- Francis Borceux. Handbook of Categorical Algebra 2. Categories and Structures. Cambridge University Press, 1994. URL: https://doi.org/10.1017/CBO9780511525865.
- Dylan Braithwaite and Jules Hedges. Dependent Bayesian Lenses: Categories of Bidirectional Markov Kernels with Canonical Bayesian Inversion, 2022. URL: https://arxiv.org/abs/2209.14728.
- Matteo Capucci, Bruno Gavranović, Jules Hedges, and Eigil Fjeldgren Rischel. Towards Foundations of Categorical Cybernetics. ACT21, 2022. URL: https://doi.org/10.4204/EPTCS.372.17.
- Kenta Cho and Bart Jacobs. Disintegration and Bayesian inversion via string diagrams. MSCS, 2019. URL: https://doi.org/10.1017/S0960129518000488.
- Geoffrey S. H. Cruttwell, Bruno Gavranović, Neil Ghani, Paul Wilson, and Fabio Zanasi. Categorical foundations of gradient-based learning. ESOP22, 2022. URL: https://doi.org/10.1007/978-3-030-99336-8_1.
- J. Nathan Foster, Michael B. Greenwald, Jonathan T. Moore, Benjamin C. Pierce, and Alan Schmitt. Combinators for bidirectional tree transformations: A linguistic approach to the view-update problem. TOPLAS, 2007. URL: https://doi.org/10.1145/1232420.1232424.
- Tobias Fritz. A synthetic approach to Markov kernels, conditional independence and theorems on sufficient statistics. Advances in Mathematics, 2020. URL: https://doi.org/10.1016/j.aim.2020.107239.
- Neil Ghani, Jules Hedges, Viktor Winschel, and Philipp Zahn. Compositional game theory. LICS18, 2018. URL: https://doi.org/10.1145/3209108.3209165.
- Michèlle Giry. A categorical approach to probability theory. Categorical aspects of Topology and Analysis, 1982. URL: https://doi.org/10.1007/BFb0092872.
-
Jules Hedges and Riu Rodríguez Sakamoto. Value Iteration Is Optic Composition. ACT22, 2023. Forthcoming.
-
Bart Jacobs. Categorical Logic and Type Theory. Elsevier Science, 1999.
- Beren Millidge, Tommaso Salvatori, Yuhang Song, Rafal Bogacz, and Thomas Lukasiewicz. Predictive coding: Towards a future of deep learning beyond backpropagation? IJCAI22, 2022. URL: https://doi.org/10.24963/ijcai.2022/774.
-
Joe Moeller and Christina Vasilakopoulou. Monoidal Grothendieck construction, 2020.
- Robert Rosenbaum. On the relationship between predictive coding and backpropagation. PLoS ONE, 2022. URL: https://doi.org/10.1371/journal.pone.0266102.
- Toby St. Clere Smithe. Bayesian updates compose optically, 2020. URL: https://arxiv.org/abs/2006.01631.
- Toby St. Clere Smithe. Compositional Active Inference I: Bayesian Lenses. Statistical Games, 2022. URL: https://arxiv.org/abs/2109.04461.
- Toby St Clere Smithe. Mathematical Foundations for a Compositional Account of the Bayesian Brain. PhD thesis, University of Oxford, 2023. URL: https://arxiv.org/abs/2212.12538.
- David I. Spivak. Generalized Lens Categories via functors 𝒞^ op → Cat, 2019. URL: https://arxiv.org/abs/2109.04461.
- Jan-Willem van de Meent, Brooks Paige, Hongseok Yang, and Frank Wood. An Introduction to Probabilistic Programming, 2021. URL: https://arxiv.org/abs/1809.10756.
- H. Wu and F. Noé. Maximum a posteriori estimation for markov chains based on gaussian markov random fields. ICCS10, 2010. URL: https://doi.org/10.1016/j.procs.2010.04.186.