Document Open Access Logo

The Compositional Structure of Bayesian Inference

Authors Dylan Braithwaite, Jules Hedges, Toby St Clere Smithe

PDF
Thumbnail PDF

File

LIPIcs.MFCS.2023.24.pdf
  • Filesize: 0.74 MB
  • 15 pages

Document Identifiers

Author Details

Dylan Braithwaite
  • Department of Computer and Information Sciences, University of Strathclyde, Glasgow, UK
Jules Hedges
  • Department of Computer and Information Sciences, University of Strathclyde, Glasgow, UK
Toby St Clere Smithe
  • Topos Institute, Berkeley, CA, USA
  • Department of Experimental Psychology, University of Oxford, UK

Funding

  • Braithwaite, Dylan: This author is funded through an NPL Industrial CASE Studentship.

Cite As Get BibTex

Dylan Braithwaite, Jules Hedges, and Toby St Clere Smithe. The Compositional Structure of Bayesian Inference. In 48th International Symposium on Mathematical Foundations of Computer Science (MFCS 2023). Leibniz International Proceedings in Informatics (LIPIcs), Volume 272, pp. 24:1-24:15, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2023) https://doi.org/10.4230/LIPIcs.MFCS.2023.24

Abstract

Bayes' rule tells us how to invert a causal process in order to update our beliefs in light of new evidence. If the process is believed to have a complex compositional structure, we may observe that the inversion of the whole can be computed piecewise in terms of the component processes. We study the structure of this compositional rule, noting that it relates to the lens pattern in functional programming. Working in a suitably general axiomatic presentation of a category of Markov kernels, we see how we can think of Bayesian inversion as a particular instance of a state-dependent morphism in a fibred category. We discuss the compositional nature of this, formulated as a functor on the underlying category and explore how this can used for a more type-driven approach to statistical inference.

Subject Classification

ACM Subject Classification
  • Theory of computation → Categorical semantics
  • Mathematics of computing → Probabilistic representations
  • Mathematics of computing → Bayesian computation
Keywords
  • monoidal categories
  • probabilistic programming
  • Bayesian inference

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Related Versions

  • This paper contains material from unpublished preprints [Toby St. Clere Smithe, 2020] and [Dylan Braithwaite and Jules Hedges, 2022].
  • Extended Version https://arxiv.org/abs/2305.06112

References

  1. Andre M. Bastos, W. Martin Usrey, Rick A. Adams, George R. Mangun, Pascal Fries, and Karl J. Friston. Canonical microcircuits for predictive coding. Neuron, 2012. URL: https://doi.org/10.1016/j.neuron.2012.10.038.
  2. Francis Borceux. Handbook of Categorical Algebra 2. Categories and Structures. Cambridge University Press, 1994. URL: https://doi.org/10.1017/CBO9780511525865.
  3. Dylan Braithwaite and Jules Hedges. Dependent Bayesian Lenses: Categories of Bidirectional Markov Kernels with Canonical Bayesian Inversion, 2022. URL: https://arxiv.org/abs/2209.14728.
  4. Matteo Capucci, Bruno Gavranović, Jules Hedges, and Eigil Fjeldgren Rischel. Towards Foundations of Categorical Cybernetics. ACT21, 2022. URL: https://doi.org/10.4204/EPTCS.372.17.
  5. Kenta Cho and Bart Jacobs. Disintegration and Bayesian inversion via string diagrams. MSCS, 2019. URL: https://doi.org/10.1017/S0960129518000488.
  6. Geoffrey S. H. Cruttwell, Bruno Gavranović, Neil Ghani, Paul Wilson, and Fabio Zanasi. Categorical foundations of gradient-based learning. ESOP22, 2022. URL: https://doi.org/10.1007/978-3-030-99336-8_1.
  7. J. Nathan Foster, Michael B. Greenwald, Jonathan T. Moore, Benjamin C. Pierce, and Alan Schmitt. Combinators for bidirectional tree transformations: A linguistic approach to the view-update problem. TOPLAS, 2007. URL: https://doi.org/10.1145/1232420.1232424.
  8. Tobias Fritz. A synthetic approach to Markov kernels, conditional independence and theorems on sufficient statistics. Advances in Mathematics, 2020. URL: https://doi.org/10.1016/j.aim.2020.107239.
  9. Neil Ghani, Jules Hedges, Viktor Winschel, and Philipp Zahn. Compositional game theory. LICS18, 2018. URL: https://doi.org/10.1145/3209108.3209165.
  10. Michèlle Giry. A categorical approach to probability theory. Categorical aspects of Topology and Analysis, 1982. URL: https://doi.org/10.1007/BFb0092872.
  11. Jules Hedges and Riu Rodríguez Sakamoto. Value Iteration Is Optic Composition. ACT22, 2023. Forthcoming. Google Scholar
  12. Bart Jacobs. Categorical Logic and Type Theory. Elsevier Science, 1999. Google Scholar
  13. Beren Millidge, Tommaso Salvatori, Yuhang Song, Rafal Bogacz, and Thomas Lukasiewicz. Predictive coding: Towards a future of deep learning beyond backpropagation? IJCAI22, 2022. URL: https://doi.org/10.24963/ijcai.2022/774.
  14. Joe Moeller and Christina Vasilakopoulou. Monoidal Grothendieck construction, 2020. Google Scholar
  15. Robert Rosenbaum. On the relationship between predictive coding and backpropagation. PLoS ONE, 2022. URL: https://doi.org/10.1371/journal.pone.0266102.
  16. Toby St. Clere Smithe. Bayesian updates compose optically, 2020. URL: https://arxiv.org/abs/2006.01631.
  17. Toby St. Clere Smithe. Compositional Active Inference I: Bayesian Lenses. Statistical Games, 2022. URL: https://arxiv.org/abs/2109.04461.
  18. Toby St Clere Smithe. Mathematical Foundations for a Compositional Account of the Bayesian Brain. PhD thesis, University of Oxford, 2023. URL: https://arxiv.org/abs/2212.12538.
  19. David I. Spivak. Generalized Lens Categories via functors 𝒞^ op → Cat, 2019. URL: https://arxiv.org/abs/2109.04461.
  20. Jan-Willem van de Meent, Brooks Paige, Hongseok Yang, and Frank Wood. An Introduction to Probabilistic Programming, 2021. URL: https://arxiv.org/abs/1809.10756.
  21. H. Wu and F. Noé. Maximum a posteriori estimation for markov chains based on gaussian markov random fields. ICCS10, 2010. URL: https://doi.org/10.1016/j.procs.2010.04.186.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact