Document Open Access Logo

Expressivity of Bisimulation Pseudometrics over Analytic State Spaces

Authors Daniel Luckhardt , Harsh Beohar , Clemens Kupke

PDF HTML 5

Files

Thumbnail PDF
PDF
LIPIcs.CALCO.2025.13.pdf
  • Filesize: 0.94 MB
  • 23 pages
HTML5 Logo HTML (experimental)
LIPIcs.CALCO.2025.13.html

Document Identifiers

Related Versions

Subject Classification

ACM Subject Classification
  • Theory of computation → Modal and temporal logics
Keywords
  • Markov decision process
  • quantitative Hennessy-Milner theorem

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

A Markov decision process (MDP) is a state-based dynamical system capable of describing probabilistic behaviour with rewards. In this paper, we view MDPs as coalgebras living in the category of analytic spaces, a very general class of measurable spaces. Note that analytic spaces were already studied in the literature on labelled Markov processes and bisimulation relations. Our results are twofold. First, we define bisimulation pseudometrics over such coalgebras using the framework of fibrations. Second, we develop a quantitative modal logic for such coalgebras and prove a quantitative form of Hennessy-Milner theorem in this new setting stating that the bisimulation pseudometric corresponds to the logical distance induced by modal formulae.

Cite As Get BibTex

Daniel Luckhardt, Harsh Beohar, and Clemens Kupke. Expressivity of Bisimulation Pseudometrics over Analytic State Spaces. In 11th Conference on Algebra and Coalgebra in Computer Science (CALCO 2025). Leibniz International Proceedings in Informatics (LIPIcs), Volume 342, pp. 13:1-13:23, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/LIPIcs.CALCO.2025.13

Author Details

Daniel Luckhardt
  • UCL, University College London, UK
Harsh Beohar
  • University of Sheffield, UK
Clemens Kupke
  • University of Strathclyde, Glasgow, UK

References

  1. Harsh Beohar, Sebastian Gurke, Barbara König, Karla Messing, Jonas Forster, Lutz Schröder, and Paul Wild. Expressive quantale-valued logics for coalgebras: An adjunction-based approach. In Olaf Beyersdorff, Mamadou Moustapha Kanté, Orna Kupferman, and Daniel Lokshtanov, editors, 41st International Symposium on Theoretical Aspects of Computer Science (STACS 2024), volume 289, pages 10:1-10:19, Dagstuhl, Germany, 2024. Schloss Dagstuhl - Leibniz-Zentrum für Informatik. URL: https://doi.org/10.4230/LIPIcs.STACS.2024.10.
  2. Harsh Beohar, Clemens Kupke, and Daniel Luckhardt. Henneysey-milner type theorem for measurable pseudometrics, 2025. arXiv:2505.23635 [cs.LO]. Google Scholar
  3. Vladimir I. Bogachev. Measures on topological spaces. Journal of Mathematical Sciences, 91(4):3033-3156, 1998. Google Scholar
  4. Filippo Bonchi, Barbara König, and Daniela Petrisan. Up-to techniques for behavioural metrics via fibrations. In Sven Schewe and Lijun Zhang, editors, 29th International Conference on Concurrency Theory (CONCUR 2018), volume 118 of Leibniz International Proceedings in Informatics (LIPIcs), pages 17:1-17:17, Dagstuhl, Germany, 2018. Schloss Dagstuhl - Leibniz-Zentrum für Informatik. URL: https://doi.org/10.4230/LIPIcs.CONCUR.2018.17.
  5. Craig Boutilier, Thomas Dean, and Steve Hanks. Decision-theoretic planning: structural assumptions and computational leverage. J. Artif. Int. Res., 11(1):1-94, July 1999. URL: https://doi.org/10.1613/JAIR.575.
  6. D. W. Bressler and M. Sion. The current theory of analytic sets. Canadian Journal of Mathematics, 16:207-230, 1964. URL: https://doi.org/10.4153/CJM-1964-021-7.
  7. Linan Chen, Florence Clerc, and Prakash Panangaden. A behavioural pseudometrics for continuous-time Markov processes. In Foundations of Software Science and Computation Structures, Lecture Notes in Computer Science, 2025. Accepted; arXiv:2501.13008 [cs.LO]; Event: FoSSaCS, Hamilton, Ontario, Canada, May 5-8, 2025. URL: https://doi.org/10.48550/arXiv.2501.13008.
  8. Josée Desharnais, Vineet Gupta, Radha Jagadeesan, and Prakash Panangaden. Metrics for labeled Markov systems. In Jos C. M. Baeten and Sjouke Mauw, editors, CONCUR'99 Concurrency Theory, pages 258-273, Berlin, Heidelberg, 1999. Springer. URL: https://doi.org/10.1007/3-540-48320-9_19.
  9. Josée Desharnais, Abbas Edalat, and Prakash Panangaden. Bisimulation for labelled Markov processes. Information and Computation, 179(2):163-193, 2002. URL: https://doi.org/10.1006/inco.2001.2962.
  10. Josée Desharnais, Vineet Gupta, Radha Jagadeesan, and Prakash Panangaden. Metrics for labelled Markov processes. Theoretical Computer Science, 318(3):323-354, 2004. URL: https://doi.org/10.1016/j.tcs.2003.09.013.
  11. Ernst-Erich Doberkat. Measures and all that - a tutorial, 2014. arXiv:1409.2662 [math.FA], Version 3. Google Scholar
  12. R. M. Dudley. Real Analysis and Probability. Number 74 in Cambridge Studies in Advanced Mathematics. Cambridge University Press, 2002. Google Scholar
  13. Nelson Dunford and Jacob Schwartz. Linear Operators, volume 3 of Pure and Applied Mathematics. Wiley-InterScience, 1958/1971. Google Scholar
  14. Ryszard Engelking. General topology, volume 6 of Sigma series in pure mathematics. Heldermann Verlag, revised and completed edition edition, 1989. Google Scholar
  15. Arnold M. Faden. The existence of regular conditional probabilities: Necessary and sufficient conditions. The Annals of Probability, 13(1):288-298, 1985. Google Scholar
  16. Neil Falkner. Generalizations of analytic and standard measurable spaces. Mathematica Scandinavica, pages 283-301, 1981. Google Scholar
  17. Norm Ferns, Prakash Panangaden, and Doina Precup. Metrics for finite Markov decision processes. In Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence (UAI 2004), pages 162-169, 2004. arXiv:1207.4114 [cs.AI]. URL: https://dslpitt.org/uai/displayArticleDetails.jsp?mmnu=1&smnu=2&article_id=1103&proceeding_id=20.
  18. Norm Ferns, Prakash Panangaden, and Doina Precup. Bisimulation metrics for continuous Markov decision processes. SIAM Journal on Computing, 40(6):1662-1714, 2011. URL: https://doi.org/10.1137/10080484X.
  19. Jonas Forster, Sergey Goncharov, Dirk Hofmann, Pedro Nora, Lutz Schröder, and Paul Wild. Quantitative Hennessy-Milner theorems via notions of density. In Bartek Klin and Elaine Pimentel, editors, 31st EACSL Annual Conference on Computer Science Logic (CSL 2023), volume 252, pages 22:1-22:20, Dagstuhl, Germany, 2023. Schloss Dagstuhl - Leibniz-Zentrum für Informatik. URL: https://doi.org/10.4230/LIPIcs.CSL.2023.22.
  20. D.H. Fremlin. Measure Theory, volume 5. Torres Fremlin, 2000/2008. Google Scholar
  21. Carles Gelada, Saurabh Kumar, Jacob Buckman, Ofir Nachum, and Marc G. Bellemare. DeepMDP: Learning continuous latent space models for representation learning. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 2170-2179. PMLR, June 2019. URL: https://proceedings.mlr.press/v97/gelada19a.html.
  22. Michèle Giry. A categorical approach to probability. In B. Banaschewski, editor, Categorical Aspects of Topology and Analysis, volume 915 of Lecture Notes in Mathematics, pages 68-85. Springer, 1982. Google Scholar
  23. Robert Givan, Thomas Dean, and Matthew Greig. Equivalence notions and model minimization in Markov decision processes. Artificial Intelligence, 147(1):163-223, 2003. Planning with Uncertainty and Incomplete Information. URL: https://doi.org/10.1016/S0004-3702(02)00376-4.
  24. Boris Vladimirovich Gnedenko and Andrey Nikolaevich Kolmogoroff. Predelnye raspredeleniya dlya summ nezavisimykh sluchaynykh velichin. GITTL, 1949. Google Scholar
  25. Matthew Hennessy and Robin Milner. On observing nondeterminism and concurrency. In Jaco de Bakker and Jan van Leeuwen, editors, Automata, Languages and Programming, pages 299-309, Berlin, Heidelberg, 1980. Springer Berlin Heidelberg. URL: https://doi.org/10.1007/3-540-10003-2_79.
  26. B. P. F. Jacobs. Categorical Logic and Type Theory. Number 141 in Studies in Logic and the Foundations of Mathematics. North Holland, Amsterdam, 1999. Google Scholar
  27. Alexander Kechris. Classical descriptive set theory, volume 156 of Graduate Texts in Mathematics. Springer Science & Business Media, 2012. Google Scholar
  28. Henning Kerstan. Coalgebraic Behavior Analysis: From Qualitative To Quantitative Analyses. PhD thesis, Universität Duisburg-Essen, May 2016. Submitted on 2016-05-09. URL: https://duepublico2.uni-due.de/receive/duepublico_mods_00041220.
  29. Yuichi Komorida, Shin-ya Katsumata, Clemens Kupke, Jurriaan Rot, and Ichiro Hasuo. Expressivity of quantitative modal logics: Categorical foundations via codensity and approximation. In Proceedings of the Thirty Sixth Annual IEEE Symposium on Logic in Computer Science (LICS 2021), pages 1-14, Rome, Italy, June 2021. IEEE Computer Society Press. URL: https://doi.org/10.1109/LICS52264.2021.9470656.
  30. Clemens Kupke and Jurriaan Rot. Expressive Logics for Coinductive Predicates. In Maribel Fernández and Anca Muscholl, editors, 28th EACSL Annual Conference on Computer Science Logic (CSL 2020), volume 152 of Leibniz International Proceedings in Informatics (LIPIcs), pages 26:1-26:18, Dagstuhl, Germany, 2020. Schloss Dagstuhl - Leibniz-Zentrum für Informatik. URL: https://doi.org/10.4230/LIPIcs.CSL.2020.26.
  31. Jan K Pachl. Two classes of measures. Colloquium Mathematicum, 42(1):331-340, 1979. Erratum in Vol. 45.2 (1981), pp. 331-333. Google Scholar
  32. D. Ramachandran and L. Rüschendorf. On the monge–kantorovich duality theorem. Teoriya veroyatnostey i ee primeneniya, 45(2), 2000. Google Scholar
  33. Doraiswamy Ramachandran and Ludger Rüschendorf. A general duality theorem for marginal problems. Probability Theory and Related Fields, 101:311-319, 1995. Google Scholar
  34. Czesław Ryll-Nardzewski. On quasi-compact measures. Fundamenta Mathematicae, 40:125-130, 1953. Google Scholar
  35. Stephen Simons. Minimax Theorems and Their Proofs, chapter 1, pages 1-23. Number 5 in Nonconvex Optimization and Its Applications. Springer, 1995. URL: https://doi.org/10.1007/978-1-4613-3557-3.
  36. M. Ya. Suslin. Sur une définition des ensembles mesurables B sans nombres transfinis. Comptes Rendus Mathématique. Académie des Sciences. Paris., 164(2):88-91, 1917. Google Scholar
  37. Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. A Bradford Book, Cambridge, MA, USA, 2018. Google Scholar
  38. Franck van Breugel and James Worrell. An algorithm for quantitative verification of probabilistic transition systems. In Kim G. Larsen and Mogens Nielsen, editors, CONCUR 2001 - Concurrency Theory, pages 336-350, Berlin, Heidelberg, 2001. Springer Berlin Heidelberg. URL: https://doi.org/10.1007/3-540-44685-0_23.
  39. Franck van Breugel and James Worrell. Towards quantitative verification of probabilistic transition systems. In Fernando Orejas, Paul G. Spirakis, and Jan van Leeuwen, editors, Automata, Languages and Programming, pages 421-432, Berlin, Heidelberg, 2001. Springer Berlin Heidelberg. URL: https://doi.org/10.1007/3-540-48224-5_35.
  40. Eric K. van Douwen. The integers and topology. In Handbook of set-theoretic topology, pages 111-167. Elsevier, 1984. Google Scholar
  41. Ignacio D. Viglizzo. Final sequences and final coalgebras for measurable spaces. In Algebra and Coalgebra in Computer Science, pages 395-407, Berlin, Heidelberg, 2005. Springer. URL: https://doi.org/10.1007/11548133_25.
  42. Itaï Ben Yaacov, Alexander Berenstein, C. Ward Henson, and Alexander Usvyatsov. Model theory for metric structures. In Zoé Chatzidakis, Dugald Macpherson, Anand Pillay, and Alex Wilkie, editors, Model Theory with Applications to Algebra and Analysis, volume 350 of London Mathematical Society Lecture Note Series, pages 315-427. Cambridge University Press, Cambridge, 2008. Google Scholar
  43. Amy Zhang, Rowan Thomas McAllister, Roberto Calandra, Yarin Gal, and Sergey Levine. Learning invariant representations for reinforcement learning without reconstruction. In International Conference on Learning Representations, 2021. URL: https://openreview.net/forum?id=-2FCwDKRREu.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact