Document Open Access Logo

Some Semantic Issues in Probabilistic Programming Languages (Invited Talk)

Author Hongseok Yang

PDF
Thumbnail PDF

File

LIPIcs.FSCD.2019.4.pdf
  • Filesize: 351 kB
  • 6 pages

Document Identifiers

Author Details

Hongseok Yang
  • School of Computing, KAIST, South Korea

Cite AsGet BibTex

Hongseok Yang. Some Semantic Issues in Probabilistic Programming Languages (Invited Talk). In 4th International Conference on Formal Structures for Computation and Deduction (FSCD 2019). Leibniz International Proceedings in Informatics (LIPIcs), Volume 131, pp. 4:1-4:6, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2019)
https://doi.org/10.4230/LIPIcs.FSCD.2019.4

Abstract

This is a slightly extended abstract of my talk at FSCD'19 about probabilistic programming and a few semantic issues on it. The main purpose of this abstract is to provide keywords and references on the work mentioned in my talk, and help interested audience to do follow-up study.

Subject Classification

ACM Subject Classification
  • Theory of computation → Probabilistic computation
  • Theory of computation → Program semantics
  • Theory of computation → Denotational semantics
  • Mathematics of computing → Bayesian nonparametric models
  • Mathematics of computing → Bayesian computation
Keywords
  • Probabilistic Programming
  • Denotational Semantics
  • Non-differentiable Models
  • Bayesian Nonparametrics
  • Exchangeability

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

References

  1. R. J. Aumann. Borel structures for function spaces. Illinois Journal of Mathematics, 5:614-630, 1961. Google Scholar
  2. Sooraj Bhat, Johannes Borgström, Andrew D. Gordon, and Claudio V. Russo. Deriving Probability Density Functions from Probabilistic Functional Programs. Logical Methods in Computer Science, 13(2), 2017. Google Scholar
  3. Eli Bingham, Jonathan P. Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul A. Szerlip, Paul Horsfall, and Noah D. Goodman. Pyro: Deep Universal Probabilistic Programming. Journal of Machine Learning Research, 20:28:1-28:6, 2019. Google Scholar
  4. J. Borgström, A. D. Gordon, M. Greenberg, J. Margetson, and J. van Gael. Measure Transformer Semantics for Bayesian Machine Learning. LMCS, 9(3):11, 2013. Google Scholar
  5. Johannes Borgström, Ugo Dal Lago, Andrew D. Gordon, and Marcin Szymczak. A lambda-calculus foundation for universal probabilistic programming. In ICFP, pages 33-46, 2016. Google Scholar
  6. Cameron Davidson-Pilon. Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference. Addison-Wesley Professional, 2015. Google Scholar
  7. Hong Ge, Kai Xu, and Zoubin Ghahramani. Turing: Composable inference for probabilistic programming. In AISTATS, pages 1682-1690, 2018. Google Scholar
  8. Andrew Gelman, Daniel Lee, and Jiqiang Guo. Stan: A Probabilistic Programming Language for Bayesian Inference and Optimization. Journal of Educational and Behavioral Statistics, 40(5):530-543, 2015. Google Scholar
  9. Noah D. Goodman, Vikash K. Mansinghka, Daniel M. Roy, Keith Bonawitz, and Joshua B. Tenenbaum. Church: A Language for Generative Models. In UAI, pages 220-229, 2008. Google Scholar
  10. Noah D Goodman and Andreas Stuhlmüller. The Design and Implementation of Probabilistic Programming Languages. http://dippl.org, 2014. Accessed: 2019-4-11.
  11. Andrew D Gordon, Thomas A Henzinger, Aditya V Nori, and Sriram K Rajamani. Probabilistic programming. In Proceedings of the on Future of Software Engineering, FOSE 2014, pages 167-181, 2014. Google Scholar
  12. Chris Heunen, Ohad Kammar, Sam Staton, and Hongseok Yang. A convenient category for higher-order probability theory. In LICS, pages 1-12, 2017. Google Scholar
  13. Daniel Huang and Greg Morrisett. An Application of Computable Distributions to the Semantics of Probabilistic Programming Languages. In ESOP, pages 337-363, 2016. Google Scholar
  14. Chung-Kil Hur, Aditya V. Nori, Sriram K. Rajamani, and Selva Samuel. A Provably Correct Sampler for Probabilistic Programs. In FSTTCS, pages 475-488, 2015. Google Scholar
  15. Paul Jung, Jiho Lee, Sam Staton, and Hongseok Yang. A Generalization of Hierarchical Exchangeability on Trees to Directed Acyclic Graphs. arXiv preprint, 2018. URL: http://arxiv.org/abs/1812.06282.
  16. Olav Kallenberg. Probabilistic Symmetries and Invariance Principles. Springer, 2005. Google Scholar
  17. D. Kozen. Semantics of Probablistic Programs. Journal of Computer and System Sciences, 22:328-350, 1981. Google Scholar
  18. Wonyeol Lee, Hangyeol Yu, and Hongseok Yang. Reparameterization Gradient for Non-differentiable Models. In NeurIPS, pages 5558-5568, 2018. Google Scholar
  19. Vikash Mansinghka, Daniel Selsam, and Yura Perov. Venture: a higher-order probabilistic programming platform with programmable inference. arXiv preprint, 2014. URL: http://arxiv.org/abs/1404.0099.
  20. Praveen Narayanan, Jacques Carette, Wren Romano, Chung-chieh Shan, and Robert Zinkov. Probabilistic inference by program transformation in Hakaru (system description). In Proceedings of the 13th International Symposium on Functional and Logic Programming, FLOPS 2016, pages 62-79, 2016. Google Scholar
  21. Akihiko Nishimura, David Dunson, and Jianfeng Lu. Discontinuous Hamiltonian Monte Carlo for Sampling Discrete Parameters. arXiv preprint, 2017. URL: http://arxiv.org/abs/1705.08510.
  22. Peter Orbanz and Daniel M. Roy. Bayesian Models of Graphs, Arrays and Other Exchangeable Random Structures. IEEE Trans. Pattern Anal. Mach. Intell., 37(2):437-461, 2015. Google Scholar
  23. Anand Patil, David Huard, and Christopher J Fonnesbeck. PyMC: Bayesian Stochastic Modelling in Python. Journal of Statistical Software, 35(4):1, 2010. Google Scholar
  24. Daniel M. Roy, Vikash Mansinghka, Noah Goodman, and Joshua Tenenbaum. A stochastic programming perspective on nonparametric Bayes. In ICML Workshop on Nonparametric Bayesian, 2008. Google Scholar
  25. Adam Scibior, Ohad Kammar, Matthijs Vákár, Sam Staton, Hongseok Yang, Yufei Cai, Klaus Ostermann, Sean K. Moss, Chris Heunen, and Zoubin Ghahramani. Denotational validation of higher-order Bayesian inference. PACMPL, 2(POPL):60:1-60:29, 2018. Google Scholar
  26. N. Siddharth, Brooks Paige, Jan-Willem van de Meent, Alban Desmaison, Noah D. Goodman, Pushmeet Kohli, Frank Wood, and Philip Torr. Learning Disentangled Representations with Semi-Supervised Deep Generative Models. In NIPS, pages 5927-5937, 2017. Google Scholar
  27. Sam Staton, Dario Stein, Hongseok Yang, Nathanael L. Ackerman, Cameron Freer, and Daniel M Roy. The Beta-Bernoulli Process and Algebraic Effects. In ICALP, 2018. Google Scholar
  28. Sam Staton, Hongseok Yang, Nathanael L. Ackerman, Cameron Freer, and Daniel M Roy. Exchangeable random process and data abstraction. In Workshop on Probabilistic Programming Semantics, PPS 2017, 2017. Google Scholar
  29. Sam Staton, Hongseok Yang, Chris Heunen, Ohad Kammar, and Frank Wood. Semantics for probabilistic programming: higher-order functions, continuous distributions, and soft constraints. In LICS, pages 525-534, 2016. Google Scholar
  30. David Tolpin, Jan-Willem van de Meent, Hongseok Yang, and Frank D. Wood. Design and Implementation of Probabilistic Programming Language Anglican. In Proceedings of the 28th Symposium on the Implementation and Application of Functional Programming Languages, IFL 2016, pages 6:1-6:12, 2016. Google Scholar
  31. Dustin Tran, Matthew D. Hoffman, Dave Moore, Christopher Suter, Srinivas Vasudevan, and Alexey Radul. Simple, Distributed, and Accelerated Probabilistic Programming. In NeurIPS, pages 7609-7620, 2018. Google Scholar
  32. Dustin Tran, Alp Kucukelbir, Adji B. Dieng, Maja R. Rudolph, Dawen Liang, and David M. Blei. Edward: A library for probabilistic modeling, inference, and criticism. CoRR, abs/1610.09787, 2016. Google Scholar
  33. Matthijs Vákár, Ohad Kammar, and Sam Staton. A domain theory for statistical probabilistic programming. PACMPL, 3(POPL):36:1-36:29, 2019. Google Scholar
  34. Jan-Willem van de Meent, Brooks Paige, Hongseok Yang, and Frank Wood. An Introduction to Probabilistic Programming. arXiv preprint, 2018. URL: http://arxiv.org/abs/1809.10756.
  35. Frank Wood, Jan Willem Meent, and Vikash Mansinghka. A New Approach to Probabilistic Programming Inference. In AISTATS, pages 1024-1032, 2014. Google Scholar
  36. Yuan Zhou, Bradley Gram-Hansen, Tobias Kohn, Tom Rainforth, Hongseok Yang, and Frank Wood. A Low-Level Probabilistic Programming Language for Non-Differentiable Models. In AISTATS, 2019. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH Imprint Privacy Contact