Document Open Access Logo

Convergence of Gibbs Sampling: Coordinate Hit-And-Run Mixes Fast

Authors Aditi Laddha , Santosh S. Vempala

PDF

File

Thumbnail PDF
PDF
LIPIcs.SoCG.2021.51.pdf
  • Filesize: 0.79 MB
  • 12 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Mathematics of computing → Markov processes
Keywords
  • Gibbs Sampler
  • Coordinate Hit and run
  • Mixing time of Markov Chain

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

The Gibbs Sampler is a general method for sampling high-dimensional distributions, dating back to 1971. In each step of the Gibbs Sampler, we pick a random coordinate and re-sample that coordinate from the distribution induced by fixing all the other coordinates. While it has become widely used over the past half-century, guarantees of efficient convergence have been elusive. We show that for a convex body K in ℝⁿ with diameter D, the mixing time of the Coordinate Hit-and-Run (CHAR) algorithm on K is polynomial in n and D. We also give a lower bound on the mixing rate of CHAR, showing that it is strictly worse than hit-and-run and the ball walk in the worst case.

Cite As Get BibTex

Aditi Laddha and Santosh S. Vempala. Convergence of Gibbs Sampling: Coordinate Hit-And-Run Mixes Fast. In 37th International Symposium on Computational Geometry (SoCG 2021). Leibniz International Proceedings in Informatics (LIPIcs), Volume 189, pp. 51:1-51:12, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2021) https://doi.org/10.4230/LIPIcs.SoCG.2021.51

Author Details

Aditi Laddha
  • Georgia Institute of Technology, Atlanta, GA, USA
Santosh S. Vempala
  • Georgia Institute of Technology, Atlanta, GA, USA

Funding

  • Laddha, Aditi: Supported in part by NSF awards 1839323 and 2007443.
  • Vempala, Santosh S.: Supported in part by NSF awards 1839323, 1909756 and 2007443.

References

  1. Hans C. Andersen and Persi Diaconis. Hit and run as a unifying device. Journal de la société française de statistique, 148(4):5-28, 2007. URL: http://www.numdam.org/item/JSFS_2007__148_4_5_0.
  2. Arnon Boneh. Preduce - a probabilistic algorithm identifying redundancy by a random feasible point generator (rfpg). In Redundancy in Mathematical Programming, pages 108-134, Berlin, Heidelberg, 1983. Springer Berlin Heidelberg. Google Scholar
  3. Ben Cousins and Santosh Vempala. Volume computation of convex bodies. MATLAB File Exchange, 2013. URL: http://www.mathworks.com/matlabcentral/fileexchange/43596-volume-computation-of-convex-bodies.
  4. Ben Cousins and Santosh Vempala. A practical volume algorithm. Mathematical Programming Computation, 8(2):133-160, 2016. Google Scholar
  5. Persi Diaconis, Kshitij Khare, and Laurent Saloff-Coste. Gibbs sampling, conjugate priors and coupling. Sankhya A, 72(1):136-169, 2010. Google Scholar
  6. Persi Diaconis, Gilles Lebeau, and Laurent Michel. Gibbs/metropolis algorithms on a convex polytope. Mathematische Zeitschrift, 272(1-2):109-129, 2012. Google Scholar
  7. Ioannis Z Emiris and Vissarion Fisikopoulos. Efficient random-walk methods for approximating polytope volume. In Proceedings of the thirtieth annual symposium on Computational geometry, pages 318-327, 2014. Google Scholar
  8. Jenny Rose Finkel, Trond Grenager, and Christopher D Manning. Incorporating non-local information into information extraction systems by gibbs sampling. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL'05), pages 363-370, 2005. Google Scholar
  9. Stuart Geman and Donald Geman. Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell., 6(6):721-741, 1984. URL: https://doi.org/10.1109/TPAMI.1984.4767596.
  10. Edward I George and Robert E McCulloch. Variable selection via gibbs sampling. Journal of the American Statistical Association, 88(423):881-889, 1993. Google Scholar
  11. R. Kannan, L. Lovász, and M. Simonovits. Isoperimetric problems for convex bodies and a localization lemma. Discrete & Computational Geometry, 13:541-559, 1995. Google Scholar
  12. R. Kannan, L. Lovász, and M. Simonovits. Random walks and an O^*(n⁵) volume algorithm for convex bodies. Random Structures and Algorithms, 11:1-50, 1997. Google Scholar
  13. Yin Tat Lee and Santosh Srinivas Vempala. Eldan’s stochastic localization and the KLS hyperplane conjecture: An improved lower bound for expansion. In Proc. of IEEE FOCS, 2017. Google Scholar
  14. L. Lovász. How to compute the volume? Jber. d. Dt. Math.-Verein, Jubiläumstagung 1990, pages 138-151, 1990. Google Scholar
  15. L. Lovász. Hit-and-run mixes fast. Math. Prog., 86:443-461, 1998. Google Scholar
  16. L. Lovász and M. Simonovits. Random walks in a convex body and an improved volume algorithm. In Random Structures and Alg., volume 4, pages 359-412, 1993. Google Scholar
  17. L. Lovász and S. Vempala. Hit-and-run from a corner. SIAM J. Computing, 35:985-1005, 2006. Google Scholar
  18. L. Lovász and S. Vempala. The geometry of logconcave functions and sampling algorithms. Random Struct. Algorithms, 30(3):307-358, 2007. URL: https://doi.org/10.1002/rsa.v30:3.
  19. László Lovász and Miklós Simonovits. Random walks in a convex body and an improved volume algorithm. Random structures & algorithms, 4(4):359-412, 1993. Google Scholar
  20. Hariharan Narayanan and Piyush Srivastava. On the mixing time of coordinate hit-and-run. arXiv preprint, 2020. URL: http://arxiv.org/abs/2009.14004.
  21. A. Sinclair and M. Jerrum. Approximate counting, uniform generation and rapidly mixing Markov chains. Information and Computation, 82:93-133, 1989. Google Scholar
  22. R.L. Smith. Efficient Monte-Carlo procedures for generating points uniformly distributed over bounded regions. Operations Res., 32:1296-1308, 1984. Google Scholar
  23. V. Turchin. On the computation of multidimensional integrals by the monte-carlo method. Theory of Probability & Its Applications, 16(4):720-724, 1971. URL: https://doi.org/10.1137/1116083.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact