Document Open Access Logo

On Approximating the f-Divergence Between Two Ising Models

Authors Weiming Feng , Yucheng Fu

PDF

File

Thumbnail PDF
PDF
LIPIcs.ITCS.2026.59.pdf
  • Filesize: 0.87 MB
  • 23 pages

Document Identifiers

Related Versions

Subject Classification

ACM Subject Classification
  • Theory of computation → Approximation algorithms analysis
  • Mathematics of computing → Markov networks
Keywords
  • Ising model
  • f-divergence
  • approximation algorithms
  • randomized algorithms

Metrics

Abstract

The f-divergence is a fundamental notion that measures the difference between two distributions. In this paper, we study the problem of approximating the f-divergence between two Ising models, which is a generalization of recent work on approximating the TV-distance. Given two Ising models ν and μ, which are specified by their interaction matrices and external fields, the problem is to approximate the f-divergence D_f (ν ‖ μ) within an arbitrary relative error e^{±ε}. For χ^α-divergence with a constant integer α, we establish both algorithmic and hardness results. The algorithm works in a parameter regime that matches the hardness result. Our algorithm can be extended to other f-divergences such as α-divergence, Kullback-Leibler divergence, Rényi divergence, Jensen-Shannon divergence, and squared Hellinger distance.

Cite As Get BibTex

Weiming Feng and Yucheng Fu. On Approximating the f-Divergence Between Two Ising Models. In 17th Innovations in Theoretical Computer Science Conference (ITCS 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 362, pp. 59:1-59:23, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026) https://doi.org/10.4230/LIPIcs.ITCS.2026.59

Author Details

Weiming Feng
  • School of Computing and Data Science, The University of Hong Kong, China
Yucheng Fu
  • School of Computing and Data Science, The University of Hong Kong, China

Funding

  • Feng, Weiming: Early Career Scheme of the Hong Kong Research Grants Council under grant number 27202725.
  • Fu, Yucheng: 2025 Summer Research Internship Programme in the School of Computing and Data Science at The University of Hong Kong.

References

  1. Amirali Abdullah, Ravi Kumar, Andrew McGregor, Sergei Vassilvitskii, and Suresh Venkatasubramanian. Sketching, embedding and dimensionality reduction in information theoretic spaces. In AISTATS, volume 51, pages 948-956. JMLR.org, 2016. URL: http://proceedings.mlr.press/v51/abdullah16.html.
  2. Antoine Amarilli, Marcelo Arenas, YooJung Choi, Mikaël Monet, Guy Van den Broeck, and Benjie Wang. A circus of circuits: Connections between decision diagrams, circuits, and automata. arXiv preprint, 2024. URL: https://doi.org/10.48550/arXiv.2404.09674.
  3. Arnab Bhattacharyya, Weiming Feng, and Piyush Srivastava. Approximating the total variation distance between Gaussians. In AISTATS, volume 258 of Proceedings of Machine Learning Research, pages 1846-1854. PMLR, 2025. URL: https://proceedings.mlr.press/v258/bhattacharyya25a.html.
  4. Arnab Bhattacharyya, Sutanu Gayen, Kuldeep S. Meel, Dimitrios Myrisiotis, A. Pavan, and N. V. Vinodchandran. On approximating total variation distance. In IJCAI, pages 3479-3487. ijcai.org, 2023. URL: https://doi.org/10.24963/IJCAI.2023/387.
  5. Arnab Bhattacharyya, Sutanu Gayen, Kuldeep S. Meel, Dimitrios Myrisiotis, A. Pavan, and N. V. Vinodchandran. Total variation distance meets probabilistic inference. In ICML. OpenReview.net, 2024. URL: https://openreview.net/forum?id=6OSLjErBhh.
  6. Arnab Bhattacharyya, Sutanu Gayen, Kuldeep S. Meel, Dimitrios Myrisiotis, A. Pavan, and N. V. Vinodchandran. Computational explorations of total variation distance. In ICLR. OpenReview.net, 2025. URL: https://openreview.net/forum?id=xak8c9l1nu.
  7. Arnab Bhattacharyya, Sutanu Gayen, Kuldeep S Meel, Dimitrios Myrisiotis, A Pavan, and NV Vinodchandran. Algorithms and hardness for estimating statistical similarity. arXiv preprint, 2025. URL: https://arxiv.org/abs/2502.10527.
  8. Arnab Bhattacharyya, Sutanu Gayen, Kuldeep S. Meel, and N. V. Vinodchandran. Efficient distance approximation for structured high-dimensional distributions via learning. In NeurIPS, 2020. Google Scholar
  9. Antonio Blanca, Zongchen Chen, Daniel Stefankovic, and Eric Vigoda. Complexity of high-dimensional identity testing with coordinate conditional sampling. ACM Trans. Algorithms, 21(1):7:1-7:58, 2025. URL: https://doi.org/10.1145/3686799.
  10. Guy Bresler. Efficiently learning Ising models on arbitrary graphs. In STOC, pages 771-782. ACM, 2015. URL: https://doi.org/10.1145/2746539.2746631.
  11. Clément L. Canonne. A Survey on Distribution Testing: Your Data is Big. But is it Blue? Number 9 in Graduate Surveys. Theory of Computing Library, 2020. URL: https://doi.org/10.4086/toc.gs.2020.009.
  12. Clément L. Canonne. Topics and techniques in distribution testing: A biased but representative sample. Found. Trends Commun. Inf. Theory, 19(6):1032-1198, 2022. URL: https://doi.org/10.1561/0100000114.
  13. Xiaoyu Chen, Zongchen Chen, Yitong Yin, and Xinyuan Zhang. Rapid mixing at the uniqueness threshold. In STOC, pages 879-890. ACM, 2025. URL: https://doi.org/10.1145/3717823.3718260.
  14. Zongchen Chen, Kuikui Liu, and Eric Vigoda. Optimal mixing of Glauber dynamics: Entropy factorization via high-dimensional expansion. In STOC, pages 1537-1550. ACM, 2021. URL: https://doi.org/10.1145/3406325.3451035.
  15. Constantinos Daskalakis, Nishanth Dikkala, and Gautam Kamath. Testing Ising models. In SODA, pages 1989-2007. SIAM, 2018. URL: https://doi.org/10.1137/1.9781611975031.130.
  16. Weiming Feng, Heng Guo, Mark Jerrum, and Jiaheng Wang. A simple polynomial-time approximation algorithm for the total variation distance between two product distributions. TheoretiCS, 2, 2023. URL: https://doi.org/10.46298/THEORETICS.23.7.
  17. Weiming Feng, Hongyang Liu, and Minji Yang. Approximating the total variation distance between spin systems. In COLT, volume 291 of Proceedings of Machine Learning Research, pages 1974-2025. PMLR, 2025. URL: https://proceedings.mlr.press/v291/feng25a.html.
  18. Weiming Feng, Liqiang Liu, and Tianren Liu. On deterministically approximating total variation distance. In SODA, pages 1766-1791. SIAM, 2024. URL: https://doi.org/10.1137/1.9781611977912.70.
  19. Andreas Galanis, Daniel Štefankovič, and Eric Vigoda. Inapproximability of the partition function for the antiferromagnetic Ising and hard-core models. Combin. Probab. Comput., 25(4):500-559, 2016. URL: https://doi.org/10.1017/S0963548315000401.
  20. William Gay, William He, Nicholas Kocurek, and Ryan O'Donnell. Sampling and identity-testing without approximate tensorization of entropy. arXiv preprint arXiv:2506.23456, 2025. URL: https://doi.org/10.48550/arXiv.2506.23456.
  21. Mark Jerrum and Alistair Sinclair. Polynomial-time approximation algorithms for the Ising model. SIAM J. Comput., 22(5):1087-1116, 1993. URL: https://doi.org/10.1137/0222066.
  22. Takafumi Kanamori, Taiji Suzuki, and Masashi Sugiyama. f-divergence estimation and two-sample homogeneity test under semiparametric density-ratio models. IEEE Trans. Inf. Theory, 58(2):708-720, 2012. URL: https://doi.org/10.1109/TIT.2011.2163380.
  23. Stefan Kiefer. On computing the total variation distance of hidden Markov models. In ICALP, volume 107 of LIPIcs, pages 130:1-130:13. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2018. URL: https://doi.org/10.4230/LIPIcs.ICALP.2018.130.
  24. Aryeh Kontorovich. On the tensorization of the variational distance. Electron. Commun. Probab., 30:Paper No. 32, 10, 2025. Google Scholar
  25. Aryeh Kontorovich and Ariel Avital. Sharp bounds on aggregate expert error. In ALT, volume 272 of Proceedings of Machine Learning Research, pages 653-663. PMLR, 2025. URL: https://proceedings.mlr.press/v272/kontorovich25a.html.
  26. Loong Kuan Lee, Nico Piatkowski, François Petitjean, and Geoffrey I. Webb. Computing divergences between discrete decomposable models. In AAAI, pages 12243-12251. AAAI Press, 2023. URL: https://doi.org/10.1609/AAAI.V37I10.26443.
  27. David A Levin and Yuval Peres. Markov chains and mixing times, volume 107. American Mathematical Soc., 2017. Google Scholar
  28. Liang Li, Pinyan Lu, and Yitong Yin. Correlation decay up to uniqueness in spin systems. In SODA, pages 67-84. SIAM, 2013. URL: https://doi.org/10.1137/1.9781611973105.5.
  29. Jingcheng Liu, Alistair Sinclair, and Piyush Srivastava. The Ising partition function: Zeros and deterministic approximation. Journal of Statistical Physics, 174(2):287-315, 2019. Google Scholar
  30. Barnabás Póczos and Jeff G. Schneider. On the estimation of α-divergences. In AISTATS, volume 15 of JMLR Proceedings, pages 609-617. JMLR.org, 2011. Google Scholar
  31. Paul K. Rubenstein, Olivier Bousquet, Josip Djolonga, Carlos Riquelme, and Ilya O. Tolstikhin. Practical and consistent estimation of f-divergences. In NeurIPS, pages 4072-4082, 2019. URL: https://proceedings.neurips.cc/paper/2019/hash/3147da8ab4a0437c15ef51a5cc7f2dc4-Abstract.html.
  32. Amit Sahai and Salil Vadhan. A complete problem for statistical zero knowledge. J. ACM, 50(2):196-249, 2003. URL: https://doi.org/10.1145/636865.636868.
  33. Allan Sly and Nike Sun. Counting in two-spin models on d-regular graphs. Ann. Probab., 42(6):2383-2416, 2014. Google Scholar
  34. Sreejith Sreekumar and Ziv Goldfeld. Neural estimation of statistical divergences. J. Mach. Learn. Res., 23:126:1-126:75, 2022. URL: https://jmlr.org/papers/v23/21-1212.html.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact