Document Open Access Logo

On Computing Total Variation Distance Between Mixtures of Product Distributions

Authors Weiming Feng , Yucheng Fu , Minji Yang , Anqi Zhang

PDF

File

Thumbnail PDF
PDF
LIPIcs.APPROX-RANDOM.2026.51.pdf
  • Filesize: 0.86 MB
  • 21 pages

Document Identifiers

Related Versions

Subject Classification

ACM Subject Classification
  • Theory of computation → Randomness, geometry and discrete structures
Keywords
  • Randomized algorithm
  • Total variation distance

Metrics

Abstract

We study the problem of approximating the total variation distance between two mixtures of product distributions over an n-dimensional discrete domain. Given two mixtures ℙ and ℚ with k₁ and k₂ product distributions over [q]ⁿ, respectively, we give a randomized algorithm that approximates d_TV(ℙ,ℚ) within a multiplicative error of (1±ε) in time poly((nq)^{k₁+k₂}, 1/ε). We also study the special case of mixtures of Boolean subcubes over {0,1}ⁿ. For this class, we give a deterministic algorithm that exactly computes the total variation distance in time poly(n, 2^O(k₁+k₂)), and show that exact computation is #𝖯-hard when k₁+k₂ = Θ(n).

Cite As Get BibTex

Weiming Feng, Yucheng Fu, Minji Yang, and Anqi Zhang. On Computing Total Variation Distance Between Mixtures of Product Distributions. In Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques (APPROX/RANDOM 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 392, pp. 51:1-51:21, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026) https://doi.org/10.4230/LIPIcs.APPROX/RANDOM.2026.51

Author Details

Weiming Feng
  • School of Computing and Data Science, The University of Hong Kong, China
Yucheng Fu
  • School of Computing and Data Science, The University of Hong Kong, China
Minji Yang
  • School of Computing and Data Science, The University of Hong Kong, China
Anqi Zhang
  • The Institute for Interdisciplinary Information Sciences, Tsinghua University, Beijing, China

Funding

  • Feng, Weiming: ECS grant 27202725 from Hong Kong RGC.

Acknowledgements

We thank Arnab Bhattacharyya and Guy Van den Broeck for bringing the problem to our attention.

References

  1. Hassan Ashtiani, Shai Ben-David, Nicholas J. A. Harvey, Christopher Liaw, Abbas Mehrabian, and Yaniv Plan. Nearly tight sample complexity bounds for learning mixtures of Gaussians via sample compression schemes. In NeurIPS, pages 3416-3425, 2018. URL: https://proceedings.neurips.cc/paper/2018/hash/70ece1e1e0931919438fcfc6bd5f199c-Abstract.html.
  2. Arnab Bhattacharyya, Sutanu Gayen, Kuldeep S. Meel, Dimitrios Myrisiotis, A. Pavan, and N. V. Vinodchandran. On approximating total variation distance. In IJCAI, pages 3479-3487. ijcai.org, 2023. URL: https://doi.org/10.24963/IJCAI.2023/387.
  3. Arnab Bhattacharyya, Sutanu Gayen, Kuldeep S. Meel, Dimitrios Myrisiotis, A. Pavan, and N. V. Vinodchandran. Total variation distance meets probabilistic inference. In ICML. OpenReview.net, 2024. Google Scholar
  4. Arnab Bhattacharyya, Sutanu Gayen, Kuldeep S. Meel, Dimitrios Myrisiotis, Aduri Pavan, and N. V. Vinodchandran. Computational explorations of total variation distance. In ICLR. OpenReview.net, 2025. URL: https://openreview.net/forum?id=xak8c9l1nu.
  5. Arnab Bhattacharyya, Sutanu Gayen, Kuldeep S. Meel, and N. V. Vinodchandran. Efficient distance approximation for structured high-dimensional distributions via learning. In NeurIPS, 2020. Google Scholar
  6. Guy Blanc, Jane Lange, Ali Malik, and Li-Yang Tan. Lifting uniform learners via distributional decomposition. In STOC, pages 1755-1767. ACM, 2023. URL: https://doi.org/10.1145/3564246.3585212.
  7. Guy Blanc, Jane Lange, Carmen Strassle, and Li-Yang Tan. A distributional-lifting theorem for PAC learning. In Nika Haghtalab and Ankur Moitra, editors, COLT, Proceedings of Machine Learning Research, pages 375-379. PMLR, 2025. URL: https://proceedings.mlr.press/v291/blanc25a.html.
  8. Clément L. Canonne and Ronitt Rubinfeld. Testing probability distributions underlying aggregated data. In ICALP, volume 8572 of Lecture Notes in Computer Science, pages 283-295. Springer, 2014. URL: https://doi.org/10.1007/978-3-662-43948-7_24.
  9. Sitan Chen and Ankur Moitra. Beyond the low-degree algorithm: mixtures of subcubes and their applications. In STOC, pages 869-880. ACM, 2019. URL: https://doi.org/10.1145/3313276.3316375.
  10. Taolue Chen and Stefan Kiefer. On the total variation distance of labelled markov chains. In LICS, pages 33:1-33:10. ACM, 2014. URL: https://doi.org/10.1145/2603088.2603099.
  11. Jon Feldman, Ryan O'Donnell, and Rocco A. Servedio. Learning mixtures of product distributions over discrete domains. SIAM, 37(5):1536-1564, 2008. URL: https://doi.org/10.1137/060670705.
  12. Weiming Feng, Heng Guo, Mark Jerrum, and Jiaheng Wang. A simple polynomial-time approximation algorithm for the total variation distance between two product distributions. TheoretiCS, 2, 2023. URL: https://doi.org/10.46298/THEORETICS.23.7.
  13. Weiming Feng, Hongyang Liu, and Minji Yang. Approximating the total variation distance between spin systems. In COLT, volume 291 of Proceedings of Machine Learning Research, pages 1974-2025. PMLR, 2025. URL: https://proceedings.mlr.press/v291/feng25a.html.
  14. Weiming Feng, Liqiang Liu, and Tianren Liu. On deterministically approximating total variation distance. In SODA, pages 1766-1791. SIAM, 2024. URL: https://doi.org/10.1137/1.9781611977912.70.
  15. Spencer Gordon, Bijan H. Mazaheri, Yuval Rabani, and Leonard J. Schulman. Source identification for mixtures of product distributions. In COLT, Proceedings of Machine Learning Research, pages 2193-2216. PMLR, 2021. URL: http://proceedings.mlr.press/v134/gordon21a.html.
  16. Spencer L. Gordon, Erik Jahn, Bijan Mazaheri, Yuval Rabani, and Leonard J. Schulman. Identification of mixtures of discrete product distributions in near-optimal sample and time complexity. In COLT, Proceedings of Machine Learning Research, pages 2071-2091. PMLR, 2024. URL: https://proceedings.mlr.press/v247/gordon24a.html.
  17. Prateek Jain and Sewoong Oh. Learning mixtures of discrete product distributions using spectral decompositions. In COLT, JMLR Workshop and Conference Proceedings, pages 824-856. JMLR.org, 2014. URL: http://proceedings.mlr.press/v35/jain14.html.
  18. Chinmaya Kausik, Kevin Tan, and Ambuj Tewari. Learning mixtures of Markov Chains and MDPs. In ICML, Proceedings of Machine Learning Research, pages 15970-16017. PMLR, 2023. URL: https://proceedings.mlr.press/v202/kausik23a.html.
  19. Stefan Kiefer. On computing the total variation distance of hidden Markov models. In ICALP, volume 107 of LIPIcs, pages 130:1-130:13. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2018. URL: https://doi.org/10.4230/LIPIcs.ICALP.2018.130.
  20. Jeongyeol Kwon, Yonathan Efroni, Constantine Caramanis, and Shie Mannor. Reward-Mixing MDPs with few latent contexts are learnable. In ICML, volume 202 of Proceedings of Machine Learning Research, pages 18057-18082. PMLR, 2023. URL: https://proceedings.mlr.press/v202/kwon23b.html.
  21. Jerry Li and Ludwig Schmidt. Robust and proper learning for mixtures of Gaussians via systems of polynomial inequalities. In COLT, Proceedings of Machine Learning Research, pages 1302-1382. PMLR, 2017. URL: http://proceedings.mlr.press/v65/li17a.html.
  22. Amit Sahai and Salil Vadhan. A complete problem for statistical zero knowledge. J. ACM, 50(2):196-249, 2003. URL: https://doi.org/10.1145/636865.636868.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2026 Schloss Dagstuhl – LZI GmbH Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact