Document Open Access Logo

The Space Complexity of Sampling

Authors Eshan Chattopadhyay, Jesse Goodman, David Zuckerman

PDF

File

Thumbnail PDF
PDF
LIPIcs.ITCS.2022.40.pdf
  • Filesize: 0.8 MB
  • 23 pages

Document Identifiers

Related Versions

Subject Classification

ACM Subject Classification
  • Theory of computation → Computational complexity and cryptography
Keywords
  • Complexity of distributions
  • complexity of sampling
  • extractors
  • list decodable codes
  • lower bounds
  • read-once branching programs
  • small-space computation

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

Recently, there has been exciting progress in understanding the complexity of distributions. Here, the goal is to quantify the resources required to generate (or sample) a distribution. Proving lower bounds in this new setting is more challenging than in the classical setting, and has yielded interesting new techniques and surprising applications. In this work, we initiate a study of the complexity of sampling with limited memory, and obtain the first nontrivial sampling lower bounds against oblivious read-once branching programs (ROBPs).
In our first main result, we show that any distribution sampled by an ROBP of width 2^{Ω(n)} has statistical distance 1-2^{-Ω(n)} from any distribution that is uniform over a good code. More generally, we obtain sampling lower bounds for any list decodable code, which are nearly tight. Previously, such a result was only known for sampling in AC⁰ (Lovett and Viola, CCC'11; Beck, Impagliazzo and Lovett, FOCS'12). As an application of our result, a known connection implies new data structure lower bounds for storing codewords.
In our second main result, we prove a direct product theorem for sampling with ROBPs. Previously, no direct product theorems were known for the task of sampling, for any computational model. A key ingredient in our proof is a simple new lemma about amplifying statistical distance between sequences of somewhat-dependent random variables. Using this lemma, we also obtain a simple new proof of a known lower bound for sampling disjoint sets using two-party communication protocols (Göös and Watson, RANDOM'19).

Cite As Get BibTex

Eshan Chattopadhyay, Jesse Goodman, and David Zuckerman. The Space Complexity of Sampling. In 13th Innovations in Theoretical Computer Science Conference (ITCS 2022). Leibniz International Proceedings in Informatics (LIPIcs), Volume 215, pp. 40:1-40:23, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2022) https://doi.org/10.4230/LIPIcs.ITCS.2022.40

Author Details

Eshan Chattopadhyay
  • Cornell University, Ithaca, NY, USA
Jesse Goodman
  • Cornell University, Ithaca, NY, USA
David Zuckerman
  • University of Texas at Austin, TX, USA

Funding

  • Chattopadhyay, Eshan: Research supported by NSF CAREER Award 2045576.
  • Goodman, Jesse: Research supported by NSF CAREER Award 2045576.
  • Zuckerman, David: Research supported by NSF Grants CCF-1705028 and CCF-2008076 and a Simons Investigator Award (#409864).

Acknowledgements

We thank William Hoza and anonymous reviewers for extremely helpful comments.

References

  1. Scott Aaronson. The equivalence of sampling and searching. Theory of Computing Systems, 55(2):281-298, 2014. Google Scholar
  2. Andris Ambainis, Leonard J. Schulman, Amnon Ta-Shma, Umesh Vazirani, and Avi Wigderson. The quantum communication complexity of sampling. SIAM Journal on Computing, 32(6):1570-1585, 2003. Google Scholar
  3. László Babai. Random oracles separate pspace from the polynomial-time hierarchy. Information Processing Letters, 26(1):51-53, 1987. Google Scholar
  4. Paul Beame. A general sequential time-space tradeoff for finding unique elements. In Proceedings of the twenty-first annual ACM symposium on Theory of computing, pages 197-203, 1989. Google Scholar
  5. Chris Beck, Russell Impagliazzo, and Shachar Lovett. Large deviation bounds for decision trees and sampling lower bounds for AC0-circuits. In 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science, pages 101-110. IEEE, 2012. Google Scholar
  6. Itai Benjamini, Gil Cohen, and Igor Shinkar. Bi-Lipschitz bijection between the Boolean cube and the Hamming ball. Israel Journal of Mathematics, 212(2):677-703, 2016. Google Scholar
  7. Ravi B. Boppana and Jeffrey C. Lagarias. One-way functions and circuit complexity. Information and Computation, 74(3):226-240, 1987. Google Scholar
  8. Allan Borodin and Stephen Cook. A time-space tradeoff for sorting on a general sequential model of computation. SIAM Journal on Computing, 11(2):287-297, 1982. Google Scholar
  9. Allan Borodin, Michael J. Fischer, David G. Kirkpatrick, Nancy A. Lynch, and Martin Tompa. A time-space tradeoff for sorting on non-oblivious machines. In 20th Annual Symposium on Foundations of Computer Science (sfcs 1979), pages 319-327. IEEE, 1979. Google Scholar
  10. Eshan Chattopadhyay and Jesse Goodman. Improved extractors for small-space sources. To appear in the 62nd Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2021. Google Scholar
  11. Eshan Chattopadhyay, Jesse Goodman, and David Zuckerman. The space complexity of sampling. Electron. Colloquium Comput. Complex., page 106, 2021. URL: https://eccc.weizmann.ac.il/report/2021/106.
  12. Anindya De and Thomas Watson. Extractors and lower bounds for locally samplable sources. ACM Transactions on Computation Theory (TOCT), 4(1):1-21, 2012. Google Scholar
  13. Oded Goldreich, Shafi Goldwasser, and Asaf Nussboim. On the implementation of huge random objects. SIAM Journal on Computing, 39(7):2761-2822, 2010. Google Scholar
  14. Mika Göös and Thomas Watson. A lower bound for sampling disjoint sets. ACM Transactions on Computation Theory (TOCT), 12(3):1-13, 2020. Google Scholar
  15. Johan Håstad. Computational limitations of small-depth circuits. MIT press, 1987. Google Scholar
  16. Russell Impagliazzo and Moni Naor. Efficient cryptographic schemes provably as secure as subset sum. Journal of cryptology, 9(4):199-216, 1996. Google Scholar
  17. Rahul Jain, Yaoyun Shi, Zhaohui Wei, and Shengyu Zhang. Efficient protocols for generating bipartite classical distributions and quantum states. IEEE Transactions on Information Theory, 59(8):5171-5178, 2013. Google Scholar
  18. Jesse Kamp, Anup Rao, Salil Vadhan, and David Zuckerman. Deterministic extractors for small-space sources. Journal of Computer and System Sciences, 77(1):191-220, 2011. Google Scholar
  19. Robert Koenig and Ueli Maurer. Extracting randomness from generalized symbol-fixing and markov sources. In International Symposium on Information Theory, 2004. ISIT 2004. Proceedings., page 232. IEEE, 2004. Google Scholar
  20. Robert Koenig and Ueli Maurer. Generalized strong extractors and deterministic privacy amplification. In IMA International Conference on Cryptography and Coding, pages 322-339. Springer, 2005. Google Scholar
  21. Shachar Lovett and Emanuele Viola. Bounded-depth circuits cannot sample good codes. Comput. Complex., 21(2):245-266, 2012. URL: https://doi.org/10.1007/s00037-012-0039-3.
  22. Emanuele Viola. The complexity of distributions. SIAM Journal on Computing, 41(1):191-218, 2012. Google Scholar
  23. Emanuele Viola. Extractors for Turing-machine sources. In Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques, pages 663-671. Springer, 2012. Google Scholar
  24. Emanuele Viola. Extractors for circuit sources. SIAM Journal on Computing, 43(2):655-672, 2014. Google Scholar
  25. Emanuele Viola. Quadratic maps are hard to sample. ACM Transactions on Computation Theory (TOCT), 8(4):1-4, 2016. Google Scholar
  26. Emanuele Viola. Sampling lower bounds: boolean average-case and permutations. SIAM Journal on Computing, 49(1):119-137, 2020. Google Scholar
  27. Thomas Watson. Time hierarchies for sampling distributions. SIAM Journal on Computing, 43(5):1709-1727, 2014. Google Scholar
  28. Thomas Watson. Nonnegative rank vs. binary rank. Chicago Journal of Theoretical Computer Science, 2016(2), February 2016. Google Scholar
  29. Thomas Watson. Communication complexity with small advantage. computational complexity, 29(1):1-37, 2020. Google Scholar
  30. Andrew C. Yao. Theory and application of trapdoor functions. In 23rd Annual Symposium on Foundations of Computer Science (SFCS 1982), pages 80-91. IEEE, 1982. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact