Document Open Access Logo

Decision Problems in Information Theory

Authors Mahmoud Abo Khamis, Phokion G. Kolaitis, Hung Q. Ngo, Dan Suciu

PDF
Thumbnail PDF

File

LIPIcs.ICALP.2020.106.pdf
  • Filesize: 0.68 MB
  • 20 pages

Document Identifiers

Author Details

Mahmoud Abo Khamis
  • relationalAI, Berkeley, CA, USA
Phokion G. Kolaitis
  • University of California, Santa Cruz, CA, USA
  • IBM Research - Almaden, CA, USA
Hung Q. Ngo
  • relationalAI, Berkeley, CA, USA
Dan Suciu
  • University of Washington, Seattle, WA, USA

Funding

  • Kolaitis, Phokion G.: Research partially supported by NSF Grant IIS-1814152.
  • Suciu, Dan: Research partially supported by NSF IIS-1907997, NSF III-1703281, NSF III-1614738, and NSF AiTF-1535565.

Acknowledgements

We thank Miika Hannula for several useful pointers to earlier work on the implication problem for conditional independence.

Cite AsGet BibTex

Mahmoud Abo Khamis, Phokion G. Kolaitis, Hung Q. Ngo, and Dan Suciu. Decision Problems in Information Theory. In 47th International Colloquium on Automata, Languages, and Programming (ICALP 2020). Leibniz International Proceedings in Informatics (LIPIcs), Volume 168, pp. 106:1-106:20, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2020)
https://doi.org/10.4230/LIPIcs.ICALP.2020.106

Abstract

Constraints on entropies are considered to be the laws of information theory. Even though the pursuit of their discovery has been a central theme of research in information theory, the algorithmic aspects of constraints on entropies remain largely unexplored. Here, we initiate an investigation of decision problems about constraints on entropies by placing several different such problems into levels of the arithmetical hierarchy. We establish the following results on checking the validity over all almost-entropic functions: first, validity of a Boolean information constraint arising from a monotone Boolean formula is co-recursively enumerable; second, validity of "tight" conditional information constraints is in Π⁰₃. Furthermore, under some restrictions, validity of conditional information constraints "with slack" is in Σ⁰₂, and validity of information inequality constraints involving max is Turing equivalent to validity of information inequality constraints (with no max involved). We also prove that the classical implication problem for conditional independence statements is co-recursively enumerable.

Subject Classification

ACM Subject Classification
  • Mathematics of computing → Information theory
  • Theory of computation → Computability
  • Theory of computation → Complexity classes
Keywords
  • Information theory
  • decision problems
  • arithmetical hierarchy
  • entropic functions

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Related Versions

References

  1. Mahmoud Abo Khamis, Phokion G. Kolaitis, Hung Q. Ngo, and Dan Suciu. Bag query containment and information theory. In Proceedings of the 39th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems, PODS 2020, Portland, CA, USA, 2020. to appear. Google Scholar
  2. Mahmoud Abo Khamis, Phokion G. Kolaitis, Hung Q. Ngo, and Dan Suciu. Decision problems in information theory. CoRR, abs/2004.08783, 2020. URL: http://arxiv.org/abs/2004.08783.
  3. Mahmoud Abo Khamis, Hung Q. Ngo, and Dan Suciu. Computing join queries with functional dependencies. In Proceedings of the 35th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems, PODS 2016, San Francisco, CA, USA, June 26 - July 01, 2016, pages 327-342, 2016. Google Scholar
  4. Mahmoud Abo Khamis, Hung Q. Ngo, and Dan Suciu. What do Shannon-type inequalities, submodular width, and disjunctive Datalog have to do with one another? In Proceedings of the 36th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems, PODS 2017, pages 429-444, 2017. Google Scholar
  5. Noga Alon. On the number of subgraphs of prescribed type of graphs with a given number of edges. Israel J. Math., 38(1-2):116-130, 1981. URL: https://doi.org/10.1007/BF02761855.
  6. Albert Atserias, Martin Grohe, and Dániel Marx. Size bounds and query plans for relational joins. SIAM J. Comput., 42(4):1737-1767, 2013. URL: https://doi.org/10.1137/110859440.
  7. Amos Beimel. Secret-sharing schemes: A survey. In Yeow Meng Chee, Zhenbo Guo, San Ling, Fengjing Shao, Yuansheng Tang, Huaxiong Wang, and Chaoping Xing, editors, Coding and Cryptology - Third International Workshop, IWCC 2011, Qingdao, China, May 30-June 3, 2011. Proceedings, volume 6639 of Lecture Notes in Computer Science, pages 11-46. Springer, 2011. URL: https://doi.org/10.1007/978-3-642-20901-7_2.
  8. G. R. Blakley. Safeguarding cryptographic keys. In Managing Requirements Knowledge, International Workshop on, page 313, Los Alamitos, CA, USA, June 1979. IEEE Computer Society. URL: https://doi.org/10.1109/AFIPS.1979.98.
  9. Carlo Blundo, Alfredo De Santis, and Ugo Vaccaro. On secret sharing schemes. Inf. Process. Lett., 65(1):25-32, 1998. URL: https://doi.org/10.1016/S0020-0190(97)00194-4.
  10. Renato M. Capocelli, Alfredo De Santis, Luisa Gargano, and Ugo Vaccaro. On the size of shares for secret sharing schemes. J. Cryptology, 6(3):157-167, 1993. URL: https://doi.org/10.1007/BF00198463.
  11. Terence H. Chan. Group characterizable entropy functions. In IEEE International Symposium on Information Theory, ISIT 2007, Nice, France, June 24-29, 2007, pages 506-510. IEEE, 2007. Google Scholar
  12. Terence H. Chan and Raymond W. Yeung. On a relation between information inequalities and group theory. IEEE Transactions on Information Theory, 48(7):1992-1995, 2002. Google Scholar
  13. Surajit Chaudhuri and Moshe Y. Vardi. Optimization of Real conjunctive queries. In Catriel Beeri, editor, Proceedings of the Twelfth ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, May 25-28, 1993, Washington, DC, USA, pages 59-70. ACM Press, 1993. URL: http://dl.acm.org/citation.cfm?id=153850.
  14. F. R. K. Chung, R. L. Graham, P. Frankl, and J. B. Shearer. Some intersection theorems for ordered sets and graphs. J. Combin. Theory Ser. A, 43(1):23-37, 1986. URL: https://doi.org/10.1016/0097-3165(86)90019-1.
  15. László Csirmaz. The size of a share must be large. J. Cryptology, 10(4):223-231, 1997. URL: https://doi.org/10.1007/s001459900029.
  16. Peter R. de Waal and Linda C. van der Gaag. Stable independance and complexity of representation. In David Maxwell Chickering and Joseph Y. Halpern, editors, UAI '04, Proceedings of the 20th Conference in Uncertainty in Artificial Intelligence, Banff, Canada, July 7-11, 2004, pages 112-119. AUAI Press, 2004. URL: https://dslpitt.org/uai/displayArticleDetails.jsp?mmnu=1&smnu=2&article_id=1165&proceeding_id=20.
  17. Ehud Friedgut and Jeff Kahn. On the number of copies of one hypergraph in another. Israel J. Math., 105:251-256, 1998. URL: https://doi.org/10.1007/BF02780332.
  18. Dan Geiger and Judea Pearl. Logical and algorithmic properties of conditional independence and graphical models. The Annals of Statistics, 21(4):2001-2021, 1993. Google Scholar
  19. Georg Gottlob, Stephanie Tien Lee, Gregory Valiant, and Paul Valiant. Size and treewidth bounds for conjunctive queries. J. ACM, 59(3):16:1-16:35, 2012. URL: https://doi.org/10.1145/2220357.2220363.
  20. Peter Grunwald and Paul Vitányi. Shannon information and Kolmogorov complexity. arXiv preprint, 2004. URL: http://arxiv.org/abs/cs/0410002.
  21. Marc Gyssens, Mathias Niepert, and Dirk Van Gucht. On the completeness of the semigraphoid axioms for deriving arbitrary from saturated conditional independence statements. Inf. Process. Lett., 114(11):628-633, 2014. URL: https://doi.org/10.1016/j.ipl.2014.05.010.
  22. Terence H. Chan. Balanced information inequalities. Information Theory, IEEE Transactions on, 49:3261-3267, January 2004. URL: https://doi.org/10.1109/TIT.2003.820037.
  23. Daniel Hammer, Andrei Romashchenko, Alexander Shen, and Nikolai Vereshchagin. Inequalities for Shannon entropy and Kolmogorov complexity. Journal of Computer and System Sciences, 60(2):442-464, 2000. Google Scholar
  24. Miika Hannula, Åsa Hirvonen, Juha Kontinen, Vadim Kulikov, and Jonni Virtema. Facets of distribution identities in probabilistic team semantics. CoRR, abs/1812.05873, 2018. URL: http://arxiv.org/abs/1812.05873.
  25. Tarik Kaced and Andrei E. Romashchenko. Conditional information inequalities for entropic and almost entropic points. IEEE Trans. Information Theory, 59(11):7149-7167, 2013. URL: https://doi.org/10.1109/TIT.2013.2274614.
  26. Ehud D. Karnin, J. W. Greene, and Martin E. Hellman. On secret sharing systems. IEEE Trans. Information Theory, 29(1):35-41, 1983. URL: https://doi.org/10.1109/TIT.1983.1056621.
  27. Batya Kenig and Dan Suciu. Integrity constraints revisited: From exact to approximate implication. CoRR, abs/1812.09987, 2018. URL: http://arxiv.org/abs/1812.09987.
  28. Phokion G. Kolaitis and Moshe Y. Vardi. Conjunctive-query containment and constraint satisfaction. In Proceedings of the Seventeenth ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, June 1-3, 1998, Seattle, Washington, USA, pages 205-213, 1998. URL: https://doi.org/10.1145/275487.275511.
  29. Swastik Kopparty and Benjamin Rossman. The homomorphism domination exponent. European Journal of Combinatorics, 32(7):1097-1114, 2011. Homomorphisms and Limits. Google Scholar
  30. Tony T. Lee. An information-theoretic analysis of relational databases - part I: data dependencies and information metric. IEEE Trans. Software Eng., 13(10):1049-1061, 1987. URL: https://doi.org/10.1109/TSE.1987.232847.
  31. Konstantin Makarychev, Yury Makarychev, Andrei Romashchenko, and Nikolai Vereshchagin. A new class of non-Shannon-type inequalities for entropies. Commun. Inf. Syst., 2(2):147-165, 2002. URL: https://doi.org/10.4310/CIS.2002.v2.n2.a3.
  32. David Marker. Model theory and exponentiation, 1996. Google Scholar
  33. Frantisek Matúš. Probabilistic conditional independence structures and matroid theory: Background, 1994. Google Scholar
  34. Frantisek Matúš. Infinitely many information inequalities. In IEEE International Symposium on Information Theory, ISIT 2007, Nice, France, June 24-29, 2007, pages 41-44, 2007. URL: https://doi.org/10.1109/ISIT.2007.4557201.
  35. C.F. Miller. Decision problems for groups — survey and reflections. In G. Baumslag and C.F. Miller, editors, Algorithms and Classification in Combinatorial Group Theory. Mathematical Sciences Research Institute Publications, volume 23. Springer, NY, 1992. Google Scholar
  36. Hung Q. Ngo. Worst-case optimal join algorithms: Techniques, results, and open problems. In Jan Van den Bussche and Marcelo Arenas, editors, Proceedings of the 37th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems, Houston, TX, USA, June 10-15, 2018, pages 111-124. ACM, 2018. URL: https://doi.org/10.1145/3196959.3196990.
  37. Mathias Niepert, Dirk Van Gucht, and Marc Gyssens. Logical and algorithmic properties of stable conditional independence. Int. J. Approx. Reason., 51(5):531-543, 2010. URL: https://doi.org/10.1016/j.ijar.2010.01.011.
  38. Mathias Niepert, Marc Gyssens, Bassem Sayrafi, and Dirk Van Gucht. On the conditional independence implication problem: A lattice-theoretic approach. Artif. Intell., 202:29-51, 2013. URL: https://doi.org/10.1016/j.artint.2013.06.005.
  39. Judea Pearl and Azaria Paz. Graphoids: Graph-based logic for reasoning about relevance relations or when would x tell you more about y if you already know z? In ECAI, pages 357-363, 1986. Google Scholar
  40. Nicholas Pippenger. What are the laws of information theory. In 1986 Special Problems on Communication and Computation Conference, pages 3-5, 1986. Google Scholar
  41. Hartley Rogers and H Rogers. Theory of recursive functions and effective computability, volume 5. McGraw-Hill New York, 1967. Google Scholar
  42. Adi Shamir. How to share a secret. Commun. ACM, 22(11):612-613, 1979. URL: https://doi.org/10.1145/359168.359176.
  43. Larry J Stockmeyer. The polynomial-time hierarchy. Theoretical Computer Science, 3(1):1-22, 1976. Google Scholar
  44. Milan Studený. Conditional independence relations have no finite complete characterization. In 11th Prague Conf. Information Theory, Statistical Decision Foundation and Random Processes, pages 377-396. Norwell, MA, 1990. Google Scholar
  45. Alfred Tarski. A decision method for elementary algebra and geometry. In Quantifier elimination and cylindrical algebraic decomposition, pages 24-84. Springer, 1998. Google Scholar
  46. Raymond W. Yeung. A first course in information theory. Information Technology: Transmission, Processing and Storage. Kluwer Academic/Plenum Publishers, New York, 2002. With a foreword by Toby Berger, With 1 CD-ROM. URL: https://doi.org/10.1007/978-1-4419-8608-5.
  47. Raymond W. Yeung. Information Theory and Network Coding. Springer Publishing Company, Incorporated, 1 edition, 2008. Google Scholar
  48. Raymond W. Yeung and Zhen Zhang. A class of non-Shannon-type information inequalities and their applications. Commun. Inf. Syst., 1(1):87-100, 2001. URL: https://doi.org/10.4310/CIS.2001.v1.n1.a6.
  49. Zhen Zhang. On a new non-Shannon type information inequality. Commun. Inf. Syst., 3(1):47-60, 2003. URL: https://doi.org/10.4310/CIS.2003.v3.n1.a4.
  50. Zhen Zhang and Raymond W. Yeung. A non-Shannon-type conditional inequality of information quantities. IEEE Trans. Information Theory, 43(6):1982-1986, 1997. Google Scholar
  51. Zhen Zhang and Raymond W Yeung. On characterization of entropy function via information inequalities. IEEE Transactions on Information Theory, 44(4):1440-1452, 1998. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2024 Schloss Dagstuhl – LZI GmbH Imprint Privacy Contact