Asymptotic Divergences and Strong Dichotomy

Authors Xiang Huang , Jack H. Lutz, Elvira Mayordomo, Donald M. Stull

Thumbnail PDF


  • Filesize: 0.51 MB
  • 15 pages

Document Identifiers

Author Details

Xiang Huang
  • Le Moyne College, Syracuse, NY 13214, USA
  • Iowa State University, Ames, IA 50011, USA
Jack H. Lutz
  • Iowa State University, Ames, IA 50011, USA
Elvira Mayordomo
  • Departamento de Informática e Ingeniería de Sistemas, Instituto de Investigación en Ingeniería de Aragón, Universidad de Zaragoza, 50018 Zaragoza, Spain
Donald M. Stull
  • Iowa State University, Ames, IA 50011, USA


We thank students in Iowa State University’s fall, 2017, advanced topics in computational randomness course for listening to a preliminary version of this research. We thank three anonymous reviewers of an earlier draft of this paper for useful comments and corrections.

Cite AsGet BibTex

Xiang Huang, Jack H. Lutz, Elvira Mayordomo, and Donald M. Stull. Asymptotic Divergences and Strong Dichotomy. In 37th International Symposium on Theoretical Aspects of Computer Science (STACS 2020). Leibniz International Proceedings in Informatics (LIPIcs), Volume 154, pp. 51:1-51:15, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2020)


The Schnorr-Stimm dichotomy theorem [Schnorr and Stimm, 1972] concerns finite-state gamblers that bet on infinite sequences of symbols taken from a finite alphabet Σ. The theorem asserts that, for any such sequence S, the following two things are true. (1) If S is not normal in the sense of Borel (meaning that every two strings of equal length appear with equal asymptotic frequency in S), then there is a finite-state gambler that wins money at an infinitely-often exponential rate betting on S. (2) If S is normal, then any finite-state gambler betting on S loses money at an exponential rate betting on S. In this paper we use the Kullback-Leibler divergence to formulate the lower asymptotic divergence div(S||α) of a probability measure α on Σ from a sequence S over Σ and the upper asymptotic divergence Div(S||α) of α from S in such a way that a sequence S is α-normal (meaning that every string w has asymptotic frequency α(w) in S) if and only if Div(S||α)=0. We also use the Kullback-Leibler divergence to quantify the total risk Risk_G(w) that a finite-state gambler G takes when betting along a prefix w of S. Our main theorem is a strong dichotomy theorem that uses the above notions to quantify the exponential rates of winning and losing on the two sides of the Schnorr-Stimm dichotomy theorem (with the latter routinely extended from normality to α-normality). Modulo asymptotic caveats in the paper, our strong dichotomy theorem says that the following two things hold for prefixes w of S. (1') The infinitely-often exponential rate of winning in 1 is 2^{Div(S||α)|w|}. (2') The exponential rate of loss in 2 is 2^{-Risk_G(w)}. We also use (1') to show that 1-Div(S||α)/c, where c= log(1/ min_{a∈Σ} α(a)), is an upper bound on the finite-state α-dimension of S and prove the dual fact that 1-div(S||α)/c is an upper bound on the finite-state strong α-dimension of S.

Subject Classification

ACM Subject Classification
  • Theory of computation
  • Theory of computation → Formal languages and automata theory
  • finite-state dimension
  • finite-state gambler
  • Kullback-Leibler divergence
  • normal sequences


  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    PDF Downloads


  1. Valerii Nikolaevich Agafonov. Normal sequences and finite automata. In Doklady Akademii Nauk, volume 179 (2), pages 255-256, 1968. Google Scholar
  2. Krishna B. Athreya, John M. Hitchcock, Jack H. Lutz, and Elvira Mayordomo. Effective strong dimension in algorithmic information and computational complexity. SIAM Journal on Computing, 37(3):671-705, 2007. Google Scholar
  3. Verónica Becher and Pablo Ariel Heiber. Normal numbers and finite automata. Theoretical Computer Science, 477:109-116, 2013. Google Scholar
  4. Verónica Becher, Pablo Ariel Heiber, and Theodore A. Slaman. A polynomial-time algorithm for computing absolutely normal numbers. Information and Computation, 232:1-9, 2013. Google Scholar
  5. Patrick Billingsley. Hausdorff dimension in probability theory. Illinois Journal of Mathematics, 4(2):187-209, 1960. Google Scholar
  6. Patrick Billingsley. Probability and Measure. Wiley-Interscience, 1995. Google Scholar
  7. mile Borel. Les probabilités dénombrables et leurs applications arithmétiques. Rendiconti del Circolo Matematico di Palermo (1884-1940), 27(1):247-271, 1909. Google Scholar
  8. Chris Bourke, John M. Hitchcock, and N. V. Vinodchandran. Entropy rates and finite-state dimension. Theoretical Computer Science, 349(3):392-406, 2005. Google Scholar
  9. Yann Bugeaud. Distribution Modulo One and Diophantine Approximation. Cambridge University Press, 2012. Google Scholar
  10. Helmut Cajar. Billingsley Dimension in Probability Spaces. Springer-Verlag, 1981. Google Scholar
  11. Arthur H. Copeland and Paul Erdös. Note on normal numbers. Bulletin of the American Mathematical Society, 52(10):857-860, 1946. Google Scholar
  12. Thomas R. Cover and Joy A. Thomas. Elements of Information Theory. John Wiley & Sons, Inc., 2006. Google Scholar
  13. Imre Csiszár and Paul C. Shields. Information Theory and Statistics: A Tutorial, volume 1 (4). Now Publishers, Inc., 2004. Google Scholar
  14. Jack J. Dai, James I. Lathrop, Jack H. Lutz, and Elvira Mayordomo. Finite-state dimension. Theoretical Computer Science, 310(1-3):1-33, 2004. Google Scholar
  15. Karma Dajani and Cor Kraaikamp. Ergodic Theory of Numbers. Cambridge University Press, 2002. Google Scholar
  16. David Doty, Jack H. Lutz, and Satyadev Nandakumar. Finite-state dimension and real arithmetic. Information and Computation, 205(11):1640-1651, 2007. Google Scholar
  17. Kenneth Falconer. Fractal Geometry: Mathematical Foundations and Applications. Wiley, 2014. Google Scholar
  18. Meir Feder. Gambling using a finite state machine. IEEE Transactions on Information Theory, 37(5):1459-1465, 1991. Google Scholar
  19. Xiaoyang Gu, Jack H. Lutz, and Philippe Moser. Dimensions of Copeland-Erdös sequences. Information and Computation, 205(9):1317-1333, 2007. Google Scholar
  20. F. Hausdorff. Dimension und äußeres maß. Mathematische Annalen, 79:157-179, 1919. Google Scholar
  21. Teturo Kamae and Benjamin Weiss. Normal numbers and selection rules. Israel Journal of Mathematics, 21(2-3):101-110, 1975. Google Scholar
  22. Donald E. Knuth. Art of Computer Programming, volume 2: Seminumerical Algorithms. Addison-Wesley Professional, 2014. Google Scholar
  23. Solomon Kullback and Richard A. Leibler. On information and sufficiency. The Annals of Mathematical Statistics, 22(1):79-86, 1951. Google Scholar
  24. Jack H. Lutz. Dimension in complexity classes. SIAM Journal on Computing, 32(5):1236-1259, 2003. Google Scholar
  25. Jack H. Lutz. The dimensions of individual strings and sequences. Information and Computation, 187(1):49-79, 2003. URL:
  26. Jack H. Lutz. A divergence formula for randomness and dimension. Theoretical Computer Science, 412(1-2):166-177, 2011. Google Scholar
  27. Jack H. Lutz and Elvira Mayordomo. Computing absolutely normal numbers in nearly linear time. arXiv preprint, 2016. URL:
  28. Per Martin-Löf. The definition of random sequences. Information and Control, 9(6):602-619, 1966. Google Scholar
  29. Wolfgang Merkle and Jan Reimann. Selection functions that do not preserve normality. Theory of Computing Systems, 39(5):685-697, 2006. Google Scholar
  30. Claus-Peter Schnorr. A unified approach to the definition of random sequences. Mathematical Systems Theory, 5(3):246-258, 1971. Google Scholar
  31. Claus-Peter Schnorr and Hermann Stimm. Endliche Automaten und Zufallsfolgen. Acta Informatica, 1(4):345-359, 1972. Google Scholar
  32. Claude Elwood Shannon. A mathematical theory of communication. Bell System Technical Journal, 27(3):379-423, 1948. Google Scholar
  33. Alexander Shen. Automatic Kolmogorov complexity and normality revisited. In International Symposium on Fundamentals of Computation Theory, pages 418-430. Springer, 2017. Google Scholar
  34. Dennis Sullivan. Entropy, Hausdorff measures old and new, and limit sets of geometrically finite Kleinian groups. Acta Mathematica, 153(1):259-277, 1984. Google Scholar
  35. Claude Tricot. Two definitions of fractional dimension. In Mathematical Proceedings of the Cambridge Philosophical Society, volume 91 (1), pages 57-74. Cambridge University Press, 1982. Google Scholar
  36. Donald Dines Wall. Normal numbers. Ph.D. thesis, University of California, Berkeley, 1949. Google Scholar
Questions / Remarks / Feedback

Feedback for Dagstuhl Publishing

Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail