Document Open Access Logo

Randomness Extraction Fails for Finite-State Dimension

Authors Subin Pulari , Akhil S

PDF

File

Thumbnail PDF
PDF
LIPIcs.LICS.2026.78.pdf
  • Filesize: 0.81 MB
  • 26 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Mathematics of computing → Information theory
  • Theory of computation → Formal languages and automata theory
  • Theory of computation → Transducers
Keywords
  • Finite-state dimension
  • normal numbers
  • randomness extraction
  • finite-state transducers

Metrics

Abstract

Finite-state dimension, introduced as a finite-state analogue of Hausdorff dimension, quantifies the lower asymptotic density of information in an infinite sequence as perceived by finite-state automata. It admits several equivalent formulations; two particularly useful are via finite-state gambling strategies and via the optimal asymptotic compression ratio achieved by information-lossless finite-state compressors.
Normal sequences represent the highest level of algorithmic randomness visible to finite automata, and are exactly those sequences having finite-state dimension equal to 1. This motivates a bounded-memory notion of randomness extraction: can a finite-state transducer, reading a single sequence streamingly, extract a normal output from a single input source? More modestly, can it always transform the input into an output of strictly higher finite-state dimension? Finite-state transducers can perform surprisingly effective one-pass transformations: even with constant memory they can implement variable-length coding schemes including Shannon-Fano coding, remove local redundancy, and increase the apparent randomness rate on many structured or stochastic inputs.
We show randomness extraction using transducers is impossible in a strong, explicit form. For every rational s ∈ (0,1), we construct a near linear-time computable binary sequence X with dim_FS(X) = s such that for every finite-state transducer T, the output satisfies dim_FS(T(X)) ≤ s. Thus, for these sequences, finite-state transduction cannot extract normality - indeed it cannot even improve finite-state dimension. Our proof proceeds by a structural analysis of finite-state transducers together with a dimension-preserving diagonal construction that, for each target s, builds a sequence whose organization defeats every such transducer’s attempt to concentrate randomness. The result is a finite-state analogue of Miller’s non-extractability phenomenon for effective dimension, but its proof relies on substantially different techniques, tailored to the finite-state setting.
Furthermore, we show that the impossibility persists even with multiple independent input streams. We treat two notions of independence: (i) Kolmogorov-complexity–based independence (via joint prefix complexity), and (ii) a finite-state notion of relative independence, formulated via relative finite-state dimension. By sharp contrast with the effective-dimension setting - where two independent sources suffice for a uniform effective procedure that boosts randomness rate arbitrarily close to 1 - we show that finite-state dimension exhibits no comparable multi-source extraction phenomenon. Specifically, for every rational s ∈ (0,1) and every fixed k ≥ 2, there exist k independent sources, each of finite-state dimension s, such that for every k-input finite-state transducer T, the output satisfies dim_FS(T(X_1,… ,X_k)) ≤ s. Thus, even independent streams do not allow bounded-memory transduction to output a normal sequence or to increase finite-state dimension.

Cite As Get BibTex

Subin Pulari and Akhil S. Randomness Extraction Fails for Finite-State Dimension. In 41st Annual Symposium on Logic in Computer Science (LICS 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 380, pp. 78:1-78:26, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026) https://doi.org/10.4230/LIPIcs.LICS.2026.78

Author Details

Subin Pulari
  • National Research University Higher School of Economics Moscow, Russia
Akhil S
  • Department of Computer Science and Engineering, Indian Institute of Technology Kanpur, India

Funding

  • Pulari, Subin: supported by the Basic Research Program at HSE University and this paper was prepared in the framework of this program.
  • S, Akhil: supported by the Research-I Foundation of the Department of Computer Science and Engineering, IIT Kanpur, for research and conference travel.

Acknowledgements

The authors would like to thank Jun Le Goh for proposing the question studied in this paper. They also thank Laurent Bienvenu, Elvira Mayordomo and Satyadev Nandakumar for helpful discussions.

References

  1. Krishna Athreya, John M. Hitchcock, Jack H. Lutz, and Elvira Mayordomo. Effective strong dimension in algorithmic information and computational complexity. In Proceedings of the twenty-sixth annual ACM symposium on Theory of computing, pages 615-624, New York, NY, USA, 2007. ACM. Google Scholar
  2. Verónica Becher, Olivier Carton, and Pablo Ariel Heiber. Finite-state independence. Theory of Computing Systems, 62(7):1555-1572, 2018. URL: https://doi.org/10.1007/S00224-017-9821-6.
  3. Chris Bourke, John M. Hitchcock, and N. V. Vinodchandran. Entropy rates and finite-state dimension. In Proceedings of the 32nd International Colloquium on Automata, Languages, and Programming, volume 3580 of Lecture Notes in Computer Science, pages 144-155, Berlin, 2005. Springer. Google Scholar
  4. D. G. Champernowne. Construction of decimals normal in the scale of ten. J. London Math. Soc., 2(8):254-260, 1933. Google Scholar
  5. Thomas M. Cover and Joy A. Thomas. Elements of information theory. Wiley Series in Telecommunications. John Wiley & Sons, Inc., New York, 1991. Google Scholar
  6. Jin-Yi Dai, James I. Lathrop, Jack H. Lutz, and Elvira Mayordomo. Finite-state dimension. Theoretical Computer Science, 310(1-3):1-33, 2004. URL: https://doi.org/10.1016/S0304-3975(03)00244-5.
  7. David Doty. Dimension extractors and optimal decompression. Theory of Computing Systems, 43(3):425-463, 2008. URL: https://doi.org/10.1007/S00224-007-9024-7.
  8. Lance Fortnow, John M. Hitchcock, A. Pavan, N.V. Vinodchandran, and Fengming Wang. Extracting kolmogorov complexity with applications to dimension zero-one laws. Information and Computation, 209(4):627-636, 2011. URL: https://doi.org/10.1016/j.ic.2010.09.006.
  9. John M. Hitchcock. Fractal dimension and logarithmic loss unpredictability. Theoretical Computer Science, 304(1):431-441, 2003. URL: https://doi.org/10.1016/S0304-3975(03)00138-5.
  10. Richard M. Karp. A characterization of the minimum cycle mean in a digraph. Discrete Mathematics, 23(3):309-311, 1978. URL: https://doi.org/10.1016/0012-365X(78)90011-0.
  11. Anton Kozachinskiy and Alexander Shen. Automatic Kolmogorov complexity, normality, and finite-state dimension revisited. Journal of Computer and System Sciences, 2021. URL: https://doi.org/10.1016/j.jcss.2020.12.003.
  12. Ming Li, Paul Vitányi, et al. An introduction to Kolmogorov complexity and its applications, volume 3. Springer, 2008. Google Scholar
  13. Jack H. Lutz. The dimensions of individual strings and sequences. Information and Computation, 187(1):49-79, 2003. URL: https://doi.org/10.1016/S0890-5401(03)00187-1.
  14. Jack H. Lutz and Elvira Mayordomo. Computing absolutely normal numbers in nearly linear time. Information and Computation, 281:104746, 2021. URL: https://doi.org/10.1016/j.ic.2021.104746.
  15. Elvira Mayordomo. A kolmogorov complexity characterization of constructive hausdorff dimension. Information Processing Letters, 84(1):1-3, 2002. URL: https://doi.org/10.1016/S0020-0190(02)00343-5.
  16. Joseph S. Miller. Extracting information is hard: A turing degree of non-integral effective hausdorff dimension. Advances in Mathematics, 226(1):373-384, 2011. URL: https://doi.org/10.1016/j.aim.2010.06.024.
  17. Satyadev Nandakumar, Subin Pulari, and Akhil S. Finite-state relative dimension, dimensions of a. p. subsequences and a finite-state van lambalgen’s theorem. Information and Computation, 298:105156, 2024. URL: https://doi.org/10.1016/j.ic.2024.105156.
  18. Igal Sason. Entropy bounds for discrete random variables via maximal coupling. IEEE Trans. Inf. Theor., 59(11):7118-7131, 2013. URL: https://doi.org/10.1109/TIT.2013.2274515.
  19. C.-P. Schnorr and H. Stimm. Endliche Automaten und Zufallsfolgen. Acta Informat., 1(4):345-359, 1971/72. URL: https://doi.org/10.1007/bf00289514.
  20. Alexander Shen. Conditional normality and finite-state dimensions revisited. Theoretical Computer Science, 1057:115570, 2025. URL: https://doi.org/10.1016/j.tcs.2025.115570.
  21. Robert Tarjan. Depth-first search and linear graph algorithms. SIAM Journal on Computing, 1(2):146-160, 1972. URL: https://doi.org/10.1137/0201010.
  22. Marius Zimand. Two sources are better than one for increasing the kolmogorov complexity of infinite sequences. Theory of Computing Systems, 46(4):707-722, 2010. URL: https://doi.org/10.1007/S00224-009-9214-6.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2026 Schloss Dagstuhl – LZI GmbH Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact