Document Open Access Logo

Wavelet Tree, Part II: Text Indexing

Authors Paolo Ferragina , Raffaele Giancarlo , Roberto Grossi , Giovanna Rosone , Rossano Venturini , Jeffrey Scott Vitter

PDF

File

Thumbnail PDF
PDF
OASIcs.Manzini.4.pdf
  • Filesize: 0.83 MB
  • 10 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Theory of computation → Data structures design and analysis
  • Theory of computation → Data compression
Keywords
  • Wavelet tree
  • data compression
  • text indexing
  • compressed suffix array
  • Burrows-Wheeler transform
  • rank and select

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

The Wavelet Tree data structure introduced in Grossi, Gupta, and Vitter [Grossi et al., 2003] is a space-efficient technique for rank and select queries that generalizes from binary symbols to an arbitrary multisymbol alphabet. Over the last two decades, it has become a pivotal tool in modern full-text indexing and data compression because of its properties and capabilities in compressing and indexing data, with many applications to information retrieval, genome analysis, data mining, and web search. In this paper, we survey the fascinating history and impact of Wavelet Trees; no doubt many more developments are yet to come. Our survey borrows some content from the authors' earlier works.
This paper is divided into two parts: The first part gives a brief history of Wavelet Trees, including its varieties and practical implementations, which appears in the Festschrift dedicated to Roberto Grossi; the second part (this one) deals with Wavelet Tree-based text indexing and is included in the Festschrift dedicated to Giovanni Manzini.

Cite As Get BibTex

Paolo Ferragina, Raffaele Giancarlo, Roberto Grossi, Giovanna Rosone, Rossano Venturini, and Jeffrey Scott Vitter. Wavelet Tree, Part II: Text Indexing. In The Expanding World of Compressed Data: A Festschrift for Giovanni Manzini's 60th Birthday. Open Access Series in Informatics (OASIcs), Volume 131, pp. 4:1-4:10, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/OASIcs.Manzini.4

Author Details

Paolo Ferragina
  • Department L’EMbeDS, Sant’Anna School of Advanced Studies, Pisa, Italy
  • Department of Computer Science, University of Pisa, Italy
Raffaele Giancarlo
  • Department of Mathematics and Computer Science, University of Palermo, Italy
Roberto Grossi
  • Department of Computer Science, University of Pisa, Italy
Giovanna Rosone
  • Department of Computer Science, University of Pisa, Italy
Rossano Venturini
  • Department of Computer Science, University of Pisa, Italy
Jeffrey Scott Vitter
  • Department of Computer Science, Tulane University, New Orleans, LA, USA
  • Department of Computer and Information Science, The University of Mississippi, MS, USA

Acknowledgements

The authors would like to thank Hongwei Huo for helpful comments and figures.

References

  1. M. Burrows and D. J. Wheeler. A block-sorting lossless data compression algorithm, 1994. Google Scholar
  2. Paolo Ferragina. Pearls of Algorithm Engineering. Cambridge University Press, 2023. Google Scholar
  3. Paolo Ferragina, Raffaele Giancarlo, and Giovanni Manzini. The myriad virtues of Wavelet Trees. Information and Computation, 207(8):849-866, 2009. URL: https://doi.org/10.1016/j.ic.2008.12.010.
  4. Paolo Ferragina, Raffaele Giancarlo, Giovanni Manzini, Giovanna Rosone, Rossano Venturini, and Jeffrey Scott Vitter. Wavelet Tree, Part I: A Brief History. appearing in the Festschrift dedicated to Roberto Grossi. Google Scholar
  5. Paolo Ferragina, Raffaele Giancarlo, Giovanni Manzini, and Marinella Sciortino. Boosting textual compression in optimal linear time. Journal of the ACM, 52(4):688-713, July 2005. URL: https://doi.org/10.1145/1082036.1082043.
  6. Paolo Ferragina and Giovanni Manzini. Opportunistic data structures with applications. In Proceedings of the 41st Annual IEEE Symposium on Foundations of Computer Science (FOCS'00), pages 390-398, Washington, DC, November 2000. IEEE Computer Society. URL: https://doi.org/10.1109/SFCS.2000.892127.
  7. Paolo Ferragina and Giovanni Manzini. Indexing compressed text. Journal of the ACM, 52(4):552-581, 2005. URL: https://doi.org/10.1145/1082036.1082039.
  8. Paolo Ferragina, Giovanni Manzini, Veli Mäkinen, and Gonzalo Navarro. Compressed representations of sequences and full-text indexes. ACM Transactions on Algorithms, 3(2):Article 20, 2007. An earlier version appeared in Proceedings of the 11th Symposium on String Processing and Information Retrieval (SPIRE), 150–160, October 2004. Google Scholar
  9. Luca Foschini, Roberto Grossi, Ankur Gupta, and Jeffrey S. Vitter. When indexing equals compression: Experiments with compressing suffix arrays and applications. ACM Transactions on Algorithms, 2(4):611-639, 2006. This work combined earlier versions from SIAM Symposium on Discrete Algorithms (SODA), January 2004, and "Fast compression with a static model in high-order entropy," Data Compression Conference (DCC), March 2004. URL: https://doi.org/10.1145/1198513.1198521.
  10. Simon Gog, Juha Kärkkäinen, Dominik Kempa, Matthias Petri, and Simon J. Puglisi. Fixed block compression boosting in FM-Indexes: Theory and practice. Algorithmica, 81:1370-1391, 2019. URL: https://doi.org/10.1007/S00453-018-0475-9.
  11. Roberto Grossi, Ankur Gupta, and Jeffrey S. Vitter. High-order entropy-compressed text indexes. In Proceedings of the 14th ACM-SIAM Symposium on Discrete Algorithms (SODA'03), pages 841-850, New York, NY, January 2003. Association for Computing Machinery. Google Scholar
  12. Roberto Grossi and Jeffrey S. Vitter. Compressed suffix arrays and suffix trees with applications to text indexing and string matching. SIAM Journal on Computing, 35(2):378-407, 2005. An earlier version appeared in Proceedings of the 32nd ACM Symposium on Theory of Computing (STOC'00), May 2000, 397-406. URL: https://doi.org/10.1137/S0097539702402354.
  13. Roberto Grossi, Jeffrey S. Vitter, and Bojian Xu. Wavelet trees: From theory to practice. In 2011 First International Conference on Data Compression, Communications and Processing, pages 210-221, June 2011. URL: https://doi.org/10.1109/CCP.2011.16.
  14. Wing-Kai Hon, Kunihiko Sadakane, and Wing-Kin Sung. Breaking a time-and-space barrier in constructing full-text indices. SIAM J. Comput., 38(6):2162-2178, 2009. URL: https://doi.org/10.1137/070685373.
  15. Hongwei Huo, Zongtao He, Pengfei Liu, and Jeffrey S. Vitter. FM-Adaptive: A practical data-aware FM-index, 2025. submitted. Google Scholar
  16. Juha Kärkkäinen, Peter Sanders, and Stefan Burkhardt. Linear work suffix array construction. Journal of the ACM, 53(6):918-936, 2006. URL: https://doi.org/10.1145/1217856.1217858.
  17. Dong Kyue Kim, Jeong Seop Sim, Heejin Park, and Kunsoo Park. Constructing suffix arrays in linear time. Journal of Discrete Algorithms, 3(2):126-142, 2005. URL: https://doi.org/10.1016/J.JDA.2004.08.019.
  18. Alan Kuhnle, Taher Mun, Christina Boucher, Travis Gagie, Ben Langmead, and Giovanni Manzini. Efficient construction of a complete index for pan-genomics read alignment. In Lenore J. Cowen, editor, Research in Computational Molecular Biology - 23rd Annual International Conference, RECOMB 2019, Washington, DC, USA, May 5-8, 2019, Proceedings, volume 11467 of Lecture Notes in Computer Science, pages 158-173. Springer, 2019. URL: https://doi.org/10.1007/978-3-030-17083-7_10.
  19. Veli Mäkinen and Gonzalo Navarro. Implicit compression boosting with applications to self-indexing. In Proceedings of the 14th International Symposium on String Processing and Information Retrieval (SPIRE'07), pages 229-241, Heidelberg, Germany, 2007. Springer-Verlag Berlin. URL: https://doi.org/10.1007/978-3-540-75530-2_21.
  20. Veli Mäkinen and Gonzalo Navarro. Dynamic entropy-compressed sequences and full-text indexes. ACM Transactions on Algorithms, 4(3):article 32, 2008. Google Scholar
  21. Giovanni Manzini. An analysis of the Burrows-Wheeler transform. Journal of the ACM, 48(3):407-430, 2001. An earlier version appeared in Proceedings of the 10th Symposium on Discrete Algorithms (SODA ’99), January 1999, 669-677. URL: https://doi.org/10.1145/382780.382782.
  22. Giovanni Manzini and Paolo Ferragina. Engineering a lightweight suffix array construction algorithm. In Rolf H. Möhring and Rajeev Raman, editors, Algorithms - ESA 2002, 10th Annual European Symposium, Rome, Italy, September 17-21, 2002, Proceedings, volume 2461 of Lecture Notes in Computer Science, pages 698-710. Springer, 2002. URL: https://doi.org/10.1007/3-540-45749-6_61.
  23. J. Ian Munro, Gonzalo Navarro, and Yakov Nekrich. Fast compressed self-indexes with deterministic linear-time construction. Algorithmica, 82(2):316-337, 2020. A conference version of this paper appeared in Proc. ISAAC 2017. URL: https://doi.org/10.1007/S00453-019-00637-X.
  24. Gonzalo Navarro and Veli Mäkinen. Compressed full-text indexes. ACM Computing Surveys, 39(1):Article 2, 2007. Google Scholar
  25. Daisuke Okanohara and Kunihiko Sadakane. A linear-time burrows-wheeler transform using induced sorting. In Jussi Karlgren, Jorma Tarhio, and Heikki Hyyrö, editors, String Processing and Information Retrieval, 16th International Symposium, SPIRE 2009, Saariselkä, Finland, August 25-27, 2009, Proceedings, volume 5721 of Lecture Notes in Computer Science, pages 90-101. Springer, 2009. URL: https://doi.org/10.1007/978-3-642-03784-9_9.
  26. Rajeev Raman, Venkatesh Raman, and S.Srinivasa Rao. Succinct indexable dictionaries with applications to encoding k-ary trees, prefix sums and multisets. ACM Transactions on Algorithms, 3(4):Article 43, 2007. URL: https://doi.org/10.1145/1290672.1290680.
  27. Kunihiko Sadakane. Compressed suffix trees with full functionality. Theory Comput. Syst., 41(4):589-607, 2007. URL: https://doi.org/10.1007/S00224-006-1198-X.
  28. Kunihiko Sadakane. Compressed text databases with efficient query algorithms based on the compressed suffix array. In Proceedings of the 11th Symposium on Algorithms and Computation (ISAAC'00), pages 410-421, Banff Alberta, Canada, August 2000. Springer-Verlag Berlin, Heidelberg. Google Scholar
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact