Document Open Access Logo

Fast Select Queries Using Hybrid Bitvectors

Authors Eric Chiu , Dominik Kempa

PDF

File

Thumbnail PDF
PDF
LIPIcs.SEA.2026.12.pdf
  • Filesize: 1.11 MB
  • 11 pages

Document Identifiers

Related Versions

Subject Classification

ACM Subject Classification
  • Theory of computation → Design and analysis of algorithms
  • Theory of computation → Data compression
  • Information systems → Information retrieval
Keywords
  • compressed bitvectors
  • hybrid bitvector
  • select queries

Metrics

Abstract

One of the central problems in the design of compressed data structures is the efficient support for rank and select queries on bitvectors. These two operations form the backbone of more complex data structures used for the compact representation of texts, trees, graphs, or grids. One effective solution is the so-called hybrid bitvector implementation, which partitions the input bitvector into blocks and adaptively selects an encoding method - such as run-length, plain, or minority encoding - based on local redundancy. Experiments have shown that hybrid bitvectors achieve excellent all-around performance on repetitive and non-repetitive inputs. Current hybrid bitvector implementations, however, support only rank queries (i.e., counting the number of ones up to a given position) and lack support for select queries (which ask for the position of a given occurrence of a given bit), which limits their applicability. In this paper, we propose a method to add support for select queries to hybrid bitvectors, and we evaluate the resulting implementation on repetitive and non-repetitive inputs. Our results show that hybrid bitvectors offer very strong all-around performance, combining high query speed with space efficiency and remaining consistently on or near the Pareto frontier.

Cite As Get BibTex

Eric Chiu and Dominik Kempa. Fast Select Queries Using Hybrid Bitvectors. In 24th International Symposium on Experimental Algorithms (SEA 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 371, pp. 12:1-12:11, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026) https://doi.org/10.4230/LIPIcs.SEA.2026.12

Author Details

Eric Chiu
  • Department of Computer Science, Stony Brook University, NY, USA
Dominik Kempa
  • Department of Computer Science, Stony Brook University, NY, USA

Funding

Partially funded by the NSF CAREER Award 2337891.

Supplementary Materials

References

  1. Djamal Belazzougui and Gonzalo Navarro. Alphabet-independent compressed text indexing. ACM Transactions on Algorithms, 10(4):23:1-23:19, 2014. URL: https://doi.org/10.1145/2635816.
  2. Antonio Boffa, Paolo Ferragina, and Giorgio Vinciguerra. A "learned" approach to quicken and compress rank/select dictionaries. In Proceedings of the 2021 SIAM Symposium on Algorithm Engineering and Experiments (ALENEX 2021), pages 46-59, 2021. URL: https://doi.org/10.1137/1.9781611976472.4.
  3. Antonio Boffa, Paolo Ferragina, and Giorgio Vinciguerra. A learned approach to design compressed rank/select data structures. ACM Transactions on Algorithms, 2022. URL: https://doi.org/10.1145/3524060.
  4. Michael Burrows. A block-sorting lossless data compression algorithm. SRS Research Report, 124, 1994. Google Scholar
  5. Laura Clarke, Xiangqun Zheng-Bradley, Richard Smith, Eugene Kulesha, Chunlin Xiao, Iliana Toneva, Brendan Vaughan, Don Preuss, Rasko Leinonen, Martin Shumway, et al. The 1000 genomes project: data management and community access. Nature Methods, 9(5):459-462, 2012. URL: https://doi.org/10.1038/nmeth.1974.
  6. Peter Elias. Efficient storage and retrieval by content and address of static files. Journal of the ACM, 21(2):246-260, 1974. URL: https://doi.org/10.1145/321812.321820.
  7. Robert M. Fano. On the Number of Bits Required to Implement an Associative Memory. MIT Project MAC Computer Structures Group, 1971. Google Scholar
  8. Paolo Ferragina and Giovanni Manzini. Indexing compressed text. Journal of the ACM, 52(4):552-581, 2005. URL: https://doi.org/10.1145/1082036.1082039.
  9. Paolo Ferragina, Giovanni Manzini, Veli Mäkinen, and Gonzalo Navarro. An alphabet-friendly FM-index. In Proceedings of the 11th International Conference on String Processing and Information Retrieval (SPIRE 2004), pages 150-160. Springer, 2004. URL: https://doi.org/10.1007/978-3-540-30213-1_23.
  10. Paolo Ferragina and Gonzalo Navarro. Pizza&Chili Corpus: Compressed Indexes and their Testbeds. URL: https://pizzachili.dcc.uchile.cl/.
  11. Johannes Fischer. Wee LCP. Information Processing Letters, 110(8-9):317-320, 2010. URL: https://doi.org/10.1016/J.IPL.2010.02.010.
  12. Simon Gog, Timo Beller, Alistair Moffat, and Matthias Petri. From theory to practice: Plug and play with succinct data structures. In Proceedings of the 13th International Symposium on Experimental Algorithms (SEA 2014), pages 326-337. Springer, 2014. URL: https://doi.org/10.1007/978-3-319-07959-2_28.
  13. Adrián Gómez-Brandón. Zombit: Exploiting runs in bitvectors. Software: Practice and Experience, 55(12):1975-1992, 2025. URL: https://doi.org/10.1002/SPE.70019.
  14. Roberto Grossi, Ankur Gupta, and Jeffrey Scott Vitter. High-order entropy-compressed text indexes. In Proceedings of the 14th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA 2003), pages 841-850. ACM/SIAM, 2003. URL: http://dl.acm.org/citation.cfm?id=644108.644250.
  15. Roberto Grossi and Jeffrey Scott Vitter. Compressed suffix arrays and suffix trees with applications to text indexing and string matching. SIAM Journal on Computing, 35(2):378-407, 2005. URL: https://doi.org/10.1137/S0097539702402354.
  16. Juha Kärkkäinen, Dominik Kempa, and Simon J. Puglisi. Hybrid compression of bitvectors for the FM-index. In Proceedings of the 2014 Data Compression Conference (DCC 2014), pages 302-311. IEEE, 2014. URL: https://doi.org/10.1109/DCC.2014.87.
  17. Dominik Kempa and Tomasz Kociumaka. Resolution of the Burrows-Wheeler transform conjecture. In Sandy Irani, editor, Proceedings of the 61st IEEE Annual Symposium on Foundations of Computer Science (FOCS 2020), pages 1002-1013. IEEE, 2020. URL: https://doi.org/10.1109/FOCS46700.2020.00097.
  18. Dominik Kempa and Tomasz Kociumaka. Breaking the O(n)-barrier in the construction of compressed suffix arrays and suffix trees. In Proceedings of the 2023 ACM-SIAM Symposium on Discrete Algorithms (SODA 2023), pages 5122-5202. SIAM, 2023. URL: https://doi.org/10.1137/1.9781611977554.CH187.
  19. Florian Kurpicz. Engineering compact data structures for rank and select queries on bit vectors. In Proceedings of the 29th International Symposium on String Processing and Information Retrieval (SPIRE 2022), pages 257-272. Springer, 2022. URL: https://doi.org/10.1007/978-3-031-20643-6_19.
  20. Udi Manber and Eugene W. Myers. Suffix arrays: A new method for on-line string searches. SIAM Journal on Computing, 22(5):935-948, 1993. URL: https://doi.org/10.1137/0222058.
  21. J. Ian Munro. Tables. In Proceedings of the 16th Conference on Foundations of Software Technology and Theoretical Computer Science (FSTTCS 1996), pages 37-42. Springer, 1996. URL: https://doi.org/10.1007/3-540-62034-6_35.
  22. Gonzalo Navarro. Compact data structures: A practical approach. Cambridge University Press, Cambridge, UK, 2016. URL: https://doi.org/10.1017/cbo9781316588284.
  23. Daisuke Okanohara and Kunihiko Sadakane. Practical entropy-compressed rank/select dictionary. In Proceedings of the Ninth Workshop on Algorithm Engineering and Experiments (ALENEX 2007). SIAM, 2007. URL: https://doi.org/10.1137/1.9781611972870.6.
  24. Mihai Pătraşcu. Succincter. In Proceedings of the 49th Annual IEEE Symposium on Foundations of Computer Science (FOCS 2008), pages 305-313. IEEE Computer Society, 2008. URL: https://doi.org/10.1109/FOCS.2008.83.
  25. Rajeev Raman, Venkatesh Raman, and Srinivasa Rao Satti. Succinct indexable dictionaries with applications to encoding k-ary trees, prefix sums and multisets. ACM Transactions on Algorithms, 3(4):43, 2007. URL: https://doi.org/10.1145/1290672.1290680.
  26. Kunihiko Sadakane. Succinct representations of lcp information and improvements in the compressed suffix arrays. In Proceedings of the 13th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA 2002), pages 225-232. ACM/SIAM, 2002. URL: http://dl.acm.org/citation.cfm?id=545381.545410.
  27. Kunihiko Sadakane. Compressed suffix trees with full functionality. Theory of Computing Systems, 41(4):589-607, 2007. URL: https://doi.org/10.1007/S00224-006-1198-X.
  28. Jouni Sirén, Niko Välimäki, Veli Mäkinen, and Gonzalo Navarro. Run-length compressed indexes are superior for highly repetitive sequence collections. In Proceedings of the 15th International Symposium on String Processing and Information Retrieval (SPIRE 2008), pages 164-175. Springer, 2008. URL: https://doi.org/10.1007/978-3-540-89097-3_17.
  29. Clare Turnbull, Richard H Scott, Ellen Thomas, Louise Jones, Nirupa Murugaesu, Freya Boardman Pretty, Dina Halai, Emma Baple, Clare Craig, Angela Hamblin, et al. The 100000 genomes project: bringing whole genome sequencing to the NHS. BMJ, 361, 2018. URL: https://doi.org/10.1136/bmj.k1687.
  30. Dong Zhou, David G. Andersen, and Michael Kaminsky. Space-efficient, high-performance rank and select structures on uncompressed bit sequences. In Vincenzo Bonifaci, Camil Demetrescu, and Alberto Marchetti-Spaccamela, editors, Proceedings of the 12th International Symposium on Experimental Algorithms (SEA 2013), pages 151-163. Springer, 2013. URL: https://doi.org/10.1007/978-3-642-38527-8_15.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2026 Schloss Dagstuhl – LZI GmbH Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact