Document Open Access Logo

Elias-Fano Compression for Space-Efficient Rank and Select Structures

Authors Lannie Dalton Hough , Abhinav Bhatele

PDF HTML 5

Files

Thumbnail PDF
PDF
LIPIcs.SEA.2025.23.pdf
  • Filesize: 0.87 MB
  • 15 pages
HTML5 Logo HTML (experimental)
LIPIcs.SEA.2025.23.html

Document Identifiers

Subject Classification

ACM Subject Classification
  • Theory of computation → Sorting and searching
  • Theory of computation → Data compression
Keywords
  • rank and select
  • cache-aware
  • succinct data structures
  • bit vector

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

Bit vectors are an important component in many data structures. Such data structures are used in a variety of applications and domains including databases, search engines, and computational biology. Many use cases depend on being able to perform rank and/or select queries on the bit vector. No existing rank and select structure enabling these queries is most efficient both for space and for time; there is a tradeoff between the two. In practice, the smallest rank and select data structures, cs-poppy and pasta-flat, impose a space overhead of 3.51%, or 3.125% if only rank needs to be supported. In this paper, we present a new data structure, orzo, which reduces the overhead of the rank component by a further 26.5%. We preserve desirable cache-centric design decisions made in prior work, which allows us to minimize the performance penalty of creating a smaller data structure.

Cite As Get BibTex

Lannie Dalton Hough and Abhinav Bhatele. Elias-Fano Compression for Space-Efficient Rank and Select Structures. In 23rd International Symposium on Experimental Algorithms (SEA 2025). Leibniz International Proceedings in Informatics (LIPIcs), Volume 338, pp. 23:1-23:15, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/LIPIcs.SEA.2025.23

Author Details

Lannie Dalton Hough
  • Department of Computer Science, University of Maryland, College Park, MD, USA
Abhinav Bhatele
  • Department of Computer Science, University of Maryland, College Park, MD, USA

Funding

This material is based upon work supported in part by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, through solicitation DE-FOA-0003264, "Advancements in Artificial Intelligence for Science", under Award Number DE-SC0025598.

Acknowledgements

The authors acknowledge the University of Maryland supercomputing resources made available for conducting the research reported in this paper. We would also like to thank Dr. Rob Patro for sharing valuable insights into this area of research.

Supplementary Materials

References

  1. Peter Elias. On binary representations of monotone sequences. In Proceedings of the Sixth Princeton Conference on Information Sciences and Systems, Department of Electrical Engineering, Princeton University, Princeton, NJ, pages 54-57, 1972. Google Scholar
  2. Peter Elias. Efficient storage and retrieval by content and address of static files. J. ACM, 21(2):246-260, April 1974. URL: https://doi.org/10.1145/321812.321820.
  3. Robert Mario Fano. On the number of bits required to implement an associative memory. Massachusetts Institute of Technology, Project MAC, 1971. Google Scholar
  4. Paolo Ferragina and Giorgio Vinciguerra. Learned data structures. In Recent Trends in Learning From Data: Tutorials from the INNS Big Data and Deep Learning Conference (INNSBDDL2019), pages 5-41. Springer, 2020. URL: https://doi.org/10.1007/978-3-030-43883-8_2.
  5. Richard F Geary, Naila Rahman, Rajeev Raman, and Venkatesh Raman. A simple optimal representation for balanced parentheses. In Combinatorial Pattern Matching: 15th Annual Symposium, CPM 2004, Istanbul, Turkey, July 5-7, 2004. Proceedings 15, pages 159-172. Springer, 2004. URL: https://doi.org/10.1007/978-3-540-27801-6_12.
  6. Rodrigo González, Szymon Grabowski, Veli Mäkinen, and Gonzalo Navarro. Practical implementation of rank and select queries. In Poster Proc. Volume of 4th Workshop on Efficient and Experimental Algorithms (WEA), pages 27-38. CTI Press and Ellinika Grammata Greece, 2005. Google Scholar
  7. Roberto Grossi, Ankur Gupta, and Jeffrey Scott Vitter. High-order entropy-compressed text indexes. In Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA '03, pages 841-850, USA, 2003. Society for Industrial and Applied Mathematics. URL: http://dl.acm.org/citation.cfm?id=644108.644250.
  8. Roberto Grossi and JS Vitter. Compressed suffix arrays and suffix trees with applications to text indexing and string matching. In Proc. 32nd ACM Symp. Theory of Computing (STOC'2000), pages 397-406. Citeseer, 2000. Google Scholar
  9. Donald E Knuth. The art of computer programming, volume 4, fascicle 1: Bitwise tricks & techniques; binary decision diagrams. Addison-Wesley Professional, 2009. Google Scholar
  10. Tim Kraska, Alex Beutel, Ed H. Chi, Jeffrey Dean, and Neoklis Polyzotis. The case for learned index structures. CoRR, abs/1712.01208, 2017. URL: https://arxiv.org/abs/1712.01208.
  11. Florian Kurpicz. Engineering compact data structures for rank and select queries on bit vectors. In Diego Arroyuelo and Barbara Poblete, editors, String Processing and Information Retrieval, pages 257-272, Cham, 2022. Springer International Publishing. URL: https://doi.org/10.1007/978-3-031-20643-6_19.
  12. Matthew D. Laws, Jocelyn Bliven, Kit Conklin, Elyes Laalai, Samuel McCauley, and Zach S. Sturdevant. SPIDER: Improved Succinct Rank and Select Performance. In Leo Liberti, editor, 22nd International Symposium on Experimental Algorithms (SEA 2024), volume 301 of Leibniz International Proceedings in Informatics (LIPIcs), pages 21:1-21:18, Dagstuhl, Germany, 2024. Schloss Dagstuhl - Leibniz-Zentrum für Informatik. URL: https://doi.org/10.4230/LIPIcs.SEA.2024.21.
  13. Gonzalo Navarro and Eliana Providel. Fast, small, simple rank/select on bitmaps. In Proceedings of the 11th International Conference on Experimental Algorithms, SEA'12, pages 295-306, Berlin, Heidelberg, 2012. Springer-Verlag. URL: https://doi.org/10.1007/978-3-642-30850-5_26.
  14. Prashant Pandey, Michael A. Bender, and Rob Johnson. A fast x86 implementation of select. CoRR, abs/1706.00990, 2017. URL: https://arxiv.org/abs/1706.00990.
  15. Prashant Pandey, Michael A. Bender, Rob Johnson, and Rob Patro. A general-purpose counting filter: Making every bit count. In Proceedings of the 2017 ACM International Conference on Management of Data, SIGMOD '17, pages 775-787, New York, NY, USA, 2017. Association for Computing Machinery. URL: https://doi.org/10.1145/3035918.3035963.
  16. Mihai Patrascu. Succincter. In 2008 49th Annual IEEE Symposium on Foundations of Computer Science, pages 305-313, 2008. URL: https://doi.org/10.1109/FOCS.2008.83.
  17. Sebastiano Vigna. Broadword implementation of rank/select queries. In Proceedings of the 7th International Conference on Experimental Algorithms, WEA'08, pages 154-168, Berlin, Heidelberg, 2008. Springer-Verlag. URL: https://doi.org/10.1007/978-3-540-68552-4_12.
  18. Sebastiano Vigna. Quasi-succinct indices. In Proceedings of the sixth ACM international conference on Web search and data mining, pages 83-92, 2013. URL: https://doi.org/10.1145/2433396.2433409.
  19. Huacheng Yu. Optimal succinct rank data structure via approximate nonnegative tensor decomposition. In Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing, STOC 2019, pages 955-966, New York, NY, USA, 2019. Association for Computing Machinery. URL: https://doi.org/10.1145/3313276.3316352.
  20. Dong Zhou, David G. Andersen, and Michael Kaminsky. Space-efficient, high-performance rank and select structures on uncompressed bit sequences. In Vincenzo Bonifaci, Camil Demetrescu, and Alberto Marchetti-Spaccamela, editors, Experimental Algorithms, pages 151-163, Berlin, Heidelberg, 2013. Springer Berlin Heidelberg. URL: https://doi.org/10.1007/978-3-642-38527-8_15.
  21. Justin Zobel and Alistair Moffat. Inverted files for text search engines. ACM Comput. Surv., 38(2):6-es, July 2006. URL: https://doi.org/10.1145/1132956.1132959.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact