Document Open Access Logo

MorphisHash: Improving Space Efficiency of ShockHash for Minimal Perfect Hashing

Author Stefan Hermann

PDF

File

Thumbnail PDF
PDF
LIPIcs.ESA.2025.9.pdf
  • Filesize: 0.96 MB
  • 16 pages

Document Identifiers

Related Versions

Subject Classification

ACM Subject Classification
  • Theory of computation → Data compression
  • Information systems → Point lookups
  • Theory of computation → Bloom filters and hashing
  • Mathematics of computing → Random graphs
Keywords
  • compressed data structure
  • perfect hashing
  • random graph
  • pseudoforest
  • component

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads
    0
    Metadata Views

Abstract

A minimal perfect hash function (MPHF) maps a set of n keys to unique positions {1, …, n}. Representing an MPHF requires at least log₂(e)≈ 1.443 bits per key. ShockHash is a technique to construct an MPHF and requires just slightly more space. It gives each key two random candidate positions. If each key can be mapped to one of its two candidate positions such that there is exactly one key mapped to each position, then an MPHF is found. If not, ShockHash repeats the process with a new set of random candidate positions. ShockHash has to store how many repetitions were required and for each key to which of the two candidate positions it is mapped. However, when a given set of candidate positions can be used as MPHF then there is not only one but multiple ways of mapping the keys to one of their candidate positions such that the mapping results in an MPHF. This redundancy makes up for the majority of the remaining space overhead in ShockHash. In this paper, we present MorphisHash which almost completely eliminates this redundancy. Our theoretical result is that MorphisHash saves Θ(ln(n)) bits in expectation compared to ShockHash. This corresponds to a factor of 20 less space overhead in practice. Just like ShockHash, MorphisHash can be used as a building block within RecSplit to obtain MorphisHash-RS. When compared for same space consumption, MorphisHash-RS can be constructed up to 21 times faster than ShockHash-RS. The technique to accomplish this might be of a more general interest to compress data structures.

Cite As Get BibTex

Stefan Hermann. MorphisHash: Improving Space Efficiency of ShockHash for Minimal Perfect Hashing. In 33rd Annual European Symposium on Algorithms (ESA 2025). Leibniz International Proceedings in Informatics (LIPIcs), Volume 351, pp. 9:1-9:16, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/LIPIcs.ESA.2025.9

Author Details

Stefan Hermann
  • Karlsruhe Institute of Technology, Germany

Funding

This work was supported by funding from the pilot program Core Informatics at KIT (KiKIT) of the Helmholtz Association (HGF).

Acknowledgements

I thank Stefan Walzer for proofreading an earlier version of this paper.

Supplementary Materials

References

  1. Djamal Belazzougui, Paolo Boldi, Rasmus Pagh, and Sebastiano Vigna. Fast prefix search in little space, with applications. In ESA (1), volume 6346 of Lecture Notes in Computer Science, pages 427-438. Springer, 2010. URL: https://doi.org/10.1007/978-3-642-15775-2_37.
  2. Djamal Belazzougui and Gonzalo Navarro. Alphabet-independent compressed text indexing. ACM Trans. Algorithms, 10(4):23:1-23:19, 2014. URL: https://doi.org/10.1145/2635816.
  3. Dominik Bez, Florian Kurpicz, Hans-Peter Lehmann, and Peter Sanders. High performance construction of recsplit based minimal perfect hash functions. In ESA, volume 274 of LIPIcs, pages 19:1-19:16. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2023. URL: https://doi.org/10.4230/LIPICS.ESA.2023.19.
  4. Jean Bourgain, Van H Vu, and Philip Matchett Wood. On the singularity probability of discrete random matrices. Journal of Functional Analysis, 258(2):559-603, 2010. Google Scholar
  5. Andrei Z. Broder and Michael Mitzenmacher. Survey: Network applications of Bloom filters: A survey. Internet Math., 1(4):485-509, 2003. URL: https://doi.org/10.1080/15427951.2004.10129096.
  6. Chin-Chen Chang and Chih-Yang Lin. Perfect hashing schemes for mining association rules. Comput. J., 48(2):168-179, 2005. URL: https://doi.org/10.1093/COMJNL/BXH074.
  7. Victoria G. Crawford, Alan Kuhnle, Christina Boucher, Rayan Chikhi, and Travis Gagie. Practical dynamic de bruijn graphs. Bioinform., 34(24):4189-4195, 2018. URL: https://doi.org/10.1093/BIOINFORMATICS/BTY500.
  8. Peter C Dillinger, Lorenz Hübschle-Schneider, Peter Sanders, and Stefan Walzer. Fast succinct retrieval and approximate membership using ribbon. arXiv preprint, 2021. URL: https://arxiv.org/abs/2109.01892.
  9. Emmanuel Esposito, Thomas Mueller Graf, and Sebastiano Vigna. RecSplit: Minimal perfect hashing via recursive splitting. In ALENEX, pages 175-185. SIAM, 2020. URL: https://doi.org/10.1137/1.9781611976007.14.
  10. Stefan Hermann. MorphisHash - GitHub. https://github.com/stefanfred/MorphisHash, 2025.
  11. Stefan Hermann. Morphishash: Improving space efficiency of shockhash for minimal perfect hashing. arXiv preprint, 2025. URL: https://doi.org/10.48550/arXiv.2503.10161.
  12. Stefan Hermann, Hans-Peter Lehmann, Giulio Ermanno Pibiri, Peter Sanders, and Stefan Walzer. PHOBIC: perfect hashing with optimized bucket sizes and interleaved coding. In ESA, volume 308 of LIPIcs, pages 69:1-69:17. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2024. URL: https://doi.org/10.4230/LIPIcs.ESA.2024.69.
  13. Hans-Peter Lehmann. MPHF Experiments - GitHub. https://github.com/ByteHamster/MPHF-Experiments, 2025.
  14. Hans-Peter Lehmann, Thomas Mueller, Rasmus Pagh, Giulio Ermanno Pibiri, Peter Sanders, Sebastiano Vigna, and Stefan Walzer. Modern minimal perfect hashing: A survey. arXiv preprint, 2025. URL: https://doi.org/10.48550/arXiv.2506.06536.
  15. Hans-Peter Lehmann, Peter Sanders, and Stefan Walzer. ShockHash: Near optimal-space minimal perfect hashing beyond brute-force. arXiv preprint, invited to Algorithmica, 2024. URL: https://doi.org/10.48550/arXiv.2310.14959.
  16. Hans-Peter Lehmann, Peter Sanders, and Stefan Walzer. Shockhash: Towards optimal-space minimal perfect hashing beyond brute-force. In ALENEX. SIAM, 2024. URL: https://doi.org/10.1137/1.9781611977929.15.
  17. Hans-Peter Lehmann, Peter Sanders, Stefan Walzer, and Jonatan Ziegler. Combined search and encoding for seeds, with an application to minimal perfect hashing. In ESA, LIPIcs. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2025. Google Scholar
  18. Yi Lu, Balaji Prabhakar, and Flavio Bonomi. Perfect hashing for network applications. In ISIT, pages 2774-2778. IEEE, 2006. URL: https://doi.org/10.1109/ISIT.2006.261567.
  19. Kurt Mehlhorn. On the program size of perfect and universal hash functions. In FOCS, pages 170-175. IEEE Computer Society, 1982. URL: https://doi.org/10.1109/SFCS.1982.80.
  20. MARK EJ Newman. Networks: an introduction, 2010. Google Scholar
  21. Anna Pagh and Rasmus Pagh. Uniform hashing in constant time and optimal space. SIAM Journal on Computing, 38(1):85-96, 2008. URL: https://doi.org/10.1137/060658400.
  22. Rasmus Pagh and Flemming Friche Rodler. Cuckoo hashing. Journal of Algorithms, 51(2):122-144, 2004. URL: https://doi.org/10.1016/J.JALGOR.2003.12.002.
  23. Giulio Ermanno Pibiri. Sparse and skew hashing of k-mers. Bioinformatics, 38(Supplement_1):i185-i194, 2022. URL: https://doi.org/10.1093/BIOINFORMATICS/BTAC245.
  24. Giulio Ermanno Pibiri and Rossano Venturini. Efficient data structures for massive N-gram datasets. In SIGIR, pages 615-624. ACM, 2017. URL: https://doi.org/10.1145/3077136.3080798.
  25. Lajos Takács. On cayley’s formula for counting forests. Journal of Combinatorial Theory, Series A, 53(2):321-323, 1990. URL: https://doi.org/10.1016/0097-3165(90)90064-4.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact