Document Open Access Logo

Engineering Minimal k-Perfect Hash Functions

Authors Stefan Hermann , Sebastian Kirmayer , Hans-Peter Lehmann , Peter Sanders , Stefan Walzer

PDF HTML 5

Files

Thumbnail PDF
PDF
LIPIcs.ESA.2025.99.pdf
  • Filesize: 1.26 MB
  • 18 pages
HTML5 Logo HTML (experimental)
LIPIcs.ESA.2025.99.html

Document Identifiers

Related Versions

Subject Classification

ACM Subject Classification
  • Theory of computation → Data compression
  • Theory of computation → Bloom filters and hashing
  • Information systems → Point lookups
Keywords
  • Compressed Data Structures
  • Perfect Hashing

Metrics

Abstract

Given a set S of n keys, a k-perfect hash function (kPHF) is a data structure that maps the keys to the first m integers, where each output integer can be hit by at most k input keys. When m = ⌈n/k⌉, the resulting function is called a minimal k-perfect hash function (MkPHF). Applications of kPHFs can be found in external memory data structures or to create efficient 1-perfect hash functions, which in turn have a wide range of applications from databases to bioinformatics.
Several papers from the 1980s look at external memory data structures with small internal memory indexes. However, actual k-perfect hash functions are surprisingly rare, and the area has not seen a lot of research recently. At the same time, recent research in 1-perfect hashing shows that there is a lack of efficient kPHFs. In this paper, we revive the area of k-perfect hashing, presenting four new constructions. Our implementations simultaneously dominate older approaches in space consumption, construction time, and query time. We see this paper as a possible starting point of an active line of research, similar to the area of 1-perfect hashing.

Cite As Get BibTex

Stefan Hermann, Sebastian Kirmayer, Hans-Peter Lehmann, Peter Sanders, and Stefan Walzer. Engineering Minimal k-Perfect Hash Functions. In 33rd Annual European Symposium on Algorithms (ESA 2025). Leibniz International Proceedings in Informatics (LIPIcs), Volume 351, pp. 99:1-99:18, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/LIPIcs.ESA.2025.99

Author Details

Stefan Hermann
  • Karlsruhe Institute of Technology, Germany
Sebastian Kirmayer
  • Karlsruhe Institute of Technology, Germany
Hans-Peter Lehmann
  • Karlsruhe Institute of Technology, Germany
Peter Sanders
  • Karlsruhe Institute of Technology, Germany
Stefan Walzer
  • Karlsruhe Institute of Technology, Germany

Funding

This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 882500). This work was also supported by funding from the pilot program Core Informatics at KIT (KiKIT) of the Helmholtz Association (HGF).

Acknowledgements

This paper is based on and has text overlaps with the bachelor’s thesis of Sebastian Kirmayer [Kirmayer, 2024].

Supplementary Materials

References

  1. Noga Alon, Raphael Yuster, and Uri Zwick. Color-coding. Journal of the ACM (JACM), 42(4):844-856, 1995. URL: https://doi.org/10.1145/210332.210337.
  2. Djamal Belazzougui, Fabiano C. Botelho, and Martin Dietzfelbinger. Hash, displace, and compress. In ESA, volume 5757 of Lecture Notes in Computer Science, pages 682-693. Springer, 2009. URL: https://doi.org/10.1007/978-3-642-04128-0_61.
  3. Piotr Beling. Fingerprinting-based minimal perfect hashing revisited. ACM J. Exp. Algorithmics, 28:1.4:1-1.4:16, 2023. URL: https://doi.org/10.1145/3596453.
  4. Francine Berman, Mary Ellen Bock, Eric Dittert, Michael J. O'Donnell, and Darrell Plank. Collections of functions for perfect hashing. SIAM J. Comput., 15(2):604-618, 1986. URL: https://doi.org/10.1137/0215044.
  5. Pedro Celia. External robin hood hashing. Technical report, Computer Science Department, Indiana University. TR246, 1988. Google Scholar
  6. Davi de Castro Reis, Djamel Belazzougui, Fabiano Cupertino Botelho, and Nivio Ziviani. CMPH - C minimal perfect hashing library. http://cmph.sourceforge.net/, 2012.
  7. Peter C. Dillinger, Lorenz Hübschle-Schneider, Peter Sanders, and Stefan Walzer. Fast succinct retrieval and approximate membership using ribbon. In SEA, volume 233 of LIPIcs, pages 4:1-4:20. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2022. URL: https://doi.org/10.4230/LIPICS.SEA.2022.4.
  8. Peter Elias. Efficient storage and retrieval by content and address of static files. J. ACM, 21(2):246-260, 1974. URL: https://doi.org/10.1145/321812.321820.
  9. Emmanuel Esposito, Thomas Mueller Graf, and Sebastiano Vigna. RecSplit: Minimal perfect hashing via recursive splitting. In ALENEX, pages 175-185. SIAM, 2020. URL: https://doi.org/10.1137/1.9781611976007.14.
  10. Ronald Fagin, Jürg Nievergelt, Nicholas Pippenger, and H. Raymond Strong. Extendible hashing - A fast access method for dynamic files. ACM Trans. Database Syst., 4(3):315-344, 1979. URL: https://doi.org/10.1145/320083.320092.
  11. Robert Mario Fano. On the number of bits required to implement an associative memory. Technical report, MIT, Computer Structures Group, 1971. Project MAC, Memorandum 61". Google Scholar
  12. Dimitris Fotakis, Rasmus Pagh, Peter Sanders, and Paul G. Spirakis. Space efficient hash tables with worst case constant access time. Theory Comput. Syst., 38(2):229-248, 2005. URL: https://doi.org/10.1007/S00224-004-1195-X.
  13. Fabian Frei and David Wehner. Bounds for c-ideal hashing. In FCT, volume 14292 of Lecture Notes in Computer Science, pages 205-220. Springer, 2023. URL: https://doi.org/10.1007/978-3-031-43587-4_15.
  14. Solomon W. Golomb. Run-length encodings. IEEE Trans. Inf. Theory, 12(3):399-401, 1966. URL: https://doi.org/10.1109/TIT.1966.1053907.
  15. Gaston H. Gonnet and Per-Åke Larson. External hashing with limited internal storage. J. ACM, 35(1):161-184, 1988. URL: https://doi.org/10.1145/42267.42274.
  16. Ragnar Groot Koerkamp. Ptrhash: Minimal perfect hashing at RAM throughput. CoRR, abs/2502.15539, 2025. URL: https://doi.org/10.48550/arXiv.2502.15539.
  17. Stefan Hermann. MorphisHash: Improving space efficiency of shockhash for minimal perfect hashing. CoRR, abs/2503.10161, 2025. URL: https://doi.org/10.48550/arXiv.2503.10161.
  18. Stefan Hermann, Sebastian Kirmayer, Hans-Peter Lehmann, Peter Sanders, and Stefan Walzer. Engineering minimal k-perfect hash functions. CoRR, abs/2504.20001, 2025. URL: https://doi.org/10.48550/arXiv.2504.20001.
  19. Stefan Hermann, Sebastian Kirmayer, Hans-Peter Lehmann, and Stefan Walzer. Engineering Minimal k-Perfect Hash Functions. Software, (visited on 2025-09-08). URL: https://github.com/stefanfred/engineering-k-perfect-hashing
    Software Heritage Logo archived version
    full metadata available at: https://doi.org/10.4230/artifacts.24698
  20. Stefan Hermann, Hans-Peter Lehmann, Giulio Ermanno Pibiri, Peter Sanders, and Stefan Walzer. PHOBIC: perfect hashing with optimized bucket sizes and interleaved coding. In ESA, volume 308 of LIPIcs, pages 69:1-69:17. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2024. URL: https://doi.org/10.4230/LIPICS.ESA.2024.69.
  21. C. A. R. Hoare. Algorithm 65: find. Commun. ACM, 4(7):321-322, July 1961. URL: https://doi.org/10.1145/366622.366647.
  22. Sebastian Kirmayer. Engineering k-perfect hashing. Abschlussarbeit - Bachelor, Karlsruher Institut für Technologie (KIT), 2024. URL: https://doi.org/10.5445/IR/1000179026.
  23. Florian Kurpicz, Hans-Peter Lehmann, and Peter Sanders. PaCHash: Packed and compressed hash tables. In ALENEX, pages 162-175. SIAM, 2023. URL: https://doi.org/10.1137/1.9781611977561.CH14.
  24. Per-Åke Larson. Linear hashing with separators - A dynamic hashing scheme achieving one-access retrieval. ACM Trans. Database Syst., 13(3):366-388, 1988. URL: https://doi.org/10.1145/44498.44500.
  25. Per-Åke Larson and Ajay Kajla. File organization: Implementation of a method guaranteeing retrieval in one access. Commun. ACM, 27(7):670-677, 1984. URL: https://doi.org/10.1145/358105.358193.
  26. Per-Åke Larson and M. V. Ramakrishna. External perfect hashing. In SIGMOD Conference, pages 190-200. ACM Press, 1985. URL: https://doi.org/10.1145/318898.318916.
  27. Hans-Peter Lehmann. Fast and Space-Efficient Perfect Hashing. PhD thesis, Karlsruhe Institute of Technology, Germany, 2024. URL: https://doi.org/10.5445/IR/1000176432.
  28. Hans-Peter Lehmann, Thomas Mueller, Rasmus Pagh, Giulio Ermanno Pibiri, Peter Sanders, Sebastiano Vigna, and Stefan Walzer. Modern minimal perfect hashing: A survey. CoRR, abs/2506.06536, 2025. URL: https://doi.org/10.48550/arXiv.2506.06536.
  29. Hans-Peter Lehmann, Peter Sanders, and Stefan Walzer. SicHash - small irregular cuckoo tables for perfect hashing. In ALENEX, pages 176-189. SIAM, 2023. URL: https://doi.org/10.1137/1.9781611977561.CH15.
  30. Hans-Peter Lehmann, Peter Sanders, and Stefan Walzer. ShockHash: Near optimal-space minimal perfect hashing beyond brute-force. CoRR, abs/2310.14959, 2024. URL: https://doi.org/10.48550/arXiv.2310.14959.
  31. Hans-Peter Lehmann, Peter Sanders, Stefan Walzer, and Jonatan Ziegler. Combined search and encoding for seeds, with an application to minimal perfect hashing. In 33nd Annual European Symposium on Algorithms (ESA 2025), Leibniz International Proceedings in Informatics (LIPIcs). Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2025. URL: https://doi.org/10.4230/LIPIcs.ESA.2025.110.
  32. Harry G. Mairson. The program complexity of searching a table. In FOCS, pages 40-47. IEEE Computer Society, 1983. URL: https://doi.org/10.1109/SFCS.1983.76.
  33. Harry G Mairson. The effect of table expansion on the program complexity of perfect hash functions. BIT Numerical Mathematics, 32:430-440, 1992. URL: https://doi.org/10.1007/BF02074879.
  34. Kurt Mehlhorn. Data structures and algorithms, vol. 1: Sorting and searching. EATCS Monographs on Theoretical Computer Science, Springer-Verlag, 1984. Google Scholar
  35. Ingo Müller, Peter Sanders, Robert Schulze, and Wei Zhou. Retrieval and perfect hashing using fingerprinting. In SEA, volume 8504 of Lecture Notes in Computer Science, pages 138-149. Springer, 2014. URL: https://doi.org/10.1007/978-3-319-07959-2_12.
  36. Rasmus Pagh. Basic external memory data structures. In Algorithms for Memory Hierarchies, volume 2625 of Lecture Notes in Computer Science, pages 14-35. Springer, 2002. URL: https://doi.org/10.1007/3-540-36574-5_2.
  37. Giulio Ermanno Pibiri and Roberto Trani. PTHash: Revisiting FCH minimal perfect hashing. In SIGIR, pages 1339-1348. ACM, 2021. URL: https://doi.org/10.1145/3404835.3462849.
  38. Robert F. Rice. Some practical universal noiseless coding techniques. Jet Propulsion Laboratory, JPL Publication, 1979. Google Scholar
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact