Document Open Access Logo

ZOR Filters: Fast and Smaller Than Fuse Filters

Author Antoine Limasset

PDF

File

Thumbnail PDF
PDF
LIPIcs.SEA.2026.24.pdf
  • Filesize: 1.84 MB
  • 17 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Theory of computation → Data structures design and analysis
Keywords
  • Data structure
  • Approximate Set Membership
  • Static filter

Metrics

Abstract

Probabilistic membership filters support fast approximate membership queries with controlled false-positive probability ε and are widely used across storage, analytics, networking, and bioinformatics [Chang et al., 2008; Niv Dayan et al., 2018; Broder and Mitzenmacher, 2004; Harris and Medvedev, 2020; Marchet and Limasset, 2023; Chikhi et al., 2025; Hernandez-Courbevoie et al., 2025]. In the static setting, low-overhead methods such as XOR, Fuse, and BuRR have been proposed [Graf and Lemire, 2020; Graf and Lemire, 2022; Dillinger et al., 2022; Ulrich and Renard, 2023]. Among these, Fuse filters are known for near-optimal query throughput. For XOR/Fuse-style peeling constructions, however, build success is only high probability, which complicates deterministic builds.
We introduce ZOR filters, a deterministic continuation of XOR/Fuse-style constructions that guarantees termination while preserving the same XOR-based query mechanism. ZOR replaces restart-on-failure with deterministic peeling that abandons a small fraction of keys, and restores false-positive-only semantics by storing the remainder in a compact auxiliary structure. In our experiments, the abandoned fraction drops below 1% for moderate arity (e.g., N ≥ 5), so the auxiliary handles a negligible fraction of keys. As a result, ZOR filters can be substantially more memory-efficient than Fuse filters, with overhead below 1%, while not yet matching the near-optimal overhead of BuRR (below 0.1%). In query performance, ZOR-pure is close to Fuse and faster than BuRR on positive queries, while the complete interleaved variant trades additional negative-query latency for deterministic continuation. Relative to optimised Fuse/BuRR implementations [Graf and Lemire, 2022; Dillinger et al., 2022], the current ZOR prototype remains slower in construction because deterministic peeling requires explicit incidence handling; reducing this construction gap is an important direction for future work.

Cite As Get BibTex

Antoine Limasset. ZOR Filters: Fast and Smaller Than Fuse Filters. In 24th International Symposium on Experimental Algorithms (SEA 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 371, pp. 24:1-24:17, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026) https://doi.org/10.4230/LIPIcs.SEA.2026.24

Author Details

Antoine Limasset
  • Univ. Lille, CNRS, Centrale Lille, UMR 9189 CRIStAL, F-59000 Lille, France

Funding

This work was supported by the French National Research Agency AGATE [ANR-21-CE45-0012] and full-RNA [ANR-22-CE45-0007]. With financial support from ITMO Cancer of Aviesan within the framework of the 2021-2030 Cancer Control Strategy, on funds administered by Inserm.

Supplementary Materials

References

  1. Michael A Bender, Martin Farach-Colton, Rob Johnson, Bradley C Kuszmaul, Dzejla Medjedovic, Pablo Montes, Pradeep Shetty, Richard P Spillane, and Erez Zadok. Don't thrash: How to cache your hash on flash. In 3rd Workshop on Hot Topics in Storage and File Systems (HotStorage 11), 2011. URL: https://doi.org/10.48550/arXiv.1208.0290.
  2. Burton H Bloom. Space/time trade-offs in hash coding with allowable errors. Communications of the ACM, 13(7):422-426, 1970. URL: https://doi.org/10.1145/362686.362692.
  3. Alex D Breslow and Nuwan S Jayasena. Morton filters: faster, space-efficient cuckoo filters via biasing, compression, and decoupled logical sparsity. Proceedings of the VLDB Endowment, 11(9):1041-1055, 2018. URL: https://doi.org/10.14778/3213880.3213884.
  4. Andrei Broder and Michael Mitzenmacher. Network applications of bloom filters: A survey. Internet mathematics, 1(4):485-509, 2004. URL: https://doi.org/10.1080/15427951.2004.10129096.
  5. Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C Hsieh, Deborah A Wallach, Mike Burrows, Tushar Chandra, Andrew Fikes, and Robert E Gruber. Bigtable: A distributed storage system for structured data. ACM Transactions on Computer Systems (TOCS), 26(2):1-26, 2008. URL: https://doi.org/10.1145/1365815.1365816.
  6. Rayan Chikhi, Téo Lemane, Raphaël Loll-Krippleber, Mercè Montoliu-Nerin, Brice Raffestin, Antonio Pedro Camargo, Carson J Miller, Mateus Bernabe Fiamenghi, Daniel Paiva Agustinho, Sina Majidian, et al. Logan: planetary-scale genome assembly surveys life’s diversity. bioRxiv, pages 2024-07, 2025. URL: https://doi.org/10.1101/2024.07.30.605881.
  7. Niv Dayan, Manos Athanassoulis, and Stratos Idreos. Optimal bloom filters and adaptive merging for lsm-trees. ACM Transactions on Database Systems, 43(4):1-48, 2018. URL: https://doi.org/10.1145/3276980.
  8. Fan Deng and Davood Rafiei. Approximately detecting duplicates for streaming data using stable bloom filters. In Proceedings of the 2006 ACM SIGMOD international conference on Management of data, pages 25-36, 2006. URL: https://doi.org/10.1145/1142473.1142477.
  9. Sarang Dharmapurikar, Praveen Krishnamurthy, and David E Taylor. Longest prefix matching using bloom filters. In Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications, pages 201-212, 2003. URL: https://doi.org/10.1145/863955.863979.
  10. Sarang Dharmapurikar, Haoyu Song, Jonathan Turner, and John Lockwood. Fast packet classification using bloom filters. In Proceedings of the 2006 ACM/IEEE symposium on Architecture for networking and communications systems, pages 61-70, 2006. URL: https://doi.org/10.1145/1185347.1185356.
  11. Martin Dietzfelbinger and Rasmus Pagh. Succinct data structures for retrieval and approximate membership. In International Colloquium on Automata, Languages, and Programming, pages 385-396. Springer, 2008. URL: https://doi.org/10.1007/978-3-540-70575-8_32.
  12. Peter C. Dillinger, Lorenz Hübschle-Schneider, Peter Sanders, and Stefan Walzer. Fast succinct retrieval and approximate membership using ribbon. In 20th International Symposium on Experimental Algorithms (SEA 2022), volume 233 of LIPIcs, pages 4:1-4:20. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2022. Software repository: https://github.com/lorenzhs/BuRR. URL: https://doi.org/10.4230/LIPIcs.SEA.2022.4.
  13. Peter C Dillinger and Stefan Walzer. Ribbon filter: practically smaller than bloom and xor. arXiv preprint arXiv:2103.02515, 2021. URL: https://doi.org/10.48550/arXiv.2103.02515.
  14. Benoit Donnet, Bruno Baynat, and Timur Friedman. Retouched bloom filters: allowing networked applications to trade off selected false positives against false negatives. In Proceedings of the 2006 ACM CoNEXT conference, pages 1-12, 2006. URL: https://doi.org/10.1145/1368436.1368454.
  15. David Eppstein. Cuckoo filter: Simplification and analysis. arXiv preprint arXiv:1604.06067, 2016. URL: https://doi.org/10.48550/arXiv.1604.06067.
  16. Bin Fan, Dave G Andersen, Michael Kaminsky, and Michael D Mitzenmacher. Cuckoo filter: Practically better than bloom. In Proceedings of the 10th ACM International on Conference on emerging Networking Experiments and Technologies, pages 75-88, 2014. URL: https://doi.org/10.1145/2674005.2674994.
  17. Li Fan, Pei Cao, Jussara Almeida, and Andrei Z Broder. Summary cache: a scalable wide-area web cache sharing protocol. IEEE/ACM transactions on networking, 8(3):281-293, 2000. URL: https://doi.org/10.1109/90.851975.
  18. Thomas Mueller Graf and Daniel Lemire. Xor filters: Faster and smaller than bloom and cuckoo filters. Journal of Experimental Algorithmics (JEA), 25:1-16, 2020. URL: https://doi.org/10.1145/3376122.
  19. Thomas Mueller Graf and Daniel Lemire. Binary fuse filters: Fast and smaller than xor filters. Journal of Experimental Algorithmics (JEA), 27(1):1-15, 2022. URL: https://doi.org/10.1145/3510449.
  20. Sven Groppe, Thomas Kiencke, Stefan Werner, Dennis Heinrich, Marc Stelzner, and Le Gruenwald. P-luposdate: using precomputed bloom filters to speed up sparql processing in the cloud. Open Journal of Semantic Web (OJSW), 1(2):25-55, 2014. No DOI registered (checked on April 10, 2026). URL: https://nbn-resolving.org/urn:nbn:de:101:1-201705194858.
  21. Robert S Harris and Paul Medvedev. Improved representation of sequence bloom trees. Bioinformatics, 36(3):721-727, 2020. URL: https://doi.org/10.1093/bioinformatics/btz662.
  22. Stefan Hermann, Hans-Peter Lehmann, Giulio Ermanno Pibiri, Peter Sanders, and Stefan Walzer. Phobic: Perfect hashing with optimized bucket sizes and interleaved coding. In 32nd Annual European Symposium on Algorithms (ESA 2024), Leibniz International Proceedings in Informatics (LIPIcs), 2024. URL: https://doi.org/10.4230/LIPIcs.ESA.2024.69.
  23. Yohan Hernandez-Courbevoie, Mikaël Salson, Chloé Bessière, Haoliang Xue, Daniel Gautheret, Camille Marchet, and Antoine Limasset. Reindeer2: Practical abundance index at scale. In International Symposium on String Processing and Information Retrieval (SPIRE 2025), Lecture Notes in Computer Science, pages 156-171. Springer, 2025. URL: https://doi.org/10.1007/978-3-032-05228-5_14.
  24. Avinash Lakshman and Prashant Malik. Cassandra: a decentralized structured storage system. ACM SIGOPS operating systems review, 44(2):35-40, 2010. URL: https://doi.org/10.1145/1773912.1773922.
  25. Harald Lang, Thomas Neumann, Alfons Kemper, and Peter Boncz. Performance-optimal filtering: Bloom overtakes cuckoo at high throughput. Proceedings of the VLDB Endowment, 12(5):502-515, 2019. URL: https://doi.org/10.14778/3303753.3303757.
  26. Taewhi Lee, Kisung Kim, and Hyoung-Joo Kim. Join processing using bloom filter in mapreduce. In Proceedings of the 2012 ACM Research in Applied Computation Symposium, pages 100-105, 2012. URL: https://doi.org/10.1145/2401603.2401626.
  27. Hans-Peter Lehmann, Thomas Mueller, Rasmus Pagh, Giulio Ermanno Pibiri, Peter Sanders, Sebastiano Vigna, and Stefan Walzer. Modern minimal perfect hashing: A survey. ACM Computing Surveys, 58(10):1-36, 2026. URL: https://doi.org/10.1145/3797036.
  28. Antoine Limasset, Guillaume Rizk, Rayan Chikhi, and Pierre Peterlongo. Fast and scalable minimal perfect hashing for massive key sets. In 24th International Symposium on Experimental Algorithms (SEA 2017), volume 75 of LIPIcs, pages 25:1-25:16. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2017. URL: https://doi.org/10.4230/LIPIcs.SEA.2017.25.
  29. Lothar F Mackert and Guy M Lohman. R* optimizer validation and performance evaluation for local queries. ACM SIGMOD Record, 15(2):84-95, 1986. URL: https://doi.org/10.1145/16856.16863.
  30. Camille Marchet, Mael Kerbiriou, and Antoine Limasset. Blight: efficient exact associative structure for k-mers. Bioinformatics, 37(18):2858-2865, 2021. URL: https://doi.org/10.1093/bioinformatics/btab217.
  31. Camille Marchet, Lolita Lecompte, Antoine Limasset, Lucie Bittner, and Pierre Peterlongo. A resource-frugal probabilistic dictionary and applications in bioinformatics. Discrete Applied Mathematics, 274:92-102, 2020. URL: https://doi.org/10.1016/j.dam.2018.03.035.
  32. Camille Marchet and Antoine Limasset. Scalable sequence database search using partitioned aggregated bloom comb trees. Bioinformatics, 39(Supplement_1):i252-i259, 2023. URL: https://doi.org/10.1093/bioinformatics/btad225.
  33. Patrick O’Neil, Edward Cheng, Dieter Gawlick, and Elizabeth O’Neil. The log-structured merge-tree (lsm-tree). Acta informatica, 33(4):351-385, 1996. URL: https://doi.org/10.1007/s002360050048.
  34. Prashant Pandey, Alex Conway, Joe Durie, Michael A Bender, Martin Farach-Colton, and Rob Johnson. Vector quotient filters: Overcoming the time/space trade-off in filter design. In Proceedings of the 2021 International Conference on Management of Data, pages 1386-1399, 2021. URL: https://doi.org/10.1145/3448016.3452841.
  35. Giulio Ermanno Pibiri. On weighted k-mer dictionaries. Algorithms for Molecular Biology, 18(1):3, 2023. URL: https://doi.org/10.1186/s13015-023-00226-2.
  36. Giulio Ermanno Pibiri, Yoshihiro Shibuya, and Antoine Limasset. Locality-preserving minimal perfect hashing of k-mers. Bioinformatics, 39(Supplement_1):i534-i543, 2023. URL: https://doi.org/10.1093/bioinformatics/btad219.
  37. Giulio Ermanno Pibiri and Roberto Trani. Pthash: Revisiting fch minimal perfect hashing. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2021), pages 1339-1348. ACM, 2021. URL: https://doi.org/10.1145/3404835.3462849.
  38. Felix Putze, Peter Sanders, and Johannes Singler. Cache-, hash-and space-efficient bloom filters. In International Workshop on Experimental and Efficient Algorithms, pages 108-121. Springer, 2007. URL: https://doi.org/10.1007/978-3-540-72845-0_9.
  39. Kaihua Qin, Henryk Hadass, Arthur Gervais, and Joel Reardon. Applying private information retrieval to lightweight bitcoin clients. In 2019 Crypto Valley Conference on Blockchain Technology (CVCBT), pages 60-72. IEEE, 2019. URL: https://doi.org/10.1109/CVCBT.2019.00012.
  40. Sukriti Ramesh, Odysseas Papapetrou, and Wolf Siberski. Optimizing distributed joins with bloom filters. In International Conference on Distributed Computing and Internet Technology, pages 145-156. Springer, 2008. URL: https://doi.org/10.1007/978-3-540-89737-8_15.
  41. Alex C Snoeren, Craig Partridge, Luis A Sanchez, Christine E Jones, Fabrice Tchakountio, Stephen T Kent, and W Timothy Strayer. Hash-based ip traceback. ACM SIGCOMM Computer Communication Review, 31(4):3-14, 2001. URL: https://doi.org/10.1145/383059.383060.
  42. Sasu Tarkoma, Christian Esteve Rothenberg, and Eemil Lagerspetz. Theory and practice of bloom filters for distributed systems. IEEE Communications Surveys & Tutorials, 14(1):131-155, 2011. URL: https://doi.org/10.1109/SURV.2011.031611.00024.
  43. Jens-Uwe Ulrich and Bernhard Y Renard. Taxor: Fast and space-efficient taxonomic classification of long reads with hierarchical interleaved xor filters. bioRxiv, pages 2023-07, 2023. URL: https://doi.org/10.1101/2023.07.20.549822.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2026 Schloss Dagstuhl – LZI GmbH Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact