Concurrent Expandable AMQs on the Basis of Quotient Filters

Authors Tobias Maier , Peter Sanders , Robert Williger



PDF
Thumbnail PDF

File

LIPIcs.SEA.2020.15.pdf
  • Filesize: 0.63 MB
  • 13 pages

Document Identifiers

Author Details

Tobias Maier
  • Karlsruhe Institute of Technology, Germany
Peter Sanders
  • Karlsruhe Institute of Technology, Germany
Robert Williger
  • Karlsruhe Institute of Technology, Germany

Cite As Get BibTex

Tobias Maier, Peter Sanders, and Robert Williger. Concurrent Expandable AMQs on the Basis of Quotient Filters. In 18th International Symposium on Experimental Algorithms (SEA 2020). Leibniz International Proceedings in Informatics (LIPIcs), Volume 160, pp. 15:1-15:13, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2020) https://doi.org/10.4230/LIPIcs.SEA.2020.15

Abstract

A quotient filter is a cache efficient Approximate Membership Query (AMQ) data structure. Depending on the fill degree of the filter most insertions and queries only need to access one or two consecutive cache lines. This makes quotient filters very fast compared to the more commonly used Bloom filters that incur multiple independent memory accesses depending on the false positive rate. However, concurrent Bloom filters are easy to implement and can be implemented lock-free while concurrent quotient filters are not as simple. Usually concurrent quotient filters work by using an external array of locks - each protecting a region of the table. Accessing this array incurs one additional memory access per operation. We propose a new locking scheme that has no memory overhead. Using this new locking scheme we achieve 1.6× times higher insertion performance and over 2.1× higher query performance than with the common external locking scheme.
Another advantage of quotient filters over Bloom filters is that a quotient filter can change its capacity when it is becoming full. We implement this growing technique for our concurrent quotient filters and adapt it in a way that allows unbounded growing while keeping a bounded false positive rate. We call the resulting data structure a fully expandable quotient filter. Its design is similar to scalable Bloom filters, but we exploit some concepts inherent to quotient filters to improve the space efficiency and the query speed.
Additionally, we propose several quotient filter variants that are aimed to reduce the number of status bits (2-status-bit variant) or to simplify concurrent implementations (linear probing quotient filter). The linear probing quotient filter even leads to a lock-free concurrent filter implementation. This is especially interesting, since we show that any lock-free implementation of other common quotient filter variants would incur significant overheads in the form of additional data fields or multiple passes over the accessed data. The code produced as part of this submission can be found at https://www.github.com/Toobiased/lpqfilter.

Subject Classification

ACM Subject Classification
  • Theory of computation → Data structures design and analysis
  • Theory of computation → Shared memory algorithms
Keywords
  • Quotient filter
  • Concurrent data structures
  • Locking

Metrics

  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    0
    PDF Downloads

References

  1. Mohammad Al-hisnawi and Mahmood Ahmadi. Deep Packet Inspection using Quotient Filter. IEEE Communications Letters, 20(11):2217-2220, November 2016. URL: https://doi.org/10.1109/LCOMM.2016.2601898.
  2. Paulo Sérgio Almeida, Carlos Baquero, Nuno Preguiça, and David Hutchison. Scalable Bloom Filters. Inf. Process. Lett., 101(6):255-261, March 2007. URL: https://doi.org/10.1016/j.ipl.2006.10.007.
  3. Michael A. Bender, Martin Farach-Colton, Rob Johnson, Russell Kraner, Bradley C. Kuszmaul, Dzejla Medjedovic, Pablo Montes, Pradeep Shetty, Richard P. Spillane, and Erez Zadok. Don't Thrash: How to cache your hash on flash. Proc. VLDB Endow., 5(11):1627-1637, July 2012. URL: https://doi.org/10.14778/2350229.2350275.
  4. Alex D. Breslow and Nuwan S. Jayasena. Morton filters: fast, compressed sparse cuckoo filters. The VLDB Journal, August 2019. URL: https://doi.org/10.1007/s00778-019-00561-0.
  5. Pedro Celis, Per-Ake Larson, and J Ian Munro. Robin Hood Hashing. In 26th Annual Symposium on Foundations of Computer Science (FOCS), pages 281-288. IEEE, 1985. Google Scholar
  6. J. G. Cleary. Compact Hash Tables Using Bidirectional Linear Probing. IEEE Transactions on Computers, C-33(9):828-834, 1984. Google Scholar
  7. Yan Collet. xxHash. https://github.com/Cyan4973/xxHash. Accessed March 21, 2019. URL: https://github.com/Cyan4973/xxHash.
  8. Bin Fan, Dave G. Andersen, Michael Kaminsky, and Michael D. Mitzenmacher. Cuckoo Filter: Practically better than Bloom. In 10th ACM International on Conference on Emerging Networking Experiments and Technologies, pages 75-88, 2014. URL: https://doi.org/10.1145/2674005.2674994.
  9. Afton Geil, Martin Farach-Colton, and John D. Owens. Quotient Filters: Approximate membership queries on the GPU. In 2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS), pages 451-462, May 2018. URL: https://doi.org/10.1109/IPDPS.2018.00055.
  10. Donald Ervin Knuth. The Art of Computer Programming: sorting and searching, volume 3. Pearson Education, 1997. Google Scholar
  11. Tobias Maier, Peter Sanders, and Roman Dementiev. Concurrent Hash Tables: Fast and general(?)! ACM Trans. Parallel Comput., 5(4):16:1-16:32, February 2019. URL: https://doi.org/10.1145/3309206.
  12. Tobias Maier, Peter Sanders, and Robert Williger. Concurrent Expandable AMQs on the basis of quotient filters. CoRR, 2019. URL: http://arxiv.org/abs/1911.08374.
  13. Michael Mitzenmacher, Salvatore Pontarelli, and Pedro Reviriego. Adaptive Cuckoo Filters. In 2018 Proceedings of the Twentieth Workshop on Algorithm Engineering and Experiments (ALENEX), pages 36-47. SIAM, 2018. URL: https://doi.org/10.1137/1.9781611975055.4.
  14. Prashant Pandey. Counting Quotient Filter. https://github.com/splatlab/cqf. Accessed August 07, 2019. URL: https://github.com/splatlab/cqf.
  15. Prashant Pandey, Fatemeh Almodaresi, Michael A. Bender, Michael Ferdman, Rob Johnson, and Rob Patro. Mantis: A fast, small, and exact large-scale sequence-search index. Cell Systems, 7(2):201-207.e4, 2018. URL: https://doi.org/10.1016/j.cels.2018.05.021.
  16. Prashant Pandey, Michael A. Bender, Rob Johnson, and Rob Patro. A General-Purpose Counting Filter: Making every bit count. In ACM Conference on Management of Data (SIGMOD), pages 775-787, 2017. URL: https://doi.org/10.1145/3035918.3035963.
Questions / Remarks / Feedback
X

Feedback for Dagstuhl Publishing


Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail