Document Open Access Logo

Deconstructed "Learned" Indexes and Their Smoothed Analysis

Authors Stefan Hermann , Mattia Odorisio , Peter Sanders , Stefan Walzer

PDF

File

Thumbnail PDF
PDF
LIPIcs.ESA.2026.15.pdf
  • Filesize: 1.67 MB
  • 24 pages

Document Identifiers

Subject Classification

ACM Subject Classification
  • Information systems → Data structures
  • Information systems → Database management system engines
  • Computing methodologies → Machine learning
Keywords
  • Learned data structure
  • sorted sequence
  • index data structure
  • smoothed analysis

Metrics

Abstract

Data structures that maintain a sorted sequence are crucial for many applications. There is a zoo of variants with recent particular interest in "learned" indexes that accelerate operations by learning the distribution of the data. This paper helps to bring some order to this complex situation. We identify important building blocks and model the input using smoothed analysis where an adversary can control the dynamically changing input except for a small amount of noise. Within a resulting design space of data structures, we prove that already a simple 2-level data structure with minimal learning can achieve constant operation times in many situations: PARROT partitions the input into equal size parts, within which keys are approximately uniformly distributed. In many of our experiments, PARROT performs very well compared to state-of-the-art learned indexes, being 2× faster than the well known ALEX and LIPP indexes on large datasets, and 10× faster than a well engineered standard B-Tree.

Cite As Get BibTex

Stefan Hermann, Mattia Odorisio, Peter Sanders, and Stefan Walzer. Deconstructed "Learned" Indexes and Their Smoothed Analysis. In 34th Annual European Symposium on Algorithms (ESA 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 388, pp. 15:1-15:24, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026) https://doi.org/10.4230/LIPIcs.ESA.2026.15

Author Details

Stefan Hermann
  • Karlsruhe Institute of Technology, Germany
Mattia Odorisio
  • University of Pisa, Italy
Peter Sanders
  • Karlsruhe Institute of Technology, Germany
Stefan Walzer
  • Karlsruhe Institute of Technology, Germany

Supplementary Materials

References

  1. Abdullah Al-Mamun, Hao Wu, Qiyang He, Jianguo Wang, and Walid G. Aref. A survey of learned indexes for the multi-dimensional space, 2024. URL: https://doi.org/10.48550/arXiv.2403.06456.
  2. Abdullah Al-Mamun, Hao Wu, Qiyang He, Jianguo Wang, and Walid G. Aref. A survey of learned indexes for the multi-dimensional space. ACM Comput. Surv., 58(4), October 2025. URL: https://doi.org/10.1145/3768575.
  3. O. Amble and D. E. Knuth. Ordered hash tables. The Computer Journal, 17(2):135-142, January 1974. URL: https://doi.org/10.1093/comjnl/17.2.135.
  4. Arne Andersson and Christer Mattsson. Dynamic interpolation search in o (log log n) time. In International Colloquium on Automata, Languages, and Programming, pages 15-27. Springer, 1993. URL: https://doi.org/10.1007/3-540-56939-1_58.
  5. R. Bayer and E. M. McCreight. Organization and maintenance of large ordered indexes. Acta Informatica, 1(3):173-189, 1972. URL: https://doi.org/10.1007/BF00288683.
  6. R. Beier and B. Vöcking. Random knapsack in expected polynomial time. In 35th ACM Symposium on Theory of Computing, 2003. Google Scholar
  7. Djamal Belazzougui, Alexis C. Kaporis, and Paul G. Spirakis. Random input helps searching predecessors. In Luca Ferrari and Malvina Vamvakari, editors, Proceedings of the 11th International Conference on Random and Exhaustive Generation of Combinatorial Structures, GASCom 2018, Athens, Greece, June 18-20, 2018, volume 2113 of CEUR Workshop Proceedings, pages 106-115. CEUR-WS.org, 2018. URL: https://ceur-ws.org/Vol-2113/paper10.pdf.
  8. Michael A. Bender, Martin Farach-Colton, and Miguel A. Mosteiro. Insertion sort is o(n log n). Theor. Comp. Sys., 39(3):391-397, June 2006. URL: https://doi.org/10.1007/s00224-005-1237-z.
  9. Timo Bingmann. TLX: Collection of sophisticated C++ data structures, algorithms, and miscellaneous helpers, 2018. , retrieved Apr. 20, 2026. URL: https://panthema.net/tlx.
  10. Antonio Boffa, Paolo Ferragina, and Giorgio Vinciguerra. A learned approach to design compressed rank/select data structures. ACM Transactions on Algorithms, 2022. URL: https://doi.org/10.1145/3524060.
  11. Randy Brown. Calendar queues: a fast O (1) priority queue implementation for the simulation event set problem. Communications of the ACM, 31(10):1220-1227, 1988. URL: https://doi.org/10.1145/63039.63045.
  12. Pedro Celis, Per-Ake Larson, and J. Ian Munro. Robin hood hashing. In Proceedings of the 26th Annual Symposium on Foundations of Computer Science, SFCS '85, pages 281-288, USA, 1985. IEEE Computer Society. URL: https://doi.org/10.1109/SFCS.1985.48.
  13. Supawit Chockchowwat, Wenjie Liu, and Yongjoo Park. Airindex: Versatile index tuning through data and storage. Proc. ACM Manag. Data, 1(3):204:1-204:26, 2023. URL: https://doi.org/10.1145/3617308.
  14. Luis Alberto Croquevielle, Roman Sokolovskii, and Thomas Heinis. Lower bounds for the algorithmic complexity of learned indexes, 2026. URL: https://doi.org/10.48550/arXiv.2601.06629.
  15. Luis Alberto Croquevielle, Guang Yang, Liang Liang, Ali Hadian, and Thomas Heinis. Beyond Logarithmic Bounds: Querying in Constant Expected Time with Learned Indexes. In 28th International Conference on Database Theory (ICDT 2025), volume 328, pages 19:1-19:21, 2025. URL: https://doi.org/10.4230/LIPIcs.ICDT.2025.19.
  16. Andrew Crotty. Hist-tree: Those who ignore it are doomed to learn. In 11th Conference on Innovative Data Systems Research, CIDR 2021, Virtual Event, January 11-15, 2021, Online Proceedings. www.cidrdb.org, 2021. URL: https://vldb.org/cidrdb/2021/hist-tree-those-who-ignore-it-are-doomed-to-learn.html.
  17. Erik D Demaine, Thouis Jones, and Mihai Pătraşcu. Interpolation search for non-independent data. In 15th ACM-SIAM symposium on Discrete algorithms (SODA), pages 529-530, 2004. Google Scholar
  18. R. Dementiev, L. Kettner, J. Mehnert, and P. Sanders. Engineering a sorted list data structure for 32 bit keys. In 6th Workshop on Algorithm Engineering & Experiments, pages 142-151, New Orleans, 2004. Google Scholar
  19. Jialin Ding, Umar Farooq Minhas, Jia Yu, Chi Wang, Jaeyoung Do, Yinan Li, Hantian Zhang, Badrish Chandramouli, Johannes Gehrke, Donald Kossmann, David Lomet, and Tim Kraska. ALEX: An updatable adaptive learned index. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, SIGMOD '20, pages 969-984, New York, NY, USA, 2020. Association for Computing Machinery. URL: https://doi.org/10.1145/3318464.3389711.
  20. Patrick Dinklage, Johannes Fischer, and Alexander Herlez. Engineering Predecessor Data Structures for Dynamic Integer Sets. In 19th International Symposium on Experimental Algorithms (SEA 2021), volume 190, pages 7:1-7:19, 2021. URL: https://doi.org/10.4230/LIPIcs.SEA.2021.7.
  21. Paolo Ferragina and Filippo Lari. Compressibility Measures and Succinct Data Structures for Piecewise Linear Approximations. In 36th International Symposium on Algorithms and Computation (ISAAC 2025), volume 359 of Leibniz International Proceedings in Informatics (LIPIcs), pages 31:1-31:15, 2025. URL: https://doi.org/10.4230/LIPIcs.ISAAC.2025.31.
  22. Paolo Ferragina and Filippo Lari. FL-RMQ: A Learned Approach to Range Minimum Queries. In 36th Annual Symposium on Combinatorial Pattern Matching (CPM 2025), volume 331, pages 7:1-7:23, 2025. URL: https://doi.org/10.4230/LIPIcs.CPM.2025.7.
  23. Paolo Ferragina, Hans-Peter Lehmann, Peter Sanders, and Giorgio Vinciguerra. Learned monotone minimal perfect hashing. In 31st European Symposium on Algorithms (ESA), 2023. URL: https://doi.org/10.4230/LIPIcs.ESA.2023.46.
  24. Paolo Ferragina, Fabrizio Lillo, and Giorgio Vinciguerra. On the performance of learned data structures. Theoretical Computer Science, 871:107-120, 2021. URL: https://doi.org/10.1016/j.tcs.2021.04.015.
  25. Paolo Ferragina and Giorgio Vinciguerra. The PGM-index: a fully-dynamic compressed learned index with provable worst-case bounds. PVLDB, 13(8):1162-1175, 2020. URL: https://doi.org/10.14778/3389133.3389135.
  26. Philippe Flajolet, Patricio Poblete, and Alfredo Viola. On the analysis of linear probing hashing. Algorithmica, 22(4):490-515, 1998. URL: https://doi.org/10.1007/PL00009236.
  27. Greg N. Frederickson. Implicit data structures for the dictionary problem. Journal of the ACM (JACM), 30(1):80-94, 1983. URL: https://doi.org/10.1145/322358.322364.
  28. Emil Toftegaard Gæde, Ivor van der Hoog, Eva Rotenberg, and Tord Stordalen. A Dynamic Piecewise-Linear Geometric Index with Worst-Case Guarantees. In 33rd Annual European Symposium on Algorithms (ESA 2025), volume 351, pages 64:1-64:18, 2025. URL: https://doi.org/10.4230/LIPIcs.ESA.2025.64.
  29. Alex Galakatos, Michael Markovitch, Carsten Binnig, Rodrigo Fonseca, and Tim Kraska. Fiting-tree: A data-aware index structure. In Proceedings of the 2019 International Conference on Management of Data, SIGMOD '19, pages 1189-1206, New York, NY, USA, 2019. Association for Computing Machinery. URL: https://doi.org/10.1145/3299869.3319860.
  30. Alon Itai, Alan G. Konheim, and Michael Rodeh. A sparse table implementation of priority queues. In Shimon Even and Oded Kariv, editors, Automata, Languages and Programming, 8th Colloquium, Acre (Akko), Israel, July 13-17, 1981, Proceedings, volume 115 of Lecture Notes in Computer Science, pages 417-431. Springer, 1981. URL: https://doi.org/10.1007/3-540-10843-2_34.
  31. Svante Janson. Tail bounds for sums of geometric and exponential variables. Statistics & Probability Letters, 135:1-6, 2018. Google Scholar
  32. Alexis C. Kaporis, Christos Makris, Spyros Sioutas, Athanasios K. Tsakalidis, Kostas Tsichlas, and Christos D. Zaroliagis. Dynamic interpolation search revisited. Inf. Comput., 270, 2020. URL: https://doi.org/10.1016/J.IC.2019.104465.
  33. Andreas Kipf, Ryan Marcus, Alexander van Renen, Mihail Stoian, Alfons Kemper, Tim Kraska, and Thomas Neumann. Radixspline: a single-pass learned index. In Proceedings of the Third International Workshop on Exploiting Artificial Intelligence Techniques for Data Management, aiDM '20, New York, NY, USA, 2020. Association for Computing Machinery. URL: https://doi.org/10.1145/3401071.3401659.
  34. Tim Kraska, Alex Beutel, Ed H. Chi, Jeffrey Dean, and Neoklis Polyzotis. The case for learned index structures. In Proceedings of the 2018 International Conference on Management of Data, SIGMOD '18, pages 489-504, New York, NY, USA, 2018. Association for Computing Machinery. URL: https://doi.org/10.1145/3183713.3196909.
  35. Ani Kristo, Kapil Vaidya, and Tim Kraska. Defeating duplicates: A re-design of the learnedsort algorithm. CoRR, abs/2107.03290, 2021. URL: https://arxiv.org/abs/2107.03290.
  36. Pengfei Li, Yu Hua, Jingnan Jia, and Pengfei Zuo. Finedex: a fine-grained learned index scheme for scalable and concurrent memory systems. Proc. VLDB Endow., 15(2):321-334, October 2021. URL: https://doi.org/10.14778/3489496.3489512.
  37. Pengfei Li, Hua Lu, Rong Zhu, Bolin Ding, Long Yang, and Gang Pan. Dili: A distribution-driven learned index. Proc. VLDB Endow., 16(9):2212-2224, May 2023. URL: https://doi.org/10.14778/3598581.3598593.
  38. Liang Liang, Guang Yang, Ali Hadian, Luis Alberto Croquevielle, and Thomas Heinis. Swix: A memory-efficient sliding window learned index. Proc. ACM Manag. Data, 2(1), March 2024. URL: https://doi.org/10.1145/3639296.
  39. Qiyu Liu, Siyuan Han, Yanlin Qi, Jingshu Peng, Jin Li, Longlong Lin, and Lei Chen. Why are learned indexes so effective but sometimes ineffective? Proc. VLDB Endow., 18(9):2886-2898, September 2025. URL: https://doi.org/10.14778/3746405.3746415.
  40. Chaohong Ma, Xiaohui Yu, Yifan Li, Aishan Maoliniyazi, and Xiaofeng Meng. LINDAS: a learned approach to index algorithm selection. Knowl. Inf. Syst., 67(12):11381-11423, 2025. URL: https://doi.org/10.1007/S10115-025-02552-W.
  41. Tobias Maier, Peter Sanders, and Roman Dementiev. Concurrent hash tables: Fast and general?(!). ACM Transactions on Parallel Computing (TOPC), 5, 2019. URL: https://doi.org/10.1145/3309206.
  42. Ryan Marcus, Andreas Kipf, Alexander van Renen, Mihail Stoian, Sanchit Misra, Alfons Kemper, Thomas Neumann, and Tim Kraska. Benchmarking learned indexes. Proc. VLDB Endow., 14(1):1-13, September 2020. URL: https://doi.org/10.14778/3421424.3421425.
  43. Kurt Mehlhorn and Stefan Näher. Bounded ordered dictionaries in O(log log N) time and O(n) space. Information Processing Letters, 35(4):183-189, 1990. URL: https://doi.org/10.1016/0020-0190(90)90022-P.
  44. Kurt Mehlhorn and Athanasios Tsakalidis. Dynamic interpolation search. J. ACM, 40(3):621-634, July 1993. URL: https://doi.org/10.1145/174130.174139.
  45. Gonzalo Navarro and Javiel Rojas-Ledesma. Predecessor search. ACM Comput. Surv., 53(5), September 2020. URL: https://doi.org/10.1145/3409371.
  46. W. W. Peterson. Addressing for random-access storage. IBM Journal of Research and Development, 1(2):130-146, 1957. URL: https://doi.org/10.1147/rd.12.0130.
  47. Mihai Pătraşcu and Mikkel Thorup. Time-space trade-offs for predecessor search. In Proceedings of the Thirty-Eighth Annual ACM Symposium on Theory of Computing, STOC '06, pages 232-240, New York, NY, USA, 2006. Association for Computing Machinery. URL: https://doi.org/10.1145/1132516.1132551.
  48. Atsuki Sato and Yusuke Matsui. PCF learned sort: a learning augmented sort algorithm with 𝒪(n log log n) expected complexity. Transactions on Machine Learning Research, 2025. Featured Certification, J2C Certification. URL: https://openreview.net/forum?id=wVkb8WHbvR.
  49. J. C. Shepherdson and H. E. Sturgis. Computability of recursive functions. Journal of the ACM, 10(2):217-255, 1963. URL: https://doi.org/10.1145/321160.321170.
  50. Benjamin Spector, Andreas Kipf, Kapil Vaidya, Chi Wang, Umar Farooq Minhas, and Tim Kraska. Bounding the last mile: Efficient learned string indexing. CoRR, abs/2111.14905, 2021. URL: https://arxiv.org/abs/2111.14905.
  51. D. Spielman and S.-H. Teng. Smoothed analysis of algorithms: why the simplex algorithm usually takes polynomial time. In 33rd ACM Symposium on Theory of Computing, pages 296-305, 2001. URL: https://doi.org/10.1145/380752.380813.
  52. Mihail Stoian, Andreas Kipf, Ryan Marcus, and Tim Kraska. PLEX: towards practical learned indexing. CoRR, abs/2108.05117, 2021. URL: https://arxiv.org/abs/2108.05117.
  53. Zhaoyan Sun, Xuanhe Zhou, and Guoliang Li. Learned index: A comprehensive experimental evaluation. Proc. VLDB Endow., 16(8):1992-2004, April 2023. URL: https://doi.org/10.14778/3594512.3594528.
  54. Chuzhe Tang, Youyun Wang, Zhiyuan Dong, Gansen Hu, Zhaoguo Wang, Minjie Wang, and Haibo Chen. Xindex: a scalable learned index for multicore data storage. In Proceedings of the 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP '20, pages 308-320, New York, NY, USA, 2020. Association for Computing Machinery. URL: https://doi.org/10.1145/3332466.3374547.
  55. Kapil Vaidya, Subarna Chatterjee, Eric Knorr, Michael Mitzenmacher, Stratos Idreos, and Tim Kraska. Snarf: a learning-enhanced range filter. Proc. VLDB Endow., 15(8):1632-1644, April 2022. URL: https://doi.org/10.14778/3529337.3529347.
  56. Peter van Emde Boas. Preserving order in a forest in less than logarithmic time and linear space. Inf. Process. Lett., 6(3):80-82, 1977. URL: https://doi.org/10.1016/0020-0190(77)90031-X.
  57. Youyun Wang, Chuzhe Tang, Zhaoguo Wang, and Haibo Chen. Sindex: a scalable learned index for string keys. In Proceedings of the 11th ACM SIGOPS Asia-Pacific Workshop on Systems, APSys '20, pages 17-24, New York, NY, USA, 2020. Association for Computing Machinery. URL: https://doi.org/10.1145/3409963.3410496.
  58. Dan E. Willard. Searching unindexed and nonuniformly generated files in log log n time. SIAM Journal on Computing, 14(4):1013-1029, 1985. URL: https://doi.org/10.1137/0214071.
  59. Dan E. Willard. Examining computational geometry, van emde boas trees, and hashing from the perspective of the fusion tree. SIAM Journal on Computing, 29(3):1030-1049, 2000. URL: https://doi.org/10.1137/S0097539797322425.
  60. Jiacheng Wu, Yong Zhang, Shimin Chen, Jin Wang, Yu Chen, and Chunxiao Xing. Updatable learned index with precise positions. Proc. VLDB Endow., 14(8):1276-1288, April 2021. URL: https://doi.org/10.14778/3457390.3457393.
  61. Guang Yang, Liang Liang, Ali Hadian, and Thomas Heinis. FLIRT: A fast learned index for rolling time frames. In EDBT, pages 234-246, 2023. URL: https://doi.org/10.48786/edbt.2023.19.
  62. Yifan Yang and Shimin Chen. Lits: An optimized learned index for strings. Proc. VLDB Endow., 17(11):3415-3427, July 2024. URL: https://doi.org/10.14778/3681954.3682010.
  63. Andrew C. Yao and F. Frances Yao. The complexity of searching an ordered random table. In 17th Annual Symposium on Foundations of Computer Science (sfcs 1976), pages 173-177, 1976. URL: https://doi.org/10.1109/SFCS.1976.32.
  64. Sepanta Zeighami and Cyrus Shahabi. On distribution dependent sub-logarithmic query time of learned indexing. In International Conference on Machine Learning (ICML), pages 40669-40680. PMLR, 2023. URL: https://proceedings.mlr.press/v202/zeighami23a.html.
  65. Sepanta Zeighami and Cyrus Shahabi. Theoretical analysis of learned database operations under distribution shift through distribution learnability. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org, 2024. Google Scholar
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2026 Schloss Dagstuhl – LZI GmbH Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact