Document Open Access Logo

Dimension Reduction for Clustering: The Curious Case of Discrete Centers

Authors Shaofeng H.-C. Jiang , Robert Krauthgamer , Shay Sapir , Sandeep Silwal , Di Yue

PDF

File

Thumbnail PDF
PDF
LIPIcs.ITCS.2026.82.pdf
  • Filesize: 0.85 MB
  • 23 pages

Document Identifiers

Related Versions

Subject Classification

ACM Subject Classification
  • Theory of computation → Random projections and metric embeddings
Keywords
  • dimension reduction
  • clustering
  • k-median
  • k-means
  • doubling dimension

Metrics

Abstract

The Johnson-Lindenstrauss transform is a fundamental method for dimension reduction in Euclidean spaces, that can map any dataset of n points into dimension O(log n) with low distortion of their distances. This dimension bound is tight in general, but one can bypass it for specific problems. Indeed, tremendous progress has been made for clustering problems, especially in the continuous setting where centers can be picked from the ambient space ℝ^d. Most notably, for k-median and k-means, the dimension bound was improved to O(log k) [Makarychev, Makarychev and Razenshteyn, STOC 2019].
We explore dimension reduction for clustering in the discrete setting, where centers can only be picked from the dataset, and present two results that are both parameterized by the doubling dimension of the dataset, denoted as ddim. The first result shows that dimension O_{ε}(ddim + log k + log log n) suffices, and is moreover tight, to guarantee that the cost is preserved within factor 1±ε for every set of centers. Our second result eliminates the log log n term in the dimension through a relaxation of the guarantee (namely, preserving the cost only for all approximately-optimal sets of centers), which maintains its usefulness for downstream applications.
Overall, we achieve strong dimension reduction in the discrete setting, and find that it differs from the continuous setting not only in the dimension bound, which depends on the doubling dimension, but also in the guarantees beyond preserving the optimal value, such as which clusterings are preserved.

Cite As Get BibTex

Shaofeng H.-C. Jiang, Robert Krauthgamer, Shay Sapir, Sandeep Silwal, and Di Yue. Dimension Reduction for Clustering: The Curious Case of Discrete Centers. In 17th Innovations in Theoretical Computer Science Conference (ITCS 2026). Leibniz International Proceedings in Informatics (LIPIcs), Volume 362, pp. 82:1-82:23, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2026) https://doi.org/10.4230/LIPIcs.ITCS.2026.82

Author Details

Shaofeng H.-C. Jiang
  • Peking University, Beijing, China
Robert Krauthgamer
  • The Harry Weinrebe Professorial Chair of Computer Science, Weizmann Institute of Science, Rehovot, Israel
Shay Sapir
  • Weizmann Institute of Science, Rehovot, Israel
Sandeep Silwal
  • University of Wisconsin-Madison, WI, USA
Di Yue
  • Peking University, Beijing, China

Funding

  • Jiang, Shaofeng H.-C.: Work partially supported by a national key R&D program of China No. 2021YFA1000900.
  • Krauthgamer, Robert: Work partially supported by the Israel Science Foundation grant #1336/23.

Acknowledgements

The authors would like to thank Lingxiao Huang and Yury Makarychev for insightful discussions during the initial stage of this work.

References

  1. Yair Bartal, Ben Recht, and Leonard J Schulman. Dimensionality reduction: beyond the johnson-lindenstrauss bound. In Proceedings of the twenty-second annual ACM-SIAM symposium on Discrete Algorithms, pages 868-887. SIAM, 2011. URL: https://doi.org/10.1137/1.9781611973082.68.
  2. Luca Becchetti, Marc Bury, Vincent Cohen-Addad, Fabrizio Grandoni, and Chris Schwiegelshohn. Oblivious dimension reduction for k-means: beyond subspaces and the Johnson-Lindenstrauss Lemma. In Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing, STOC, pages 1039-1050, 2019. URL: https://doi.org/10.1145/3313276.3316318.
  3. Christos Boutsidis, Anastasios Zouzias, and Petros Drineas. Random projections for k-means clustering. In 24th Annual Conference on Neural Information Processing Systems, NeurIPS, pages 298-306. Curran Associates, Inc., 2010. URL: https://proceedings.neurips.cc/paper/2010/hash/73278a4a86960eeb576a8fd4c9ec6997-Abstract.html.
  4. Moses Charikar and Erik Waingarten. The Johnson-Lindenstrauss Lemma for clustering and subspace approximation: From coresets to dimension reduction. In SODA, pages 3172-3209. SIAM, 2025. URL: https://doi.org/10.1137/1.9781611978322.102.
  5. Xiaoyu Chen, Shaofeng H.-C. Jiang, and Robert Krauthgamer. Streaming Euclidean Max-Cut: Dimension vs data reduction. In Proceedings of the 55th Annual ACM Symposium on Theory of Computing, STOC, pages 170-182, 2023. URL: https://doi.org/10.1145/3564246.3585170.
  6. Michael B. Cohen, Sam Elder, Cameron Musco, Christopher Musco, and Madalina Persu. Dimensionality reduction for k-means clustering and low rank approximation. In Proceedings of the Forty-Seventh Annual ACM on Symposium on Theory of Computing, STOC, pages 163-172, 2015. URL: https://doi.org/10.1145/2746539.2746569.
  7. Michael B. Cohen, Jelani Nelson, and David P. Woodruff. Optimal approximate matrix product in terms of stable rank. In 43rd International Colloquium on Automata, Languages, and Programming (ICALP 2016), volume 55 of LIPIcs, pages 11:1-11:14. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2016. URL: https://doi.org/10.4230/LIPIcs.ICALP.2016.11.
  8. Vincent Cohen-Addad, Hossein Esfandiari, Vahab Mirrokni, and Shyam Narayanan. Improved approximations for Euclidean k-means and k-median, via nested quasi-independent sets. In Proceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing, pages 1621-1628, 2022. URL: https://doi.org/10.1145/3519935.3520011.
  9. Vincent Cohen-Addad, Andreas Emil Feldmann, and David Saulpic. Near-linear time approximation schemes for clustering in doubling metrics. Journal of the ACM, 68(6):1-34, 2021. URL: https://doi.org/10.1145/3477541.
  10. Vincent Cohen-Addad, Fabrizio Grandoni, Euiwoong Lee, Chris Schwiegelshohn, and Ola Svensson. A (2+ ε)-approximation algorithm for metric k-median. In Proceedings of the 57th Annual ACM Symposium on Theory of Computing, pages 615-624, 2025. URL: https://doi.org/10.1145/3717823.3718299.
  11. Vincent Cohen-Addad and C. S. Karthik. Inapproximability of clustering in Lp metrics. In 2019 IEEE 60th Annual Symposium on Foundations of Computer Science (FOCS), pages 519-539. IEEE, 2019. URL: https://doi.org/10.1109/FOCS.2019.00040.
  12. Vincent Cohen-Addad, C. S. Karthik, and Euiwoong Lee. Johnson coverage hypothesis: Inapproximability of k-means and k-median in 𝓁_p-metrics. In Proceedings of the 2022 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pages 1493-1530. SIAM, 2022. URL: https://doi.org/10.1137/1.9781611977073.63.
  13. Sanjoy Dasgupta and Anupam Gupta. An elementary proof of a theorem of Johnson and Lindenstrauss. Random Struct. Algorithms, 22(1):60-65, 2003. URL: https://doi.org/10.1002/rsa.10073.
  14. Jie Gao, Rajesh Jayaram, Benedikt Kolbe, Shay Sapir, Chris Schwiegelshohn, Sandeep Silwal, and Erik Waingarten. Randomized dimensionality reduction for Euclidean maximization and diversity measures. In Forty-second International Conference on Machine Learning, 2025. URL: https://openreview.net/forum?id=Rcivp36KzO.
  15. Lee-Ad Gottlieb and Robert Krauthgamer. A nonlinear approach to dimension reduction. Discrete & Computational Geometry, 54(2):291-315, 2015. URL: https://doi.org/10.1007/s00454-015-9707-9.
  16. Anupam Gupta, Robert Krauthgamer, and James R. Lee. Bounded geometries, fractals, and low-distortion embeddings. In 44th Symposium on Foundations of Computer Science, FOCS, pages 534-543. IEEE Computer Society, 2003. URL: https://doi.org/10.1109/SFCS.2003.1238226.
  17. Sariel Har-Peled and Soham Mazumdar. On coresets for k-means and k-median clustering. In STOC, pages 291-300. ACM, 2004. URL: https://doi.org/10.1145/1007352.1007400.
  18. Lingxiao Huang, Shaofeng H.-C. Jiang, Robert Krauthgamer, and Di Yue. Near-optimal dimension reduction for facility location. In Proceedings of the 57th Annual ACM Symposium on Theory of Computing, STOC, pages 665-676, 2025. URL: https://doi.org/10.1145/3717823.3718214.
  19. Piotr Indyk and Rajeev Motwani. Approximate nearest neighbors: Towards removing the curse of dimensionality. In Proceedings of the Thirtieth Annual ACM Symposium on the Theory of Computing, STOC, pages 604-613, 1998. URL: https://doi.org/10.1145/276698.276876.
  20. Piotr Indyk and Assaf Naor. Nearest-neighbor-preserving embeddings. ACM Trans. Algorithms, 3(3):31, 2007. URL: https://doi.org/10.1145/1273340.1273347.
  21. Zachary Izzo, Sandeep Silwal, and Samson Zhou. Dimensionality reduction for Wasserstein barycenter. Advances in neural information processing systems, 34:15582-15594, 2021. URL: https://proceedings.neurips.cc/paper/2021/hash/8346db44a721fa863ca38180638bad3d-Abstract.html.
  22. Shaofeng H.-C. Jiang, Robert Krauthgamer, and Shay Sapir. Moderate dimension reduction for k-center clustering. In 40th International Symposium on Computational Geometry (SoCG 2024), volume 293 of Leibniz International Proceedings in Informatics (LIPIcs), pages 64:1-64:16. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2024. URL: https://doi.org/10.4230/LIPIcs.SoCG.2024.64.
  23. Shaofeng H. C. Jiang, Robert Krauthgamer, Shay Sapir, Sandeep Silwal, and Di Yue. Dimension reduction for clustering: The curious case of discrete centers, 2025. Full version. URL: https://doi.org/10.48550/arXiv.2509.07444.
  24. William B. Johnson and Joram Lindenstrauss. Extensions of Lipschitz mappings into Hilbert space. Contemporary mathematics, 26:189-206, 1984. URL: https://doi.org/10.1090/conm/026/737400.
  25. Leonard Kaufman and Peter J. Rousseeuw. Finding groups in data: an introduction to cluster analysis. John Wiley & Sons, 2009. URL: https://doi.org/10.1002/9780470316801.
  26. M. Kirszbraun. Über die zusammenziehende und lipschitzsche transformationen. Fundamenta Mathematicae, 22(1):77-108, 1934. URL: https://doi.org/10.4064/fm-22-1-77-108.
  27. Christiane Lammersen. Approximation Techniques for Facility Location and Their Applications in Metric Embeddings. PhD thesis, Technische Universität Dortmund, 2010. URL: https://doi.org/10.17877/DE290R-8506.
  28. Christiane Lammersen, Anastasios Sidiropoulos, and Christian Sohler. Streaming embeddings with slack. In 11th International Symposium on Algorithms and Data Structures, WADS, volume 5664 of Lecture Notes in Computer Science, pages 483-494. Springer, 2009. URL: https://doi.org/10.1007/978-3-642-03367-4_42.
  29. Kasper Green Larsen and Jelani Nelson. Optimality of the Johnson-Lindenstrauss Lemma. In 58th IEEE Annual Symposium on Foundations of Computer Science, FOCS, pages 633-638, 2017. URL: https://doi.org/10.1109/FOCS.2017.64.
  30. Jure Leskovec, Anand Rajaraman, and Jeffrey David Ullman. Mining of massive data sets. Cambridge University Press, 3rd edition, 2020. URL: https://doi.org/10.1017/9781108684163.
  31. Michael W. Mahoney. Randomized algorithms for matrices and data. Foundations and Trends in Machine Learning, 3(2):123-224, 2011. URL: https://doi.org/10.1561/2200000035.
  32. Konstantin Makarychev, Yury Makarychev, and Ilya P. Razenshteyn. Performance of Johnson-Lindenstrauss transform for k-means and k-medians clustering. In Proceedings of the 51st Annual Symposium on Theory of Computing, STOC, pages 1027-1038, 2019. URL: https://doi.org/10.1145/3313276.3316350.
  33. Shyam Narayanan, Sandeep Silwal, Piotr Indyk, and Or Zamir. Randomized dimensionality reduction for facility location and single-linkage clustering. In Proceedings of the 38th International Conference on Machine Learning, ICML, volume 139 of Proceedings of Machine Learning Research, pages 7948-7957. PMLR, 2021. URL: http://proceedings.mlr.press/v139/narayanan21b.html.
  34. Christos H. Papadimitriou. Worst-case and probabilistic analysis of a geometric location problem. SIAM Journal on Computing, 10(3):542-557, 1981. URL: https://doi.org/10.1137/0210040.
  35. Hae-Sang Park and Chi-Hyuck Jun. A simple and fast algorithm for K-medoids clustering. Expert Syst. Appl., 36(2):3336-3341, 2009. URL: https://doi.org/10.1016/J.ESWA.2008.01.039.
  36. Mo Tiwari, Martin J Zhang, James Mayclin, Sebastian Thrun, Chris Piech, and Ilan Shomorony. Banditpam: Almost linear time k-medoids clustering via multi-armed bandits. Advances in Neural Information Processing Systems, 33:10211-10222, 2020. Google Scholar
  37. David P. Woodruff. Sketching as a tool for numerical linear algebra. Foundations and Trends in Theoretical Computer Science, 10(1–-2):1-157, 2014. URL: https://doi.org/10.1561/0400000060.
  38. Junyuan Xie, Ross Girshick, and Ali Farhadi. Unsupervised deep embedding for clustering analysis. In Proceedings of The 33rd International Conference on Machine Learning, pages 478-487. PMLR, 2016. URL: https://proceedings.mlr.press/v48/xieb16.html.
Any Issues?
X

Feedback on the Current Page

CAPTCHA

Thanks for your feedback!

Feedback submitted to Dagstuhl Publishing

Could not send message

Please try again later or send an E-mail
© 2023-2025 Schloss Dagstuhl – LZI GmbH About DROPS Imprint Privacy Contact