Geometric Models for Musical Audio Data

Authors Paul Bendich, Ellen Gasparovic, John Harer, Christopher Tralie

Thumbnail PDF


  • Filesize: 0.9 MB
  • 5 pages

Document Identifiers

Author Details

Paul Bendich
Ellen Gasparovic
John Harer
Christopher Tralie

Cite AsGet BibTex

Paul Bendich, Ellen Gasparovic, John Harer, and Christopher Tralie. Geometric Models for Musical Audio Data. In 32nd International Symposium on Computational Geometry (SoCG 2016). Leibniz International Proceedings in Informatics (LIPIcs), Volume 51, pp. 65:1-65:5, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2016)


We study the geometry of sliding window embeddings of audio features that summarize perceptual information about audio, including its pitch and timbre. These embeddings can be viewed as point clouds in high dimensions, and we add structure to the point clouds using a cover tree with adaptive thresholds based on multi-scale local principal component analysis to automatically assign points to clusters. We connect neighboring clusters in a scaffolding graph, and we use knowledge of stratified space structure to refine our estimates of dimension in each cluster, demonstrating in our music applications that choruses and verses have higher dimensional structure, while transitions between them are lower dimensional. We showcase our technique with an interactive web-based application powered by Javascript and WebGL which plays music synchronized with a principal component analysis embedding of the point cloud down to 3D. We also render the clusters and the scaffolding on top of this projection to visualize the transitions between different sections of the music.
  • Geometric Models
  • Audio Analysis
  • High Dimensional Data Analysis
  • Stratified Space Models


  • Access Statistics
  • Total Accesses (updated on a weekly basis)
    PDF Downloads


  1. Mark A. Bartsch and Gregory H. Wakefield. To catch a chorus: Using chroma-based representations for audio thumbnailing. In 2001 IEEE Workshop on the Applications of Signal Processing to Audio and Acoustics, pages 15-18. IEEE, 2001. Google Scholar
  2. Paul Bendich, Ellen Gasparovic, John Harer, and Christopher Tralie. Scaffoldings and spines: Organizing high-dimensional data using cover trees, local principal component analysis, and persistent homology, 2016. Google Scholar
  3. Alina Beygelzimer, Sham Kakade, and John Langford. Cover trees for nearest neighbor. In Proceedings of the 23rd International Conference on Machine Learning, pages 97-104. ACM, 2006. Google Scholar
  4. Bruce P. Bogert, Michael J.R. Healy, and John W. Tukey. The quefrency analysis of time series for echoes: Cepstrum, pseudo-autocovariance, cross-cepstrum and saphe cracking. In Proceedings of the symposium on time series analysis, volume 15, pages 209-243. chapter, 1963. Google Scholar
  5. Brian McFee, Colin Raffel, Dawen Liang, Daniel PW Ellis, Matt McVicar, Eric Battenberg, and Oriol Nieto. librosa: Audio and music signal analysis in Python. In Proceedings of the 14th Python in Science Conference, 2015. Google Scholar
  6. George Tzanetakis and Perry Cook. Musical genre classification of audio signals. IEEE transactions on Speech and Audio Processing, 10(5):293-302, 2002. Google Scholar
Questions / Remarks / Feedback

Feedback for Dagstuhl Publishing

Thanks for your feedback!

Feedback submitted

Could not send message

Please try again later or send an E-mail