Abstract 1 Introduction 2 Rhythm and DJing 3 Temporal Abstraction in DJ Tasks 4 Adoption and Ethical Challenges 5 Conclusion and Future Directions References

Temporal Considerations in DJ Mix Information Retrieval and Generation

Alexander Williams111Alexander Williams and Gregor Meehan contributed equally to this work. ORCID Centre for Digital Music, Queen Mary University of London, United Kingdom Gregor Meehan111Alexander Williams and Gregor Meehan contributed equally to this work. ORCID Centre for Digital Music, Queen Mary University of London, United Kingdom Stefan Lattner ORCID Sony CSL, Paris, France Johan Pauwels ORCID Centre for Digital Music, Queen Mary University of London, United Kingdom Mathieu Barthet ORCID Aix-Marseille Univ CNRS PRISM, France
Centre for Digital Music, Queen Mary University of London, United Kingdom
Abstract

Music is the art of arranging sounds in time so as to produce a continuous, unified, and evocative composition. Electronic dance music (EDM) is a collection of musical sub-genres produced using computers and electronic instruments and often presented through the medium of DJing, where tracks are curated and mixed sequentially into a continuous stream of music to offer unique listening and dancing experiences over time periods ranging from several minutes to several hours. A DJ’s actions and decisions occur at several levels of temporal granularity, from real-time audio manipulation (e.g. of tempo) for smooth inter-track transitions to long-term planning of track selection and sequencing for mix content and flow. While human DJs can instinctively operate across these different temporal resolutions, replicating this capability in an end-to-end automated DJing system presents significant challenges. In this paper, we analyse existing works in DJ mix information retrieval and generation from this temporal perspective. We first explain the close link between DJing and the temporal notion of musical rhythm, then describe a framework for categorising DJing actions by temporal granularity. Using this framework, we summarise and contrast potential approaches for automating and augmenting sequential DJ decision making, and discuss the unique characteristics of DJ mix track selection as a sequential recommendation task. In doing so, we hope to facilitate the implementation of more robust and complete automated DJing systems in future research.

Keywords and phrases:
Music Information Retrieval, Computational Creativity, Recommender Systems, Electronic Dance Music, DJ
Category:
Short Paper
Copyright and License:
[Uncaptioned image] © Alexander Williams, Gregor Meehan, Stefan Lattner, Johan Pauwels, and Mathieu Barthet; licensed under Creative Commons License CC-BY 4.0
2012 ACM Subject Classification:
Applied computing Sound and music computing
; Computing methodologies Control methods ; Computing methodologies Planning under uncertainty
Funding:
Alexander Williams and Gregor Meehan are research students at the UKRI Centre for Doctoral Training in Artificial Intelligence and Music, supported by UK Research and Innovation [grant number EP/S022694/1] and Queen Mary University of London. Alexander Williams is also supported by Sony CSL.
Editors:
Thierry Vidal and Przemysław Andrzej Wałęga

1 Introduction

Electronic Dance Music (EDM) has seen a huge growth in popularity in recent years, from a relatively underground movement to a mainstream industry worth billions [30]. EDM is often consumed through the medium of DJ performances, leading to a corresponding increase in interest in DJing among EDM fans. Computer programs that could perform or assist with the creative task of DJing have the potential to democratise access to high quality continuous music mixes [42], and therefore have seen a corresponding increase in research attention [45]. At the highest abstraction, we can specify two key tasks for an end-to-end computational DJ system: long-term track selection and sequencing, and real-time audio manipulation for inter-track DJ mixing. However, existing computational DJ systems are typically not end-to-end: to simplify their implementation, they are instead deconstructed into a hierarchy of components, some of which involve manual curation or simple rule-based heuristics. In this paper, we aim to highlight the challenges of implementing an intelligent and fully end-to-end DJing system by viewing its actions from a temporal perspective. We first explain the link between DJing and the temporal notion of musical rhythm, then describe how a DJ’s actions can be divided across different levels of temporal granularity. Through this framework, we analyse existing works in automated DJ mixing and how its unique challenges prevent straightforward application of related techniques in sequential music recommendation. Finally, we discuss challenges relating to the adoption of computational DJ systems and outline future research directions.

2 Rhythm and DJing

Music is typically organised into beats and repetitive rhythmic patterns that collectively form the foundation of musical rhythm [32]. Here rhythm refers to the medium-scale temporal organisation of sound, characterised by the arrangement of events, beats, and accents over time. It is composed of several key components: tempo, i.e. the speed of the pulse; timing, i.e. the placement of events relative to pulses; metre, i.e. the structural relationship among pulses; and grouping, i.e. the phrasing of musical ideas independent of metre.

Rhythmic properties are particularly important in EDM, as many EDM sub-genres are defined by their timbral palette and associated rhythm patterns. Further, structural changes and progression in EDM tracks / DJ mixes are usually indicated by an evolution of timbre and rhythm rather than melody and harmony [34, 22, 46, 24, 44]. EDM tracks, and subsequently DJ mixes, are typically built around repeating loops of melodies, vocals, drums, and sound effects (FX) that change and are layered over time – for example, through an element being added or removed from the mix or being affected by continuous temporal processes such as FX and synthesizer parameter automation [20, 37] – to produce rhythmic and timbral variation and progression in the composition. Modelling these rhythmic properties is therefore an important task in automated DJing systems, as we discuss further in Section 3 below.

3 Temporal Abstraction in DJ Tasks

We next examine in more detail the necessary capabilities of an end-to-end automated DJing system. The two key tasks in DJing are transitioning from track to track (i.e. DJ mixing) and selecting and ordering the tracks to be played. In the remainder of this section, we discuss existing approaches to each of these tasks, dividing DJing actions into three temporal levels as shown in Fig. 1. From most to least granular, these are:

  1. 1.

    Transition-level: the immediate actions needed to transition from the current song to a given next song;

  2. 2.

    Track-level: the choice of what the next song should be; and

  3. 3.

    Mix-level: the overall flow and content of the entire mix.

Refer to caption
Figure 1: The temporal abstraction hierarchy for key sub-tasks in DJ mix construction.

3.1 Automatic DJ Mixing

DJ mixing can be formulated as a real-time sequential decision-making problem, where the objective is to apply a sequence of audio signal transformations to two (or more) concurrently playing tracks to optimise the perceptual quality of the resulting audio mixture for the listener. These decisions take place entirely at the transition-level (1), using features relating to the rhythm and other characteristics of the current and incoming tracks.

DJ transitions are considered an art-form and their perceived quality is highly subjective and dependent on the audio tracks being mixed and the cultural context in which the transition takes place. Nevertheless, common techniques exist and many works have proposed automatic DJ mixing systems, attempting to implement probabilistic rules informed by signal processing to produce systems exhibiting what each author believes constitutes valid DJ mixing. For example, (beat-matched) crossfading over fixed-length temporal horizons is commonly applied [7, 33, 40, 42, 10, 23, 3, 27, 26, 2, 38]. Other approaches include 3-band equaliser mixing [21, 38], avoiding vocal clashes [42, 38], double drop / rolling / relaxed transition [42], looping out [33], looping to produce a drum roll-like effect [3], adding samples [27] or an MC-like voiceover [33], slam / cut / switching [40, 2, 3], echo out [40, 2], and power down [40]. While such rules can produce reasonable transitions, the predictability arising from using the same limited transition techniques mean that computer-generated transitions are more likely to be identified and negatively viewed by listeners in repeated exposures [18].

A notable exception to rule-based systems is a deep neural network to generate audio-dependent control trajectories for DJ transitions [7]. Although described as having developed a unique style, it also employed unfamiliar mixing techniques and subsequently its performance was only considered on par with rule-based methods. This highlights the challenge of predicting effective temporal control sequences, which must balance a degree of unpredictability with some adherence to established mixing norms and contextual appropriateness to deliver robust mixes that are satisfactory at a psychoacoustic level [41]. So far, no automatic DJ transition generation system has matched the performance of human DJs.

3.2 DJ Track Selection and Sequencing

DJ track selection is influenced by a DJ’s own curatorial interests and the context of the mix (e.g. a pop music night at a club versus an EDM festival). Sequencing depends on both transition compatibility and the DJ’s longer-term goals: for example, after playing several songs with high tempo, a DJ may choose to play lower energy tracks to give the audience a break. To successfully select and sequence tracks, generative DJ mix systems must therefore act across all three levels of temporal granularity. However, operating intelligently across all three temporal levels simultaneously is a fundamental difficulty in automated DJ mixing, and, to the best of our knowledge, no existing works attempt to do so. Most existing systems operate primarily at levels (1) and (2) in their sequencing algorithms: to ensure that the automated transitions discussed in Section 3.1 can be applied effectively, they select an appropriate next track by “mixability”, i.e. by compatibility in tempo, rhythm, or harmony. This compatibility can be either fine-grained at the transition points [19, 42, 38] (at transition-level), based on coarser similarity of overall acoustic features [40] (at track-level), or a combination of both [42]. Other works [11, 2, 33] place more emphasis on flow at the mix-level but forgo mix-level content curation and/or transition-level compatibility, while DJ-MC [25] focuses only on the higher two levels.

The above temporal framework also helps explain why existing sequential music recommendation algorithms cannot be straightforwardly applied in the DJ mixing context, even though DJ track selection is closely related to tasks such as playlist generation [13, 35, 1] and next song recommendation [16, 43, 36]. In particular, transition-level factors are unique to DJ mixing, and are not modelled by generic sequential music recommenders. However, these transition-level constraints exert a considerable influence on the other levels and therefore on the overall system: track-level choices require consideration of transition-level compatibility, which in turn affects mix-level track selection. There are also practical difficulties arising from the need to model all three levels of granularity, from the coarse sequence-level view of the selected tracks down to musical content at the beat or frame level. Furthermore, although some existing sequential music recommender systems or DJing systems do implement curatorial filtering (e.g. to specific genres or “vibes”, as in [33]), they typically do not operate with long-term goals in mind in the same way a DJ would. It is not clear whether such behaviour could be explicitly encouraged during model training, and any such effort would likely require a large dataset of DJ mixes with associated audio and track labels. However, no such corpus is currently publicly available.

4 Adoption and Ethical Challenges

Many DJs are concerned about how automation may affect their livelihoods [39], as computational DJing is already reshaping both live performance [5] and radio or home listening contexts [6, 2, 4]. For instance, iHeartMedia – a global network of local radio stations – laid off up to 850 DJs and producers in favour of software to schedule music, mix songs, and mimic the voice of radio hosts [9, 17, 14]. Yet, DJ practices have continually evolved alongside technological innovation [29]. Automatic beat tracking and alignment, key / tempo detection and transposition, and various visualisation techniques [46, 15, 8] to aid musical understanding and inform DJ decision making at multiple temporal levels are now ubiquitous in digital DJ workflows. Rather than replacing DJs, computational tools, like those discussed in this paper, can complement and expand contemporary practice. While quality-cost considerations may drive some autonomous applications [28], socio-cultural factors – particularly the importance of the dynamic between DJs and live audiences [31] – ensure continued demand for human DJs and intelligent mixing and recommendation systems can be used to support their creative expression. The key challenge comes in developing application use-cases that respect DJs’ artistic autonomy while leveraging the potential of these technologies to reduce various task complexities, enable enhanced performance, and augment creative decision making [12].

5 Conclusion and Future Directions

This paper outlines how a multi-resolution temporal framework is critical for both feature extraction and the development of hierarchical, modular systems capable of intra/inter-track mixing, track-level selection, and mix-level sequencing in an EDM DJing context. We also distinguish between DJ track selection and general sequential music recommendation based on this framework, and discuss potential consequences of this technology being democratised. In previous work, we identified that audio features that capture and emphasise timbral and rhythmic features are valuable for the analysis of DJ mixes at arbitrarily fine temporal intervals [46], particularly in computing DJ transition properties such as length and relative smoothness. In the future, we hope to apply such representations at various temporal resolutions for DJ mix information retrieval and to integrate approaches at multiple temporal levels to assist in various artist-centric, co-creative scenarios such as informative visualisation, multi-track mixing, and real-time assistance in DJ mix construction.

References

  • [1] Walid Bendada, Guillaume Salha-Galvan, Thomas Bouabça, and Tristan Cazenave. A scalable framework for automatic playlist continuation on music streaming services. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 464–474, 2023. doi:10.1145/3539618.3591628.
  • [2] Rachel M Bittner, Minwei Gu, Gandalf Hernandez, Eric J Humphrey, Tristan Jehan, P Hunter McCurry, and Nicola Montecchio. Automatic Playlist Sequencing and Transitions. In Proceedings of the 18th ISMIR Conference, Suzhou, China, 2017.
  • [3] Jaume Parera Bonmati. DJ Codo Nudo: a novel method for seamless transition between songs for electronic music. Master’s thesis, Universitat Pompeu Fabra, Barcelona, 2016. URL: http://mtg.upf.edu/system/files/publications/Jaume-Parera-Master-thesis-2016.pdf.
  • [4] Georgina Born, Jeremy Morris, Fernando Diaz, and Ashton Anderson. Artificial Intelligence, Music Recommendation, and the Curation of Culture: A White Paper, June 2021.
  • [5] Becky Buckle. East London’s first “AI-rave” receives mixed reviews. Mixmag, March 2023. URL: https://mixmag.net/read/east-london-nightclub-hosts-algorhythm-rave-with-ai-
    generated-music-tech
    .
  • [6] Nyshka Chandran. RA Pro Newsletter: AI’s role in independent radio, May 2025. URL: https://ra.co/news/82747.
  • [7] Bo-Yu Chen, Wei-Han Hsu, Wei-Hsiang Liao, Marco A. Martínez Ramírez, Yuki Mitsufuji, and Yi-Hsuan Yang. Automatic DJ Transitions with Differentiable Audio Effects and Generative Adversarial Networks. In ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 466–470, May 2022. ISSN: 2379-190X. doi:10.1109/ICASSP43922.2022.9746663.
  • [8] Ya-Xi Chen and René Klüber. ThumbnailDJ: Visual Thumbnails of Music Content. In 11th International Society for Music Information Retrieval Conference, 2010.
  • [9] Martin Clancy. Reflections on the Financial and Ethical Implications of Music Generated by Artificial Intelligence. PhD Thesis, Trinity College, Dublin, Ireland, 2021.
  • [10] Dave Cliff. Hang the DJ: Automatic sequencing and seamless mixing of dance-music tracks. Technical report, Hewlett-Packard Laboratories, 2000.
  • [11] Dave Cliff. hpDJ: An Automated DJ with Floorshow Feedback. In Kenton O’Hara and Barry Brown, editors, Consuming Music Together: Social and Collaborative Aspects of Music Consumption Technologies, Computer Supported Cooperative Work, pages 241–264. Springer Netherlands, Dordrecht, 2006. doi:10.1007/1-4020-4097-0_12.
  • [12] Emmanuel Deruty, Maarten Grachten, Stefan Lattner, Javier Nistal, and Cyran Aouameur. On the Development and Practice of AI Technology for Contemporary Popular Music Production. Transactions of the International Society for Music Information Retrieval, 5(1):35–49, February 2022. Ubiquity Press. doi:10.5334/tismir.100.
  • [13] Anna Gatzioura, João Vinagre, Alípio Mário Jorge, and Miquel Sanchez-Marre. A hybrid recommender system for improving automatic playlist continuation. IEEE Transactions on Knowledge and Data Engineering, 33(5):1819–1830, 2019.
  • [14] Nell Geraets. Thy has been on the radio for six months. Turns out she isn’t real. The Sydney Morning Herald, April 2025. Section: TV & radio. URL: https://www.smh.com.au/culture/tv-and-radio/thy-has-been-on-the-radio-for-six-months-turns-out-she-
    isn-t-real-20250424-p5ltxi.html
    .
  • [15] Masatoshi Hamanaka. Real-time Future-rhythm Visualizer for DJ Performance. In Proceedings of the Sound and Music Computing Conference, Porto, Portugal, July 2024.
  • [16] Casper Hansen, Christian Hansen, Lucas Maystre, Rishabh Mehrotra, Brian Brost, Federico Tomasi, and Mounia Lalmas. Contextual and sequential user embeddings for large-scale music recommendation. In Proceedings of the 14th ACM Conference on Recommender Systems, pages 53–62, 2020. doi:10.1145/3383313.3412248.
  • [17] Drew Harwell. iHeartMedia laid off hundreds of radio DJs. Executives blame AI. DJs blame the executives. The Washington Post, January 2020. URL: https://www.washingtonpost.com/technology/2020/01/31/iheartmedia-radio-artificial-intelligence-djs/.
  • [18] Jordan V. Hawkins. Automating Music Production with Music Information Retrieval. Undergraduate Thesis, The Ohio State University, Ohio, USA, March 2013. URL: https://kb.osu.edu/handle/1811/54438.
  • [19] Tatsunori Hirai, Hironori Doi, and Shigeo Morishima. MusicMixer: Automatic DJ System Considering Beat and Latent Topic Similarity. In Proceedings, Part I, of the 22nd International Conference on MultiMedia Modeling - Volume 9516, MMM 2016, pages 698–709, Berlin, Heidelberg, January 2016. Springer-Verlag. doi:10.1007/978-3-319-27671-7_59.
  • [20] Aline Honingh, Maria Panteli, Thomas Brockmeier, David Iñaki López Mejía, and Makiko Sadakata. Perception of Timbre and Rhythm Similarity in Electronic Dance Music. Journal of New Music Research, 44(4):373–390, October 2015. doi:10.1080/09298215.2015.1107102.
  • [21] Cheng-Zhi Anna Huang, Hendrik Vincent Koops, and Ed Newton-Rex. AI Song Contest: Human-AI Co-Creation in Songwriting. In 21st International Society for Music Information Retrieval Conference,, Montreal, Canada, 2020.
  • [22] Thor Kell and George Tzanetakis. Empirical Analysis of Track Selection And Ordering In Electronic Dance Music Using Audio Feature Extraction. In Proceedings of the 14th International Society for Music Information Retrieval Conference, Curitiba, Brazil, 2013.
  • [23] Adrian Kim, Soram Park, Jangyeon Park, Jung-Woo Ha, Taegyun Kwon, and Juhan Nam. Automatic DJ Mix Generation Using Highlight Detection. In Extended abstracts for the Late-Breaking Demo Session of the 18th International Society for Music Information Retrieval Conference, Suzhou, China, October 2017.
  • [24] Taejun Kim, Minsuk Choi, Evan Sacks, Yi-Hsuan Yang, and Juhan Nam. A Computational Analysis of Real-World DJ Mixes Using Mix-to-Track Subsequence Alignment. In Proc. of the 21st Int. Society for Music Information Retrieval Conf., Montreal, Canada, 2020.
  • [25] Elad Liebman, Maytal Saar-Tsechansky, and Peter Stone. DJ-MC: A Reinforcement-Learning Agent for Music Playlist Recommendation. In Proceedings of the 14th International Conference on Autonomous Agents and Multiagent Systems, Istanbul, Turkey, May 2015. URL: https://www.ifaamas.org/Proceedings/aamas2015/aamas/p591.pdf.
  • [26] Elad Liebman, Maytal Saar-Tsechansky, and Peter Stone. The Right Music at the Right Time: Adaptive Personalized Playlists Based on Sequence Modeling. MIS Quarterly, 43(3):765–786, January 2019. doi:10.25300/MISQ/2019/14750.
  • [27] Yin-Tzu Lin, Chuan-Lung Lee, Jyh-Shing Roger Jang, and Ja-Ling Wu. Bridging Music Using Sound-Effect Insertion. IEEE MultiMedia, 22(4):30–38, October 2015. Conference Name: IEEE MultiMedia. doi:10.1109/MMUL.2015.71.
  • [28] Nigel P. Melville, Lionel Robert, and Xiao Xiao. Putting humans back in the loop: An affordance conceptualization of the 4th industrial revolution. Information Systems Journal, 33(4):733–757, 2023. doi:10.1111/isj.12422.
  • [29] Ed Montano. ‘How do you know he’s not playing Pac-Man while he’s supposed to be DJing?’: technology, formats and the digital future of DJ culture. Popular Music, 29(3):397–416, October 2010. Publisher: Cambridge University Press. doi:10.1017/S0261143010000449.
  • [30] Mark Mulligan. The ims business report 2023, 2023. URL: https://www.internationalmusicsummit.com/business-report.
  • [31] Keith Munro, Ian Ruthven, and Perla Innocenti. Can you feel it? The information behaviour of creative DJs. Journal of Documentation, 79(4), January 2022. doi:10.1108/JD-05-2022-0106.
  • [32] Meinard Müller. Fundamentals of Music Processing: Audio, Analysis, Algorithms, Applications. Springer International Publishing, Cham, 2015. doi:10.1007/978-3-319-21945-5.
  • [33] Metehan Pala. RoboDJ: automatic dj mix generation using Spotify Web API and machine learning methods. Master’s thesis, Politecnico Milano, Milan, Italy, 2021. URL: https://www.politesi.polimi.it/handle/10589/179255.
  • [34] Maria Panteli, Bruno Rocha, Niels Bogaards, and Aline Honingh. A model for rhythm and timbre similarity in electronic dance music. Musicae Scientiae, 21(3):338–361, September 2017. Publisher: SAGE Publications Ltd. doi:10.1177/1029864916655596.
  • [35] Keigo Sakurai, Ren Togo, Takahiro Ogawa, and Miki Haseyama. Controllable Music Playlist Generation Based on Knowledge Graph and Reinforcement Learning. Sensors (Basel, Switzerland), 22(10):3722, May 2022. doi:10.3390/s22103722.
  • [36] Pavan Seshadri, Shahrzad Shashaani, and Peter Knees. Enhancing sequential music recommendation with negative feedback-informed contrastive learning. In Proceedings of the 18th ACM Conference on Recommender Systems, pages 1028–1032, 2024. doi:10.1145/3640457.3688188.
  • [37] Jeremy W. Smith. The Functions of Continuous Processes in Contemporary Electronic Dance Music. Music Theory Online, 27(2), June 2021. URL: https://mtosmt.org/issues/mto.21.27.2/mto.21.27.2.smith.html.
  • [38] Robert Sowula and Peter Knees. Mosaikbox: Improving Fully Automatic DJ Mixing Through Rule-Based Stem Modification and Precise Beat-Grid Estimation. In Proceedings of the 25th International Society for Music Information Retrieval Conference, pages 850–857, San Francisco, California, USA and Online, November 2024. ISMIR. doi:10.5281/zenodo.14877463.
  • [39] Anna Spagnolli, Diletta Mora, Matteo Fanchin, Valeria Orso, and Luciano Gamberini. Automation and Creativity: A Case Study of DJs’ and VJs’ Ambivalent Positions on Automated Visual Software. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pages 1–11, Honolulu HI USA, April 2020. ACM. doi:10.1145/3313831.3376463.
  • [40] Florian Thalmann, Lucas Thompson, and Mark Sandler. A User-Adaptive Automated DJ Web App with Object-Based Audio and Crowd-Sourced Decision Trees. In Web Audio Conference, Berlin, Germany, September 2018.
  • [41] Amelia Skye Turrell. The Emotional & Neurological Effects of Expectancy and Tension Within Electronic Dance Music. PhD Thesis, University of Kent, March 2022. doi:10.22024/UniKent/01.02.93493.
  • [42] Len Vande Veire and Tijl De Bie. From raw audio to a seamless mix: creating an automated DJ system for Drum and Bass. EURASIP Journal on Audio, Speech, and Music Processing, 2018(1):13, September 2018. doi:10.1186/s13636-018-0134-8.
  • [43] Dongjing Wang, Xin Zhang, Yuyu Yin, Dongjin Yu, Guandong Xu, and Shuiguang Deng. Multi-view enhanced graph attention network for session-based music recommendation. ACM Transactions on Information Systems, 42(1):1–30, 2023. doi:10.1145/3592853.
  • [44] Brian C. Wesolowski and Alex Hofmann. There’s More to Groove than Bass in Electronic Dance Music: Why Some People Won’t Dance to Techno. PLoS ONE, 11(10):e0163938, October 2016. doi:10.1371/journal.pone.0163938.
  • [45] Alexander Williams, Stefan Lattner, and Mathieu Barthet. A Review of Music Information Retrieval and Computational Creativity Tools for Electronic Dance Music and DJing. (forthcoming).
  • [46] Alexander Williams, Haokun Tian, Stefan Lattner, Mathieu Barthet, and Charalampos Saitis. Deep Learning-based Audio Representations for the Analysis and Visualisation of Electronic Dance Music DJ Mixes. In AES International Symposium on AI and the Musician, Boston, MA, USA, June 2024. Audio Engineering Society.