Multitrack audio for MIR

[brech...@hotmail.com]

Using multitrack audio for MIR tasks 

Based on conversations during the last few days as well as a general research interest and recent pet project, I would like to propose to discuss the need for multitrack audio in MIR, and perhaps steps towards furthering the very limited availability of it. 

Without wanting to hypothesise and steer the discussion too much, these are a couple of things that could be considered. 

Why use/provide multitrack audio?

One might consider 

• applications for producers (annotation of multitrack audio for broadcast, sound design, or music production); 
• a superior performance for some MIR tasks; 
• dealing with transformations of audio such as various types of music signal processing and the implications on performance of algorithms; 
• access to separate 'exotic' tracks like separate drums or a bass guitar (high quality or with issues like bleed or noise); 
• being able to construct music with a limited set of instruments (karaoke, drums only, ...); 

etc. 

Scarcity of multitrack audio / Community efforts to make multitrack audio available

Recent efforts to provide a set of multitrack data include

• Rachel Bittner, Justin Salamon, Mike Tierney, Matthias Mauch, Chris Cannam and Juan P. Bello, "MedleyDB: A Multitrack Dataset for Annotation-Intensive MIR Research", 15th International Society for Music Information Retrieval Conference, Taipei, Taiwan, October 2014. - http://medleydb.weebly.com

which provides a dataset of multitrack audio with melody annotations, and

• Brecht De Man, Mariano Mora-Mcginity, György Fazekas and Joshua D. Reiss, "The Open Multitrack Testbed," 137th Convention of the Audio Engineering Society, Los Angeles, USA, October 2014. - http://multitrack.eecs.qmul.ac.uk

which provides a browsable/searchable testbed with metadata for songs as well as their tracks, stems, DAW files and mixes (partly hosting own content and partly linking through to other testbeds such as the above). 

Arguably the main obstacle in finding and publishing multitrack audio, are the copyright restrictions and reservations of artists and other content owners towards making their data public (by licensing it as CC or similar). How can content owners be incentivised or reassured that they will not suffer any damages from making their data available to scientists (and perhaps completely public)? 

Existing efforts use music owned by educational institutions (e.g. by having artists waive these rights when recording in the institution's facilities), using CC non-commercial licenses, only sharing data with researchers, or a combination of the aforementioned. 

If the use for multitrack audio is acknowledged by a significant part of the MIR community, perhaps the time has come to bundle efforts and think about a unified multitrack audio resource. 

Please join the conversation! 

Brecht De Man

Centre for Digital Music

Queen Mary University of London