Mar 11, 2022, 11:51:35 AMMar 11
to Magenta Discuss, j...@jbb.dev, Daniel Chung, b.lozan...@gmail.com, Magenta Discuss
Sounds promising, I'll be in touch shortly, but do let me know if I could be of help in your project! My background is in programming and IT, but not AI/machine learning. I'm somewhat active in the Sonic-Pi community, where in the past I've developed various open source algorithmic music projects exploring polyphony. I've also produced music for years as a hobby, recently incorporating Magenta Studio in my work.
I agree that such generic pre-trained models could mitigate the problem of scarce training sets. Definitely an idea to iterate upon, once there's a working prototype.
I foresee several problems in dealing with multi-instrumental music:
- With the exception of the piano, most instruments' MIDI data include "continuous controller" values for natural performances (the big CCs being expression, dynamics, and vibrato). The simple token format used by MuseNet won't cut it for realistic performances.
- Convincing orchestrations require high-quality virtual instruments, whether they be sample-based or model-based. Either way, they won't be cheap to license. VSCO 2 Community Edition would only be bearable for a proof of concept.
- Contemporary music makes extensive use of audio samples, and acquiring these samples would require negotiation with sample providers, if we can't find them for free.
The possibilities are indeed endless, and I think the time has come for this idea. An MVP should focus on the low-hanging fruit, like piano reductions. Do it well enough, and people might be willing to fund increasingly ambitious projects. Only then will artists sign up to be distributed.