an interesting BBC article on AI and music creativity

[a.k.jordanous]

Hi all, 

I would be interested in hearing people's thoughts on this article? 

anna

Imogen Heap: How AI is helping to push music creativity

Once limited to human creativity, music is now being influenced and created by artificial intelligence.

BBC Click’s Stephen Beckett meets the singer-songwriter Imogen Heap who is embracing AI and also learns about an AI Eurovision Song Contest.

https://www.bbc.co.uk/news/av/technology-52236563/imogen-heap-how-ai-is-helping-to-push-music-creativity

[Ollie Bown]

I’m technically part of the Australian EuroAI team, though I didn’t actually do any of the system creation, development or music production! A “sleeper cell” / observer academic. I’ve written and commented in the past on the way these kinds of high-profile projects are communicated and evaluated, as have many in this community, welcoming avoidance of overly simplistic Turing style tests. I quite like how the EuroAI competition went about this, with a public vote but also an expert panel who reviewed submissions of works, requiring a detailed description of the process and techniques each team used. I though that struck a good balance (CF the Neukom process, which actually required the submission of an interactive program: a more ambitious and rigorous requirement, but raising the bar and barrier to entry very high). But I note that much of the coverage of it still glosses over the real intellectual interest and fixates on the very fact that AI was involved (evoking comments like “wow a computer actually composed this?”). So Anna and others, I’m curious to know how this came across if you hadn’t seen it before. It actually fascinates me more and more how much buzz this kind of thing creates (given that the “buzz” has been around since at least David Cope and all the way back to the Illiac Suite). I would have expected there to be a degree of habituation in the public reaction — "sigh not another AI music thing" — but I spoke briefly in a radio interview about it this morning and they remained "blown away". 

The works themselves do point to emerging patterns of use where we can now say that using ML-melody generation, at least, is looking like a more and more natural thing to do, and we can do more user-based research on casual usage. There’s one work there which overtly hammers home the “democratisation of music creation” narrative — an amateur musician working with off-the-shelf ML tools. I loved that this was in here. I like Imogen Heap’s artistic incorporation of the tech, I still don’t fully understand what’s happening, but that’s not always necessary.

Was there anything in particular you wanted to draw attention to in the clip Anna?

Best wishes, and I hope you are finding breathing space in your homeschooling efforts!

Ollie

--
You received this message because you are subscribed to the Google Groups "Computational Creativity Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to computational-creativ...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/computational-creativity-forum/113cc86e-c9a4-4605-aabf-e74343498755%40googlegroups.com.

[Anna Jordanous]

Thanks Ollie. 

For me, I was reminded a bit of the debates raised by the Beyond the Fence musical project - how the AI influences the creative process but wasn’t actually driving the process or being an actual partner. In other words, the computer comes across in the featured examples as a support tool for the human creator(s), rather than a creative collaborator in its own right.  This was also the case in the musical, where the human experts had material and structures generated by creative applications or machine learning, which the humans collated and ‘curated' into the musical.

Ollie, from your knowledge, and Bob Sturm (cc-d), as  I believe you might also have been involved in the AI Eurovision contest - was this the case for most entries? All entries? Some entries?

Interesting to hear how the Eurovision contest was evaluated - I see voting is still open till 10 May… https://www.vprobroadcast.com/titles/ai-songcontest/vote.html

(Disclaimer: Anna Kantosalo and I were about to present a paper at AISB on the types of roles computers tend to occupy in human-computer co-creativity / mixed-initiative co-creation work, so I guess that is driving my interpretation, as I’m more primed to notice this sort of thing.)

I’ve cc-d in Tom Collins, who was working with Imogen Heap and was involved in the AI Eurovision too - did you want to comment at all Tom?

Anna

[Bob Sturm]

Hi all,

Yes, I was involved in this project: https://www.vprobroadcast.com/titles/ai-songcontest/teams/sweden.html 

I can’t say how the other entrants worked, but can share details about our work. It appears we were the only ones who tried to train a model to generate lyrics + melody together. Here’s one example output I selected randomly created by one system trained on syllable-note sequences of length 30:

|    MIDI     | Note  | Duration | Syllable |
|-------------+--------+------------+------------|
|            57 | A 4    |        0.5 | you        |
|            52 | E 4    |        0.5 | want       |
|            57 | A 4    |        0.5 | me         |
|            52 | E 4    |        0.5 | tru        |
|            61 | C# 5  |        1   | ly         |
|            52 | E 4    |        0.5 | el         |
|            57 | A 4    |        0.5 | and        |
|            59 | B 4    |        0.5 | me         |
|            57 | A 4    |        1   | you        |
|            33 | A 2    |        0.5 | and        |
|            36 | C 3    |        0.5 | i          |
|            52 | E 4    |        2   | you        |
|            57 | A 4    |        1   | i          |
|            45 | A 3    |        0.5 | you        |
|            49 | C# 4  |        0.5 | so         |
|            45 | A 3    |        0.5 | you        |
|            57 | A 4    |        1   | me         |
|            64 | E 5    |        0.5 | way        |
|            61 | C# 5  |        0.5 | me         |
|            57 | A 4    |        0.5 | think      |
|            52 | E 4    |        0.5 | i          |
|            57 | A 4    |        0.5 | you        |
|            33 | A 2    |        1   | and        |
|            56 | G# 4  |        0.5 | dont       |
|            64 | E 5    |        0.5 | you        |
|            42 | F# 3  |        0.5 | tell       |
|            59 | B 4    |        0.5 | just       |
|            42 | F# 3   |        0.5 | the        |
|            52 | E 4    |        1   | way        |
|            33 | A 2    |        0.5 | you        |

Out of 100 of these generated examples, maybe  2 are interesting. We also create models of sequences of length 20, 60 and 500.

This was my first involvement in modeling music other than folk music with very restricted forms. I have some immediate comparisons with regards to personal experience working with material generated by our folk-rnn models. 

In my own creative work, I have found that material generated by folk-rnn models is far more pleasant to work with because it is often highly plausible from the start. Curating from the "baskets of cherries” produced by these models is exciting, and sometimes it is hard to pull myself away from them and spend time learning _real_ traditional tunes. On the contrary, the material generated by our Eurovision models was so musically senseless that the time spent finding a few “salvagable" items didn’t feel justified. It was much less enjoying solving a puzzle, than drudgery towards a deadline. 

Bob L. T. Sturm, Associate Professor
Tal, Musik och Hörsel (Speech, Music and Hearing)
Lindstedtsvägen 24
School of Electronic Engineering and Computer Science
Royal Institute of Technology KTH, SE-100 44 Stockholm, Sweden
https://highnoongmt.wordpress.com

[Ollie Bown]

Just to chime in again with a fun reference about judging AI-generated melodies that have been human-produced: my fellow expat James Humberstone from the Sydney Conservatorium does an entertaining TedX talk where he gets the audience to pick random notes and then makes a convincing Dubstep tune out of them https://www.youtube.com/watch?v=d8s8e8JdGCc (from about 5 minutes in). It’s a cheap trick but it does highlight the issue in a fun way. This doesn’t mean the AI might not be phenomenally good, it just means it’s hard to judge.

Ollie

On 21 Apr 2020, at 10:15 pm, Tom Collins <t...@musicintelligence.co> wrote:

Hi Anna and all,

Thanks for copying me in, and for your great work in this area!

In terms of my motivations for participating, I am quite frustrated when I see things like "Daddy's car" on YouTube, where the title says "a song composed by AI". I wanted to contribute to a project that increased the transparency and public understanding of this topic. It's fundamentally problematic and wrong to mislead the public about "where AI is at", especially if you’re using public money in the form of a research grant, because it sows distrust in AI in particular, and science and academia in general.

The details of the UK submission, "Hope rose high" by Brentry, are here:

https://musicintelligence.co/2020/

as well as a chance to play with the music-generating algorithm.

There were some late twists and turns in our submission. For the song that Nancy and I entered, "the starting material is almost all AI, but the combination of these materials and the instrumentation has been determined/created by humans" (this quote from the URL above). 

For the song that Imogen Heap wrote, but couldn't finish because of the lockdown, it was a similar situation, although she selected from and added to the musical and lyrical material a bit more than did Nancy and I. Both songs began with the same "raw AI-generated musical materials" – 30 melodies (although some of these were polyphonic due to an encoding problem with 10 or so I guess of the MIDI files supplied by the Dutch broadcaster), 30 simplified chord sequences (simplified by whoever made the dataset), 30 simplified bass lines, and 30 drum beats – but different runs of the lyric-generating algorithm.

To speak to Bob's point about lyrics and melody in one go, this would be possible with my approach, but I don't think it's worth trying until there's a rhythmic representation of the lyrics available. The Dutch broadcaster only provided plain text (e.g., not synced to a melodic representation).

There didn't seem to be much interest as far as I could tell in using my algorithm as a creative collaborator. In my opinion, when you hear an output, your response is either "OK, we'll go with that even though I might change X to Y" or "Not OK, I need to change X to Y to make this work". I never felt like "Let's ask the algorithm to try again from the fifth note onwards", say.

My previous research in this area has been concerned with generating entire (albeit quite short) pieces of music. I knew that wasn't going to be of interest either – rather (A) something that could generate lots of 4-bar excerpts, and then perhaps (B) something that could suggest how multiple segments might be distributed or arranged across a whole song. I provided (A) and that was sufficient on this occasion.

Karen van Dijk (VPRO) and Vincent Hendrik Koops (RTL, formerly of Utrecht University and still very much in the MIR community) should take a lot of credit for the public/expert split, and general excellent organisation of the project. If you're interested, I could paste you the channel from the MIR slack where this project was being discussed since mid-to-late last year. Stephen Beckett and Jonathan Coates from BBC Click also did a really great job in doing background reading and speaking to a lot of people before producing their piece.

All best,

Tom

Tom Collins, PhD
https://mstrcyork.org
https://tomcollinsresearch.net
https://musicintelligence.co
Lecturer (Assistant Professor) in Music Technology
Department of Music
University of York

[Tom Collins]

In case anyone thought I was rude/slow and didn't reply, I replied from an address that it isn't part of the group so it bounced! You can see my reply below Ollie's.

Cheers!

Tom

Hi all,

To unsubscribe from this group and stop receiving emails from it, send an email to computational-creativity-forum+unsubscribe@googlegroups.com.