Keith Jarrett’s Left Hand an AI-powered virtual left hand for a pianist whose left hand once changed the world

[Peter van der Putten]

Dear students,

See an idea for a thesis project below. This is a project for a Master student in AI or CS with strong skills in deep learning, an interest and knowledge of music (ideally jazz, piano, improv), and a self starter attitude. Or students from other programs with proven similar skills. For the record, we dont have connections to the musician mentioned. 

If you are interested, contact me on my LIACS email.

Peter

Keith Jarrett’s Left Hand

an AI-powered virtual left hand for a pianist whose left hand once changed the world

---

Why:

In May 2018 the legendary jazz piano virtuoso Keith Jarett (who was known, among other things, for his revolutionary use of his left hand) had a stroke which left him partially paralyzed and unable to play with his left hand. 

What:

Can we try and create a new digital left hand for Keith Jarret in the form of a midi VST plugin powered by AI? When the plugin is turned on - Keith (or anybody else) can play with his right hand on a digital piano and the plugin will generate possible left hand notes along the way (consistent with both what the right hand is doing and previous chord progressions in the left hand).

How:

1. Transcribe all of the available Keith Jarret solo performance material and produce two synced midi files for each: one for the left hand and one for the right (transcription can be partially done by hand and then another specialized trained neural network can do the rest, only to be checked by musicians.

2. Train a machine learning model to generate the next left hand notes (a couple of bars at a time) based on the previous left hand notes in the piece up until now, and what is currently (last few bars) being played with the right hand. To optimize training we may need to transpose everything to the same key.

3. Optimize the performance of the model so that it can work in real-time.

4. Package the model into a VST plugin which can be put on a midi track, given the key of the piece and then it will augment the notes as they are being played live with what Keith Jarret’s left hand would have done in this situation