Piano Transcription in the Browser with Onsets and Frames

[Adam Roberts]

Read the post at: magenta.tensorflow.org/oaf-js

Many of the generative models in Magenta.js require music to be input as a symbolic representation like MIDI, But what if you only have audio?

We have just finished porting our piano transcription model, Onsets and Frames to JavaScript using TensorFlow.js and have added it to the Magenta.js library in v1.2. Now you can input audio of solo piano performances and have them automatically converted to MIDI in the browser.

Try out the demo app Piano Scribe shown below to see the library in action for youself. If you don’t have recordings of a piano handy, you can try singing to it, and it will do its best!

Learn how to use the library in your own app in the documentation and share what you make using #madewithmagenta!

[marcel...@gmail.com]

Hi Adam,

Magenta project is amazing. Congratulations. 

I tried to upload an mp3 file with 01min17sec and 3MB which contain just piano audio of a composition of mine. But unfortunately "Transcribing..." keeps blinking and nothing happened. I waited for 1 hour and tried on Chrome on Windows, Chrome on IOS and on Internet Explorer on Windows. Do you know what may have happened?

This is a google drive link of the song: https://drive.google.com/file/d/0Bz_y2be__ZTISjZQQ0ZXckNhbzA/view?usp=sharing

Thanks,

Marcelo.

[Sharath chandra]

This is great! 

While i didn't try discretised onsets like the piano,

 i instead tried a continuous signal by whistling into the microphone.

The performance of the transcription has been interesting with changing styles of whistling and phrases.

Seems to be better at higher frequencies on the scale, with many missing or ofscale representations in between.

Perhaps the window size of the extract is indeed narrow narrow.

Just out of curiosity, cannot this be implemented without the use of tensorflow/magenta?

Perhaps using a pitch data set, and pitch follower, and some methods of extracting timing using simpler unsupervised methods

like NMF/GCC?

Regards

Sharath Chandra Ram

--
Magenta project: magenta.tensorflow.org
To post to this group, send email to magenta...@tensorflow.org
To unsubscribe from this group, send email to magenta-discu...@tensorflow.org
---
You received this message because you are subscribed to the Google Groups "Magenta Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to magenta-discu...@tensorflow.org.

[李立峰]

Hello Marcelo,

Not giving any answer here. Just to share I am experiencing similar.

My runs show that, at my Windows device, it can only transcript .wav files no longer than 8 seconds.
Here is the logs of my runs - https://docs.google.com/spreadsheets/d/1NS_IF38j0S3EY2O20ItYrzkH9ppMCtGoS5zQYh5pIDI/edit?usp=drivesdk
I am still trying different environments. And make sense of the paper to try to figure out how to make it work.

I haven't thoroughly read all the related documents. Lazily, here, I'd like to ask the app's dev team, does this web app has any minimum system requirements?

[Adam Roberts]

Hi Marcelo!

iOS is not supported unfortunately due to inadequacies in Apple's implementations of WebAudio. We haven't tested in Windows, so i'm not sure what the issue is there.  +Monica Dinculescu may be able to help figure that out.

-Adam

[Stephen Malinowski]

Hi Adam,

I tried Piano Scribe with an Art Tatum recording and was very impressed!  I posted a graphical score here ...

     https://www.youtube.com/watch?v=QXZy3OuURHM

A few days ago I came across a recording of a performance Bach flute sonata in which the continuo part (Bach only wrote a figured bass line to accompany the flute line) was realized by a group of musicians improvising in the Baroque style.  Since there's no score for what they played, I wondered whether it might be possible to get an approximation of their performance with Piano Scribe.  I uploaded the file (it's 3 or 4 minutes long) to https://piano-scribe.glitch.me/ ... that was several hours ago, and it's blinking TRANSCRIBING ...  Should I keep waiting, or is this something that's known not to work at all?

In any case, kudos and thanks for the wonderful tool.

Stephen

[Adam Roberts]

You should try the colab notebook: https://colab.research.google.com/notebooks/magenta/onsets_frames_transcription/onsets_frames_transcription.ipynb

--
Magenta project: magenta.tensorflow.org
To post to this group, send email to magenta...@tensorflow.org
To unsubscribe from this group, send email to magenta-discu...@tensorflow.org
---

[Alexander Reeves]

Hi Stephen,

Big fan of your music animation machine stuff on YouTube. If you want help setting up a docker container to run everything local to your machine I can help you out. I’m more of a hobbyist though.

-Alex

Sent from my iPhone

[Adam Roberts]

A bit more background: the browser-based app *should* eventually finish (assuming you're not using Safari), but it may take a while for very long pieces of music, and you'll likely need to keep the focus on the tab while it's working. It runs in JavaScript using WebGL for acceleration via TensorFlow.js, but is still typically slower than the TensorFlow (Python/C++) implementation on CPU.

The colab version actually runs on Google Cloud using the TensorFlowon CPU implementation. It should run faster, doesn't need to stay in the foreground, and is likely slightly more accurate.

Both should theoretically take less time than the actual piece of music, but the TensorFlow.js implementation seems to scale super-linearly from my experience.

Hope this helps!

[Curtis "Fjord" Hawthorne]

A quick note that the colab notebook (https://goo.gl/magenta/onsets-frames-colab) is currently broken. I've fixed it internally, and the fixed version will be live in a couple days. Sorry for the delay, and let me know if you have any questions about using it once it's fixed.

Also, I love the Art Tatum video! Looking forward to other uses you have for the transcription model!

-Fjord