HowTo convert live analog audio stream to live MIDI output stream with Magenta ?
609 views
Skip to first unread message
a a
unread,
Jul 29, 2020, 6:14:19 AM7/29/20
Reply to author
Sign in to reply to author
Forward
Sign in to forward
Delete
You do not have permission to delete messages in this group
Copy link
Report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to Magenta Discuss
re: HowTo convert live analog audio stream to live MIDI output stream with Magenta ?
There are apps which can live record analog audio stream into MP3, WAV or another audio format
There are apps which can convert MP3, WAV audio into MIDI
What I am looking for is
Microphone's live audio stream input from analog Piano played to be live converted into MIDI output stream to be sent to MIDI visualizer, animation application
Let me know if Magenda can take live analog audio input from a microphone and process real-time intro live MIDI oputput stream to feed MIDI visualizer, animation app
Onsets and Frames is our new model for automatic
polyphonic piano music transcription. Using this model, we can convert
raw recordings of solo piano performances into MIDI.
Onsets and Frames is an automatic music transcription framework with
piano and drums models. This notebook demonstrates running the model on
user-supplied recordings. For more details on the architecture of the
model and training datasets, see our papers:
You do not have permission to delete messages in this group
Copy link
Report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to Magenta Discuss
thank you my friend It looks like Sonus G2M hardware solution works fine. Can you hook it up to https://synthesiagame.com/ Synthesia MIDI animation application to see what is generated ?
Microphone input is ok. Can you connect radio broadcasted music as input to watch MIDI animation results ?
You do not have permission to delete messages in this group
Copy link
Report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to Magenta Discuss, mant...@gmail.com
First of all, thank you for the wonderful TF-Lite model you have provided, it is performing rather decently.
However, I would just like to ask if there are additional post processing steps needed, because the midi score I got using the TF-Lite model differs rather significantly from the one in the browser Piano Scribe even though the audio supplied is the same. From what I can see in the midi score generated by the python code provided for the realtime TF-Lite model in the github repo, it seems like there are many very short notes which are incorrectly detected.
You do not have permission to delete messages in this group
Copy link
Report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to Vincent Neo, Magenta Discuss, a a
Hi Vincent,
The main Onsets and Frames model on GitHub is our best model. The Piano Scribe and TFLite models were trained with slightly different setups to handle things like differences in spectrogram calculation and to improve inference speed. You're right that the TFLite model in particular might need more complex post-processing because it's stitching together lots of small segments instead of being able to look at the entire piece at once. We haven't had much time to experiment with ways to improve those setups, but if you find something that works better, please let us know!
You do not have permission to delete messages in this group
Copy link
Report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to Magenta Discuss, Curtis Hawthorne, Magenta Discuss, mant...@gmail.com, Vincent Neo
Hi Curtis,
Thanks for your reply. I will look more into it to see how I can improve these setups. Is it possible for you to elaborate on the differences between the TFLite model and the main Onsets and Frames model?
- Vincent
Curtis "Fjord" Hawthorne
unread,
Aug 31, 2020, 8:45:04 PM8/31/20
Reply to author
Sign in to reply to author
Forward
Sign in to forward
Delete
You do not have permission to delete messages in this group
Copy link
Report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to Vincent Neo, Magenta Discuss, mant...@gmail.com
We trained it with a slightly different method for generating the spectrogram, and there are a few variations listed on the page about whether the LSTM is bidirection/unidirectional and whether the offset stack is included.
-Fjord
Xiao-Yu Ding
unread,
Nov 16, 2020, 11:33:02 PM11/16/20
Reply to author
Sign in to reply to author
Forward
Sign in to forward
Delete
You do not have permission to delete messages in this group
Copy link
Report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to Magenta Discuss, Curtis Hawthorne, Magenta Discuss, mant...@gmail.com, vencin...@gmail.com
Hi Curtis,
Does your team have plan and timeline to make TFLite model the same quality as Piano Scribe. If not, what’s the best way to use achieve Piano Scribe quality on mobile devices?
