Guitar note onset detection?

[Björn Lindqvist]

Hello!

I'm trying to create a proof-of-concept for automatically grading

simple finger style guitar performances of melody lines using note

onset detection. I wonder if you think Madmom would work for my use

case? From the articles I have read, it seems like Madmom contains

good general-purpose onset detectors but none tuned for specific

instruments or settings?

If Madmom would be a good fit, which of the algorithms in it do you

recommend? I've tried the RNNOnsetProcessor and it seem to work fairly

well. Do you think its performance could be improved by retraining it

on more representative recordings?

--

mvh/best regards Björn Lindqvist

[Sebastian Böck]

Hi,

On Tuesday, 29 January 2019 15:04:29 UTC+1, Björn Lindqvist wrote:

Hello!

I'm trying to create a proof-of-concept for automatically grading

simple finger style guitar performances of melody lines using note

onset detection. I wonder if you think Madmom would work for my use

case? From the articles I have read, it seems like Madmom contains

good general-purpose onset detectors but none tuned for specific

instruments or settings?

yes, it can be used if you're only interested in the onset timing and not on the pitch being played.

And you're right that it was never tuned towards a specific instrument.

 

If Madmom would be a good fit, which of the algorithms in it do you

recommend? I've tried the RNNOnsetProcessor and it seem to work fairly

well. Do you think its performance could be improved by retraining it

on more representative recordings?

If you don't have too noisy recordings, SuperFlux could be a good fit as well and comes with the benefit of having a higher temporal resolution. You could also try CNNOnsetProcessor, but it works only offline (RNNOnsetProcessor can operate on audio streams as well).

Of course retraining can help if you have some annotated training data.

[Björn Lindqvist]

Hi!

I've now played around with Madmom a bit and it seem to work nicely!
Both the SpectralFlux detector and the RNN detector detects almost all
notes played. Even when recording using a standard cell phone mic.

But an issue I didn't expect have shown up. The detectors classify
noise from the fretting hand as onsets. You can hear examples of the
noise I'm talking about in the intro to this video:

https://www.youtube.com/watch?v=ia_0Vs5WB94

Another example is around 0:22 in this video:

https://www.youtube.com/watch?v=kRIPKV0pkzA

It is barely audible, but if you listen carefully, you can hear a
slight noise as the player is moving his hand. This kind of noise is
seen as an imperfection but is mostly unavoidable on acoustic
guitars. Annoyingly, this noise is being picked up as an onset by
Madmom. I wonder if there is anything that can be done about it? Like
"anti-training" the RNN on this particular noise or maybe tweaking
some other parameters?

I have a bit of training data. About 600 labelled onsets from 20 short
recordings. Do you think that would be enough to improve the
performance of the RNN? It is not a very large dataset!

Den tis 29 jan. 2019 kl 15:10 skrev Sebastian Böck
<sebastian.bo...@gmail.com>:

> --
> You received this message because you are subscribed to the Google Groups "madmom-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to madmom-users...@googlegroups.com.
> To post to this group, send email to madmom...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/madmom-users/04c1c486-aa3d-4f3a-98e0-1731fa145d63%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Sebastian Böck]

Hi,

the simplest you could try is to increase the threshold and see if you can find one which makes these spurious sounds disappear.

But you could try to re-train the models with your annotated data. Please see my part of the ISMIR 2018 tutorial on deep learning for MIR, which deals with onset detection: https://github.com/slychief/ismir2018_tutorial/blob/master/Part_3b_RNN_Onset_Detection.ipynb

HTH

[Björn Lindqvist]

Hi!

Sorry for the late reply. I've done my best trying to tweak the
threshold so as to get it to skip fretting hand noise but still
classify notes as onsets. But I'm not making much progress. It seems
like the volume of the noise is the same or even higher than the volume
of the notes which confuses the detector.

So I'm thinking about alternative ideas to try and filter and/or
pre-process away this noise. Perhaps one can try and have the algorithm
"listen" to the sound? Fretting hand noise should have a more staccato
and perhaps "jagged" sound while notes are more mellow. E.g using
madmom's RNN to detect onsets and then use a different ANN to classify
the onset as a note or not?


--
mvh Björn

Den fre 1 feb. 2019 kl 13:01 skrev Sebastian Böck
<sebastian.bo...@gmail.com>:

> --
> You received this message because you are subscribed to the Google Groups "madmom-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to madmom-users...@googlegroups.com.
> To post to this group, send email to madmom...@googlegroups.com.

> To view this discussion on the web, visit https://groups.google.com/d/msgid/madmom-users/19502823-ace9-4ff4-9c32-9b4ff1913324%40googlegroups.com.

[Silvan Laube]

Hi both, I'm working on a similar thing but for piano

For the purpose of retraining I'd like to ask what kind of data was used for the training the provided RNN (instruments? percussion? generated sounds? how many onsets for decent results?)

Also I kinda struggle optimizing superflux parameters - they are quite similar to RNN, but still don't perform as well as aubio-specflux (at least in online mode aubio is much faster and more accurate)... i was hoping it'd outperform that method

Am Sonntag, 24. Februar 2019 18:32:26 UTC+1 schrieb Björn Lindqvist:

Hi!

Sorry for the late reply. I've done my best trying to tweak the
threshold so as to get it to skip fretting hand noise but still
classify notes as onsets. But I'm not making much progress. It seems
like the volume of the noise is the same or even higher than the volume
of the notes which confuses the detector.

So I'm thinking about alternative ideas to try and filter and/or
pre-process away this noise. Perhaps one can try and have the algorithm
"listen" to the sound? Fretting hand noise should have a more staccato
and perhaps "jagged" sound while notes are more mellow. E.g using
madmom's RNN to detect onsets and then use a different ANN to classify
the onset as a note or not?


--
mvh Björn

Den fre 1 feb. 2019 kl 13:01 skrev Sebastian Böck

<sebastian.b...@gmail.com>:

>
> Hi,
>
> the simplest you could try is to increase the threshold and see if you can find one which makes these spurious sounds disappear.
>
> But you could try to re-train the models with your annotated data. Please see my part of the ISMIR 2018 tutorial on deep learning for MIR, which deals with onset detection: https://github.com/slychief/ismir2018_tutorial/blob/master/Part_3b_RNN_Onset_Detection.ipynb
>
> HTH
>
> --
> You received this message because you are subscribed to the Google Groups "madmom-users" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to madmom...@googlegroups.com.

[Sebastian Böck]

Replying to the last two messages.

The RNN for onset detection was trained on a dataset of mixed sounds. For details, please refer to [1].

It doesn't matter if your trying to detect piano or guitar onsets, it should work for both. But at least for guitar you'd have the problem of false positives (e.g. fretting sounds) which are onsets in a sense that there is a (new) sound. So if you're interested in detecting only the "real" onsets, I guess there's no other way than training a network with positive and negative examples. Of course you could also define a rule to discard the false positive detections, but I think it is easier and more straight forward to train a network to not detect the spurious onsets in the first place.

[1] Sebastian Böck, Florian Krebs and Markus Schedl, Evaluating the Online Capabilities of Onset Detection Methods, Proceedings of the 13th International Society for Music Information Retrieval Conference (ISMIR), 2012.