Habibur Rahman, employed as research assistant for MiningSuite

[Olivier Lartillot]

Dear dev list,

I am pleased to announce that the RITMO group (where I work) is employing Habibur Rahman as research assistant for 100 hours, helping me to prepare a public version of the MiningSuite for this Fall.

Welcome, Habib!

This is what was aimed initially:
• Checking that MIRtoolbox and MiningSuite give same results. Continuing integrating the MIRtoolbox documentation into the MiningSuite wiki. Documenting the changes between MIRtoolbox and the MiningSuite.
• Continuing the integration of MIDI Toolbox and MoCap Toolbox into the MiningSuite. Integrating the documentation.
• Helping integrating and documenting new features in the MiningSuite.
• Documentation oriented towards would-be contributors to the open source project: Documentation of the MiningSuite architecture and code. Guide explaining how to develop new operators in the toolbox.
• Assisting in users’ support: investigating and replying to questions asked in the discussion list, open issues, etc.

Trying to following good open-source project standards, I think we should keep our MiningSuite dev related discussion public, via this mailing list.

Habib, could you write a short email indicating what you have been doing so far in this work? We can continue the discussion from that point.

Cheers,

Olivier

[habibur.rahman]

Hi Olivier. Thank you very much to have me on your team. 

So far, I have started comparing the result of the routines in mirtoolbox with the corresponding routines implemented in miningsuite. The testing script can be found in the dev branch of the repository. 

An excel file named 'migration' lists of the result of the comparison (if they have a matched result, or different result or the comparison of a routine could not be completed due to some error. The other objective of this list is to help develop a fully structured migration guide, and also list those functions/ feature parameters that are not implemented in the miningsuite version. 

The testing has been completed for the miraudio, mirframe and mirfilterbank. Some of the functions have matched results for both mirtoolbox as well as miningsuite, some have a different result, and some comparison could not be performed due to some error returned while calling the routine. 

When mirspectrum in mirtoolbox was compared to sig.spectrum in miningsuite, the result did not match. A close inspection showed that the results were almost same, but the length of the array was different. (i.e. for a sample file, sig.spectrum samples one less element compared to mirspectrum). The other routines that failed giving matching result, can also be suspected to have this issue and will need to be looked into further detail. 

[Olivier Lartillot]

Hi Habib,

Thanks for your introduction.

To explain further, you first committed the file migrationTest/migrationTest.m containing the Matlab tests (commits 697358707ee4532fc79640292255dc6d3ade3669 and 347e5dabe1bc82c947244ef7d14ffbeae6104eae)

In your most recent commit (1d39c06e35c8e2f1f441e5e771c429f366a3873c), you decided to split migrationTest.m into 3 files:

- mirgetdata_vs_objgetdata.m

- mirframe_vs_sigframe.m

- mirfilterbank_vs_sigfilterbank.m

I did meanwhile an update of migrationTest.m (commit be8c9fdd7e8fa903c6dd800920dbf04e5904df30) with the following suggestions:

- We need to test on a stereo file instead of a mono file.

- There was an error with the ‘Mix’ option in sig.signal, which is actually the reverse of the ‘Mono’ option in miraudio. As such, ‘Mix’ needs to be added everywhere.

- Testing 'Sampling' with lower sampling rate, so that we can see an actual downsampling.

- I suggest to use clearvars at beginning of section of code, not end. In that way, when we evaluate a single section, we can examine the computed variables in Matlab.

Since you removed most of the code from migrationTest.m to structure them into separate files dedicated to the different operators, I have committed my suggestions into a new file migrationTest.m. So it would be great, Habib, if you could have a look at my suggestions and integrate them in the rest of your files if you agree.

About “mirgetdata_vs_objgetdata.m”, I suggest to call it instead miraudio_vs_sigsignal.m, because all the operators are about getdata anyway.

Cheers

Olivier

--
You received this message because you are subscribed to the Google Groups "miningsuite-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to miningsuite-d...@googlegroups.com.
To post to this group, send email to minings...@googlegroups.com.
Visit this group at https://groups.google.com/group/miningsuite-dev.
To view this discussion on the web visit https://groups.google.com/d/msgid/miningsuite-dev/aa59d1a3-ddc2-43e6-939d-6d4bc53a8adf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[habibur.rahman]

Commit: aa1b755d10a2a0b176bc01f1cfa2be1283a6ba7a 

created test script for comparing mirenvelope with sig.envelope.

I have not been able to implement the function feature with the parameter  (…, 'Frame', .1, 's', .1, '/1', ' Window', 'hanning', b) as it is giving me some error. 

The testing has been done with stereo file 'ragtime.wav' and as Olivier suggested, the parameter 'Mix' has to be included in all functions in miningsuite. Most functions have a matching result for both the function, except for the following feature parameters 

'Frame'
'Mu' 

'Diff'

'HalfwaveDiff'

'Normal', 'AcrossSegments'

'Smooth'

'Gauss'

I will try to analyze the cause of mismatch (if due to something simple as length of array, or something simple)

The test script mirframe, when run causes my Matlab running on Windows 10, to hang completely. Some of the routine implemented engages the processor. I am not sure if it has something to do with the  parameter 'h' (length of the window in seconds where default is 0.05 seconds). I have tried changing the value in the range from this default until 3, but the problem persists. 

for simplicity, the name of each file has been changed to keep only that from mirtoolbox. For example, the filename miraudio_vs_sigsignal has now been changed to migrationTest_miraudio.

Commit ce5f9e76c42ccb72ad203d6921a4cd97472808db

implemented changes proposed by Olivier in migrationTest.m file into the corresponding script files. The keyword 'Mix' has been implemented in those functions for miningsuite and now those successful test routines gives the same result for stereo files as well. Clearvars has been moved to the beginning of each test code section, instead of at the bottom. 

[Olivier Lartillot]

Great.

1: I suggest to rename migrationTest_mirgetdata.m migrationTest_miraudio.m

Please add ‘FWR’ option, also available in sig.signal. You need to use the repository version of MiningSuite, as it has been added after latest release 0.10

https://github.com/olivierlar/miningsuite/wiki/Release-History#version-01011

Add also ‘Frame’ in sig.signal

For instance:

sig.signal('ragtime.wav','Frame','FrameLength',2,'FrameHop',.5)

For ‘Extract’, in ’sp’, you shuld use for instance t2 = 88200

When you test the change of some parameters, such as ’TrimThreshold’, it would be better to test with another value than the default value. In this case, for instance, 0.03

Your ’TrimStart’ routine is correct. Actually you could check that trim works OK by displaying a and b and zooming on the beginning and end of sequences.

2:  migrationTest_mirframe.m

You forgot to use ragtime.wav

I suggest the test would be with all defaults, so without using ‘Length’ and ‘Hop’. It corresponds to .05 s and hop = 0.5

Then try with a length of 1000 samples, let’s say

Then with default length and various hop factor:

- .1, with default unit (/1)

- 25 %

- .01 s

- 100 Hz

(Your choice of hop = 3 sp is not good because it leads to too many frames, hence the computation problem. I advice you to look at the explanation of frame decomposition in the doc.)

3: migrationTest_mirfilterbank.m

It seems the tests fail, right? We need to check this issue also.

(I haven’t checked the rest yet)

--
You received this message because you are subscribed to the Google Groups "miningsuite-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to miningsuite-d...@googlegroups.com.
To post to this group, send email to minings...@googlegroups.com.
Visit this group at https://groups.google.com/group/miningsuite-dev.

To view this discussion on the web visit https://groups.google.com/d/msgid/miningsuite-dev/67db54c1-2bd5-42a1-b811-b17998d67afb%40googlegroups.com.

[Olivier Lartillot]

Hi,

I have checked and correction migrationTest_mirfilterbank.m

Commit: 9d3784267f48a84cbbd874c086fabf3efd7bd811

Some comments to take into consideration for the other migration tests:

- Please make sure that the comparison between results performs a squeezing of the mirgetdata output:

tf = isequal(squeeze(mirgetdata(a)),b.getdata);

- Ideally, all options available in the MIRtoolbox documentation should be tested. Here, I added ’NbChannels’ that was missing

- I understand that the ‘Manual’ option in mirfilterbank (= sig.filterbank)  is somewhat technical and very succinctly documented in MIRtoolbox and MiningSuite guides. So I corrected the following problems in the migration tests: ‘Order’ needs to be an option of sig.filterbank, not aud.filterbank. And to use ‘Hop’, the array of frequencies in ‘CutOff’ should not be too small.

- I corrected a bug in aud.filterbank (‘Mel’) and committed it to the master branch. Please add it also to the dev branch.

Cheers,

Olivier

[Olivier Lartillot]

Concerning 1: migrationTest_mirgetdata.m 

Your new commit is good, Habib. And sorry about ‘FWR’ option. I did not realize that you could not see it in the public version of  MIRtoolbox User’s Guide.

Concerning 2:  migrationTest_mirframe.m

I am afraid this migration test file is not correct. I advise to have a closer look at it, taking int account my previous comments, while also looking at how mirframe works exactly.

We can discuss about it via Skype if necessary.

[Habibur Rahman]

in migrationTest_mircepstrum.m, i have looked into mircepstrum (..., 'Frame') -> sig.cepstrum(..., 'Frame').

If I compare the data, they look almost the same. However, I believe there are some decimal level differences.

For example, if you take the first data from 'ragtime.wav', it shows 342.5248 by both mircepstrum and sig.cepstrum. But if I subtract the two data, I get a difference of 1.1369e-13. This is causing the test to be failed. I suspect the issue could be similar in the following routines as well in the test file.  

in mirautocor and sig.autocor, the parameter 'Normal', n with n = 'biased', 'unbiased' or 'coeff', or 'none' works but without using the keyword 'Normal'. For eg. minautocor(testfile, 'Normal', 'biased') gives and error whereas minautocor(testfile, 'biased') is accepted.

mir autocor with enhanced feature returning error. Need to check how I can implement the enhanced feature 

[Habibur Rahman]

Have tested mirframe and all test results match with those of miningsuite. However, I noticed that the mirframe(...,'Hop', h, '%') does not have a corresponding function in miningsuite. Did i miss out anything?

No function matches for mirautocor and mirpeaks. The data are completely different. I tried rounding off the data, but still there are different values in the array which results in a mismatch in the data. 

I will need to discuss with you further to understand and implement the mirsegment. Let me know when you are available for a skype meeting.

[Olivier Lartillot]

> 2019/07/29 19:21、Habibur Rahman <habibur...@fys.uio.no>

>
> Have tested mirframe and all test results match with those of miningsuite. However, I noticed that the mirframe(...,'Hop', h, '%') does not have a corresponding function in miningsuite. Did i miss out anything?

Yes indeed this feature is missing. I guess it should be simple to add it in the MiningSuite. I can check that.

>
> No function matches for mirautocor and mirpeaks. The data are completely different. I tried rounding off the data, but still there are different values in the array which results in a mismatch in the data.

For mirautocor, I checked the first test, and the difference between the two toolboxes is somewhat small in absolute value:
d = mirgetdata(a)
e = b.getdata
max(max(abs(d - e)))

ans =

0.0033

(Autocorrelation values are between -1 and 1).

We should check the implementation in both toolboxes to see what explains this small discrepancy.

For mirpeaks, you should not use mirgetdata. Please look at “Accessible Output” at the end of mirpeaks doc and sig.peaks doc.

You can compare the peak indices:
get(b,'PeakIndex')
get(a,'PeakPos’)

(You need to check the data format of each output to compare in a correct way)

>
> I will need to discuss with you further to understand and implement the mirsegment. Let me know when you are available for a skype meeting.

Concerning the first type of segmentation as explained in the doc. Here is a MIRtoolbox example:

e = mirenvelope('ragtime.wav')
p = mirpeaks(e)
s = mirsegment('ragtime.wav',p)

MiningSuite:

ee = aud.envelope('ragtime.wav','Mix')
pp = sig.peaks(ee)
ss = sig.segment('ragtime.wav',pp,'Mix')

Then check that the start and end of segments are the same in each toolbox?

[Olivier Lartillot]

About mirenvelope:

For the ’Spectro’ option, you need to use this comparison test:
tf = isequal(squeeze(mirgetdata(a)),b.getdata');
I have committed this change in GitHub. It works.

All the options below ’Spectro’ are related to the ’Spectro’ option, so you need to add ’Spectro’ each time, and use the comparison test above.
For instance:
mirenvelope(…, ’Spectro’, ‘Freq’)

The line mirspectrum(..., ‘Frame’, .1, ‘s’, .1, ‘/1’, ‘Window’, ‘hanning’, b, ‘Power’) is to explain how ’Spectro’ works. You should not test it, and replacing the command with mirenvelope is incorrect.

> 2019/08/01 14:34、Olivier Lartillot <olart...@gmail.com>のメール:

[Habibur Rahman]

does mirenvelope have correspoinding functions in miningsuite for trim, postSilence and preSilence?

[Olivier Lartillot]

No : https://github.com/olivierlar/miningsuite/wiki/MIRtoolbox-1.x-features-not-integrated-into-the-MiningSuite

I just solved the problem with ‘Diff’. Will explain soon.

2019/08/08 13:55、Habibur Rahman <habibur...@fys.uio.no>のメール:

does mirenvelope have correspoinding functions in miningsuite for trim, postSilence and preSilence?

--
You received this message because you are subscribed to the Google Groups "miningsuite-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to miningsuite-d...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/miningsuite-dev/88dfba92-1ec4-498a-9616-975157eb735c%40googlegroups.com.

[Olivier Lartillot]

Sorry, maybe I was not clear about ’Spectro’ option.

I started correcting ’Spector’ in commit 2292337646a71d3c57be6ba711a7be82e7cbeee3

The default “b” option in ’Spector’ is ‘Freq’ so I tested with ‘Mel’ instead. I had to update the comparison test, as I explained in my previous mail. But the test fails because there are tiny difference between the two results. We need to check why.

So what needs to be done is to test also ‘Bark’ and ‘Cents’ and the rest of ’Spectro’ option:
mirenvelope(testfile, ’Spectro’, ‘Frame’) with different frame defaults
mirenvelope(testfile, ’Spectro’, ‘UpSample’)
etc

> 2019/08/02 18:22、Olivier Lartillot <olart...@gmail.com>のメール:

[Olivier Lartillot]

Hi,

Here is an update of what needs to be done in mirenvelope:

For mirenvelope(…,’Frame’), we need to add also ’Spectro’, because ‘Frame’ works only for ’Spectro’. Besides, in sig.envelope, the option is renamed ’SpectroFrame’. Finally, that (Spectro)Frame option should not used alone, but followed with arguments, else it is useless.

Finally, for all the ‘Spectro’ tests, the comparison needs to be updated, as I previously mentioned:
tf = isequal(squeeze(mirgetdata(a)),b.getdata’);

So to sum up, here is how it should look like:
a = mirenvelope(testfile, 'Spectro', 'Frame',.1,.2);
b = sig.envelope(testfile, 'Spectro', 'SpectroFrame',.1,.2, 'Mix');
tf = isequal(squeeze(mirgetdata(a)),b.getdata’);

Cheers

[Olivier Lartillot]

Oh, and I finally removed the ‘Mel’, ‘Bark’ and ‘Cents’ options in sig.envelope, so no need to care about those.

[Habibur Rahman]

I have created a small debugFail routine that identifies if the results between the routines in mirtoolbox and miningsuite match. If they fail, the routine tries to find out if the error is due to a dimension mismatch (and by how much) or differences in the data itself (showing absolute maximum difference). Using the routine, I have been able to identify and categorize the following problems. 

1) differences in data size just by 1. 

 

mirenvelope (…, 'Gauss', o)  -> sig.envelope (…, 'Gauss', o)  ; sig.envelope produces one extra element than mirenvelope

mirspectrum (…, 'Max', mi) -< sig.spectrum( …, 'Max', mi) ; sig.spectrumproduces one less element than mirspectrum

mirspectrum (…, 'Gauss', o)  -> sig.spectrum (…, 'Gauss', o)  ; sig.envelope produces one extra element than mirenvelope

2) insignificant differences in data values

mirspectrum -> sig.spectrum

'NormalLength'

mircepstrum -> sig.cepstrum

..., 'Frame'

..., 'Freq'

..., 'Min',  min

..., 'Max', max

mirautocor -> sig.autocor

…, 'Frame'

…, 'Min', mi

…, 'Min', mi, 's'

…, 'Min', mi, 'Hz'

…, 'Max', ma

…, 'biased'

…, 'unbiased'

…, 'coeff'

…, 'none'

…, ‘NormalWindow’, ‘hanning’

…, ‘Halfwave’

…, ‘Compress’, k

3) data different by exactly 10 times (possibly a missing/extra 0 in the code)

mirspectrum (…, 'Smooth', o)   = 1/10 the value of  sig.spectrum (…, 'Smooth', o) 

4) significant data differences

mirenvelope -> sig.envelope

…, 'Diff'

…, 'HalfwaveDiff'

…,Normal', 'AcrossSegments'

mirspectrum-> sig.spectrum

…, 'TimeSmooth', o

5) significant dimension error

mirenvelope -> sig.envelope

…, 'Smooth', o

mirspectrum-> sig.spectrum

…, ‘MinRes, r, ‘OctaveRatio, tol

6)  no output produced (data dimension = 0 0)

mircepstrum -> sig.cepstrum 

…, ‘Min’, min  (miningsuite version produces no output with 0 dimension)

7) implementation error

mirsegment -> sig.segment

…, "HCDF"  

[Olivier Lartillot]

Sorry for the very late reply.

Replies below, related to bugs concerning mirenvelope

10. nov. 2019 kl. 08:47 skrev Habibur Rahman <habibur...@fys.uio.no>:

I have created a small debugFail routine that identifies if the results between the routines in mirtoolbox and miningsuite match. If they fail, the routine tries to find out if the error is due to a dimension mismatch (and by how much) or differences in the data itself (showing absolute maximum difference). Using the routine, I have been able to identify and categorize the following problems. 

1) differences in data size just by 1. 

 

mirenvelope (…, 'Gauss', o)  -> sig.envelope (…, 'Gauss', o)  ; sig.envelope produces one extra element than mirenvelope

I found the bug. Solved. :)

mirspectrum (…, 'Max', mi) -< sig.spectrum( …, 'Max', mi) ; sig.spectrumproduces one less element than mirspectrum

mirspectrum (…, 'Gauss', o)  -> sig.spectrum (…, 'Gauss', o)  ; sig.envelope produces one extra element than mirenvelope

Same as above.

The test does not fail on Diff and HalfwaveDiff if I use version 1.7.2.1 of MIRtoolbox or my current latest one. Which version are you using?

When I use 1.7.2 it fails but actually debugFail itself returns an error, because you call    debugFail(a,b); instead of    debugFail(x,y);

…,Normal', ‘AcrossSegments'

AcrossSegments is not available in mining suite. This is indicated in the wiki page.

mirspectrum-> sig.spectrum

…, 'TimeSmooth', o

5) significant dimension error

mirenvelope -> sig.envelope

…, 'Smooth', o

Nice bug found here in MiningSuite. I just corrected it. :)

mirspectrum-> sig.spectrum

…, ‘MinRes, r, ‘OctaveRatio, tol

6)  no output produced (data dimension = 0 0)

mircepstrum -> sig.cepstrum 

…, ‘Min’, min  (miningsuite version produces no output with 0 dimension)

7) implementation error

mirsegment -> sig.segment

…, "HCDF"  

--
You received this message because you are subscribed to the Google Groups "miningsuite-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to miningsuite-d...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/miningsuite-dev/dddb9ac4-6265-4ea9-b263-3f5c3101f08e%40googlegroups.com.

[Olivier Lartillot]

Here are my answers concerning this time mirspectrum.

Please note that you should also check aud.spectrum and mus.spectrum. So for instance mirspectrum(...,‘Terhardt’) is in fact implemented, and is in aud.spectrum.

10. nov. 2019 kl. 08:47 skrev Habibur Rahman <habibur...@fys.uio.no>:

I have created a small debugFail routine that identifies if the results between the routines in mirtoolbox and miningsuite match. If they fail, the routine tries to find out if the error is due to a dimension mismatch (and by how much) or differences in the data itself (showing absolute maximum difference). Using the routine, I have been able to identify and categorize the following problems. 

1) differences in data size just by 1. 

 

mirenvelope (…, 'Gauss', o)  -> sig.envelope (…, 'Gauss', o)  ; sig.envelope produces one extra element than mirenvelope

mirspectrum (…, 'Max', mi) -< sig.spectrum( …, 'Max', mi) ; sig.spectrumproduces one less element than mirspectrum

MIRtoolbox is faulty here, I corrected mirspectrum, and now it works OK. I can send you the newer version of MIRtoolbox, or you can simply consider that this test now succeeds.

mirspectrum (…, 'Gauss', o)  -> sig.spectrum (…, 'Gauss', o)  ; sig.envelope produces one extra element than mirenvelope

Cf. previous mail, now it works.

2) insignificant differences in data values

mirspectrum -> sig.spectrum

‘NormalLength'

Yes that is negligible. We can consider the test as success.

mircepstrum -> sig.cepstrum

..., 'Frame'

..., 'Freq'

..., 'Min',  min

..., 'Max', max

mirautocor -> sig.autocor

…, 'Frame'

…, 'Min', mi

…, 'Min', mi, 's'

…, 'Min', mi, 'Hz'

…, 'Max', ma

…, 'biased'

…, 'unbiased'

…, 'coeff'

…, 'none'

…, ‘NormalWindow’, ‘hanning’

…, ‘Halfwave’

…, ‘Compress’, k

3) data different by exactly 10 times (possibly a missing/extra 0 in the code)

mirspectrum (…, 'Smooth', o)   = 1/10 the value of  sig.spectrum (…, 'Smooth', o) 

Indeed a bug: I forgot to divide by the filter length (10)

4) significant data differences

mirenvelope -> sig.envelope

…, 'Diff'

…, 'HalfwaveDiff'

…,Normal', 'AcrossSegments'

mirspectrum-> sig.spectrum

…, 'TimeSmooth', o

That was a bug also.

5) significant dimension error

mirenvelope -> sig.envelope

…, 'Smooth', o

mirspectrum-> sig.spectrum

…, ‘MinRes, r, ‘OctaveRatio, tol

That was a bug in MiningSuite, now corrected.

6)  no output produced (data dimension = 0 0)

mircepstrum -> sig.cepstrum 

…, ‘Min’, min  (miningsuite version produces no output with 0 dimension)

7) implementation error

mirsegment -> sig.segment

…, "HCDF"