How to ues clearROCHistory

[吴宗泽]

When I click the clearROCHistory button, no images or records appear. Also ask how to view AUC as a metric in jAER?

[Tobi Delbruck (INI)]

Dear ??? (not sure who you are), I'm currently intensely working on NoiseTesterFilter in the jaer master branch. You are running the version from the DND21 paper (https://sites.google.com/view/dnd21) that needs the python scripts to do a ROC sweep.

The ClearROCHistory button just clears out the samples that have been collected. The max sample length is set by rocHistoryLength.

Tobi

[吴宗泽]

Dear Prof. tobi, thank you for your reply, I am an academic who is working on event camera related research, my name is Zongze Wu.
I tried to scan the ROC curve in the data hotel-bar.aedat using the python script provided in the paper, but the actual results I got were not satisfactory, as shown in the two graphs below. Are there any key parameters that need to be adjusted?

Currently my goal is to compare the goodness of various denoising models through the AUC metric, and I understand that you are refining and simplifying this work in jAER, thank you again for your great contribution, and may I ask how much longer it will take for this feature to be publicly available?

Tobi Delbruck (INI) <to...@ini.uzh.ch> 于2025年2月26日周三 14:38写道：

Dear ??? (not sure who you are), I'm currently intensely working on NoiseTesterFilter in the jaer master branch. You are running the version from the DND21 paper (https://sites.google.com/view/dnd21) that needs the python scripts to do a ROC sweep.

The ClearROCHistory button just clears out the samples that have been collected. The max sample length is set by rocHistoryLength.

Tobi

--
You received this message because you are subscribed to the Google Groups "jaer-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jaer-users+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/jaer-users/eca6622d-7443-449b-851c-c1d4cf706bb6%40ini.uzh.ch.

[Tobi Delbruck INI]

Hi Zongze, if you fork jAER and pull the latest changes from master then
you can use the latest code.

I just updated netbeans to netbeans 25.  The most painless way is to
install the coderity package of JDK+netbeans

https://www.codelerity.com/netbeans/

After installing netbeans, you can open your fork by just opening the
project in netbeans.

The first time you build, it will be slow and might fail because it has
to download lots of libraries. But it should work.

About using the python scripts I don't know. I never used them myself.
It was Shasha who did all that stuff. Probably something changed in the
control of the parameters by the jAER RemoteControl. Are you controlling
the swept parameter correctly? It looks like all the ROC points have the
same signal/noise discrimination threshold in your sweep.

I've been using jAER to compare ROC curves the last few days within the
new NoiseTesterFilter

Tobi

[Tobi Delbruck INI]

Thanks for sharing that stuff. Clearly you are not sweeping the discimination threshold. Is there logging output that is informative? Does your method implement the threshold control function as a RemoteControl outside NoiseTesterFilter, so that the python script can control it?

I'll try to record a video of using NoiseTesterFilter directly to sweep ROC in next few days.

Tobi

-- 
Prof. Tobi Delbruck https://sensors.ini.ch
www.linkedin.com/in/tobi-delbruck-b91257259, https://www.youtube.com/channel/UC70-pVoJ4pwIAxqpgcR7oxg
Swiss office # +41 44 635 3038 Swiss cell +41 76 629 1500 (whatsapp), USA cell +1 626 510 2646, skype tobidelbruck

[吴宗泽]

Thank you very much for your reply.
First of all, I tried to update my jAER from 2.6.3 to 2.7.1 in “https://www.dropbox.com/scl/fo/ibqmrztay51g7fg5d7mu3/h?e=1&dl=0”, but after the update the following error occurs when reading the read data

 Other than that, following your instructions I downloaded the netbeans and running the latest branch  of jAER，I have no experience with netbeans and JAVA in the past, I am trying to learn, when I opened the project in netbeans and ran it, according to the prompts he did download a lot of libraries, but still the error was reported, as in the picture below，maybe these errors are due to my unfamiliarity with this ide.    If it runs smoothly, does it compile the same as running the exe file downloaded from Dropbox above?



I have another question about the details of jAER.
When I enter an event and add a 1HZ shot noise via 'Noise TesterFiliter', and then export it as a txt file1(below left) with 'Davis TextOutputWriter' . At the same time I add a filter (BAF/Density Filiter...) for the event where the 1HZ noise was originally added , then exported as a txt file2 (below right).

I found a strange phenomenon, theoretically every event point in file 2 can be found in file 1 (I marked the corresponding points in both files with red boxes), but for example the unmarked event point in file 2 [1606.5834 185 200 0],this point seems to be generated by the filter. When the noise level is raised to 7HZ/10HZ, this phenomenon is even more pronounced, and there are even more points that cannot be corresponded to than the number of corresponding points. I'm curious as to why these extra points are created and what category these points should be categorized into when calculating ROC (TP,TN,FP,FN)? I tested the following five filters and only DWF and MLPF do not produce new points


Thanks again for your patience and best wishes!

Zongze Wu

Tobi Delbruck INI <to...@ini.uzh.ch> 于2025年3月3日周一 15:03写道：

[Tobi Delbruck (INI)]

Hi Zongze, don't use that 2.7.1 installer version, it is buggy wrt
NoiseTesterFilter and that is why not announced yet as a release... 
Once I think things are stable on my end and in my hands I announce a
"pre-release". I don't think I have done that for 2.7.1.

The best way is just to pull changes from master to your fork and build
from there. I just pushed my own experiments with
LinearCorrelationDenoiser so you can build with that if you want (that's
the error below in your build; the rest of the messages are just
warnings about deprecated java web applet classes). I also pushed a
massive number of changes to NoiseTesterFilter, FilterPanel, etc. Check
the commit log.

You can always comment out denoisers from the static list of
noiseTesterFilters at the start of NoiseTesterFilter (line 113
currently) if they are not available for the compile.

About your question "When I enter an event and add a 1HZ shot noise via

'Noise TesterFiliter', and then export it as a txt file1(below left)
with 'Davis TextOutputWriter' . At the same time I add a filter
(BAF/Density Filiter...) for the event where the 1HZ noise was
originally added , then exported as a txt file2 (below right)."

 I don't understand what you are doing exactly. What do you mean by
"when I enter an event"?  And what do you mean by "the unmarked event

point in file 2 [1606.5834 185 200 0],this point seems to be generated

by the filter"? By point, do you mean DVS event?

I don't understand what you are doing here, sorry.

Tobi

[吴宗泽]

Thanks to Prof. Tobi, “when i enter an event” means that I am using jAER to read a new event file (e.g. driving.aedat). My purpose is to add a 1HZ noise to driving.aedat and output a txt file1, and then add a filter to driving.aedat with 1HZ noise and output a txt file2. By this way I can get the event data before and after the noise event passes through the filter, and I can calculate the relevant metrics to evaluate how good or bad the filter is by utilizing the two txt data.

My problem is that after the noise event passes through the filter, I find that there are points in the data that don't belong to the original noise event(e.g.   [1606.5834 185 200 0]  ), and I can't tell how this extra point was created and what type of point it should be categorized as (TP,TN,FP,FN).

I apologize for confusing you with my inappropriate expression and thank you for your patience. I will keep you posted and look forward to the “pre-release”!

Tobi Delbruck (INI) <to...@ini.uzh.ch> 于2025年3月8日周六 20:05写道：

[Tobi Delbruck (INI)]

You mean you are adding the labeled noise events yourself by creating with your python (or some other language) code that augments driving.aedat with your own 1Hz shot noise (not noise simulated by NoiseTesterFilter)? Your noise is labeled as noise by the "special event" bit?

Did you turn off all the noise generated by NoiseTesterFilter (leakNoiseRateHz=0 and shotNoiseRateHz=0)? If not, NTF will add noise also.

[吴宗泽]

Dear  Prof. Tobi,
     I'm really very sorry that I still haven't expressed myself correctly, and I'll detail my steps and purpose below.

1.Use jAER to read hotel-bar-segment.aedat, then use 'Noise TesterFiliter' to add 1hz noise (Fig. 1), then use 'StartRecordingAnd Save As' button (Fig. 2) in 'Davis TextOutputWriter' to export the txt file. 1 (Fig. 3), which represents the event with 1hz noise.
，，

2.Add an additional 'BAF' to 1（Fig. 4） and leave the rest of the parameters unchanged.and the same again using TextOutputWriter output for txt file. 2(Fig .5),which represents  that the event with 1HZ noise after the filter to get the result.
, 

3.Iterate through each line of file 1, if the point in this line can be found in file 2, this indicates that BAF considers this point to be a real signal (keep it); if the point in this line is not found in file 2, this indicates that BAF considers this point to be noise (been removed)
   This operation makes it easy for me to obtain the prediction labels，But a peculiar situation has arisen in this process, as I mentioned in my previous reply.
   Since the filter is only a removal of some of the points and not a change, theoretically every point on every line in file.2 should be found in file.1. But in the output, I find that file.2 contains points that do not exist within file.1 , such as [ 1606.5834 185 200 0  ] in Fig. 6 （The rest of the rows in file.2 that are framed by rectangular boxes can be found in file.1）



    I'm not sure what causes this condition, so I'm asking you about it. by the way ,I found that in jAER 2.6.3, when I check 'MLPNoiseFiliter' in 'NoiseTesterFiliter' I get the following error(Fig .7), which is incidental, but has been happening a lot lately.Is there a good way to avoid it?


   Your guidance has helped me a lot, best wishes to you!

   Wu Zongze

Tobi Delbruck (INI) <to...@ini.uzh.ch> 于2025年3月9日周日 19:44写道：

[Tobi Delbruck (INI)]

Thanks for explaining, it's clear now what you are doing.

Every noise sample is random. If you rewind, the noise will be different the next time. Or do what you are doing. It is no surprise the noise events differ. But the signal events (the hotel bar events) should be identical between recordings.

If you want to make the noise exactly the same each time it will be difficult, not just I think making sure the random seed is reset to the same value. That is because generating shot noise depends also on the time resolution. Please check code for details.

One thing you can try is to use the prerecorded noise in the DND21 AEDAT folder. In particular the 5Hz shot noise recording. You can load this noise recording using the "OpenNoiseSourceRecording" button in NoiseTesterFilter.

Let me know how this goes.

[Tobi Delbruck]

Please pull changes, I pushed a lot more improvements to doing ROC sweeps in NTF. And I fixed a bug in populating the TPI (timestamp / polarity images) after reset with correct distribution of noise event times. And now ROC sweeps only collect statistics *after* reset + correlationTimeS, so that the statistics are correct for denoising, i.e. so that denoising has past noise *and* signal events on the first sample that make sense.

[吴宗泽]

Dear  Prof. Tobi ,  Thank you very much for your guidance, it solved my problem perfectly through your method!

In the meantime, I updated jAER to the latest branch, may I ask if he supports direct output of AUC values after denoising each filter, and what are the steps to realize it?

[Tobi Delbruck (gmail)]

Thanks for your feedback!  So far only a CSV of ROC values (TPR and FPR) is output  by using the SaveROCSweepsToCSV button. One CSV file is generated per recorded sweep.

Do you think it is useful to add the computed AUC value to the CSV header comment?

(CSVs are only suitable for tabular data. Special values like AUC don't really fit to CSV format.)

BTW, I believe there is some kind of memory leak currently in NTF, because the processing slows down to a crawl after recording some ROC sweeps. I'm trying to diagnose this now.

[吴宗泽]

I think it's useful. My goal with jAER is to get quick AUC values to evaluate the various filters in jAER

To share with you my doubts in using it:

1.When my input is a txt file, I add filters using NTF ,  then output using 'StartRecordingAnd Save As' button in 'Davis TextOutputWriter'. If I don't uncheck 'closeOnRewind'（fig 1）, the output txt file is empty.The reason for this is that the first scan only reads 1 event (I don't know why), and only when using a loop scan, the second scan will get the full data


2. Due to the problem in 1, I unchecked 'closeOnRewind' to get the denoised data I wanted. However, I found that the first few event points of the data obtained in this way are not denoised (hotel_bar_segment is not denoised for roughly the first 16,000 events, and driving is not denoised for roughly the first 21,000 points), and this is now also visible in the visualization from jAER, which shows that when a segment of the stream of events plays out and starts to loop through the stream for a second time, there's a frame in between The image is not denoised, and everything is normal after that. I think the reason for this is the wrong sequence of plugins in the loop or a delay in filter import.

3.The NTF filter of the latest branch is not available.

Tobi Delbruck (gmail) <tobide...@gmail.com> 于2025年3月18日周二 19:53写道：

--

You received this message because you are subscribed to the Google Groups "jaer-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jaer-users+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/jaer-users/50eb9a2a-dee2-44b3-8def-bc4b8cadd922%40gmail.com.

[Tobi Delbruck (gmail)]

I believe that currently NoiseTesterFilter has a serious memory leak that only appears when doing ROC sweeps. I don't understand the cause so far, but the symptom is that after doing a ROC sweep or a couple of them, everything starts out fine, but processing suddenly slows down by a factor of 100X after some time. I am trying to debug this now. It may be that ROC sweeps hold a reference to the denoiser instance which prevents garbage collection.  It is quite strange because it happens well under the 10GB  java JVM memory limit.

If you want to record denoised output, I suggest you use the File option "Enable filtering of logged or network output events". See screenshot below.

then pause the playback, hit "r" for rewind, and "l" for "start logging". The file will then close automatically at the of the recording or at the OUT marker. You can then save this filtered AEDAT file and play that back for outputting CSV with DavisTextOutputWriter.

I don't know why the first packet is not filtered for you now. The enabled (checkbox selected) filters run from top to bottom. But I never recorded the output this way with filtered data. I think DavisTextOutputWriter should not output events that are filtered out. The event iterators should skip these events but perhaps you could check it with debugger. Here is the code that writes out the events

[Tobi Delbruck (INI)]

In latest commit to master branch (and also latest pre-release 2.7.0), I think I fixed the NoiseTesterFilter memory leak problem and did a lot more cleanup of NoiseTesterFilter. NTF looks like this now:

Let me know if that works for you.

I also improved the CSV export. A CSV file from ROC sweep looks like this now and includes the computed AUC in header

# ROC sweep of parameter correlationTimeS for filter SpatioTemporalCorrelationFilter
# created Thu Mar 20 07:14:05 CET 2025
# NoiseTesterFilter noise: NTF: Synthetic noise: CoV +1.0  dec, Leak +100.0mHz+/-+200.0m jitter, Shot +5.0 Hz. Overlay 
# source-file: H:\My Drive\INI - Sensors group\Datasets\DND21 - DVS BA denoising\AEDAT\driving\driving.aedat
# Marks: IN=1,189,200 OUT=1,892,192
# Computed ROC Sweep AUC=0.8811306
correlationTimeS, FPR, TPR
0.000917,0.055451,0.466964
0.001261,0.079208,0.555988
0.001735,0.106652,0.658437
0.002386,0.147999,0.736841
0.003283,0.187985,0.809207
0.004515,0.236310,0.867904
0.006211,0.290521,0.909116
0.008543,0.359963,0.938834
0.011751,0.437765,0.959104
0.016163,0.518802,0.972788

[Tobi Delbruck (gmail)]

I made some major improvement in the speed of NoiseTesterFilter in the branch "ntf-optimized". 

NTF uses new event and packet classes SignalNoiseEvent and SignalNoisePacket so that it tracks the labels (signal/noise) and classifications during denoising. Now there is no need for the expensive sorting and matching. It runs in real time now on even dense recordings.

[吴宗泽]

Dear  Prof. Tobi ,  Thank you very much for your great contribution. I will continue to use the updated jAER in the near future to keep following up with you with feedback!

Tobi Delbruck (gmail) <tobide...@gmail.com> 于2025年3月24日周一 16:52写道：

I made some major improvement in the speed of NoiseTesterFilter in the branch "ntf-optimized". 

NTF uses new event and packet classes SignalNoiseEvent and SignalNoisePacket so that it tracks the labels (signal/noise) and classifications during denoising. Now there is no need for the expensive sorting and matching. It runs in real time now on even dense recordings.

--
You received this message because you are subscribed to the Google Groups "jaer-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jaer-users+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/jaer-users/3d04e42d-1b1d-4ede-a821-11d31cb9e41a%40gmail.com.

[吴宗泽]

Dear  Prof. Tobi ,
  I would like to ask jAER if it has the function to process all the data in the whole folder at once,
  I would like to read my own data for denoising by using the filter in AER,
  but at present I can only read the individual data in the folder one by one to process it, even if the data is large in number.

吴宗泽 <zongz...@gmail.com> 于2025年3月29日周六 22:09写道：

[Tobi Delbruck (gmail)]

No way to do this except by scripting external python script (for
example) to send remote control commands. Of course you could just
concatenate your sequences to one file.