batch detections

[Mike Pitzrick]

Howdy,

I noticed a couple of days ago that if I right-click on a sound in the XBAT Palette, that I can run batch detections on one or more sounds. Is this feature ready to use?

The reason I ask is that my first impression is that detections run with FFT size 512 are running in about 1/2 as much time compared to detections run using the green Data Template.

On the other hand, running detections with FFT size 1024 seems to run at the same speed using the two methods. I wonder if running detections from the XBAT Palette sets the FFT size to 1024 regardless of what the FFT size selected on the Spectrogram Palette.

I'll let you know as more evidence accumulates.

-Mike

--
I believe that Ronald Reagan can make this country what it once was - an Arctic region covered with ice.    --Steve Martin

[Mike Pitzrick]

PS - Assuming that detections initiated from the XBAT Palette can be made to work, can I determine the name the resulting log?

Also, it appears that I may have the option of controlling the name of each event in the log in the new Detect Palette. Can I change the default "%DETECTOR_NAME%_%PRESET_NAME%_%TIME%"? If so, are there other variable names to choose from (eg - %SOUND_NAME%) would be handy)?

[Matt Robbins]

Hey Mike,

   The template detector always pulls the frequency resolution from the browser spectrogram settings.  I'm not actually sure how you know what FFT size the batch detector is using without opening the sound and looking at the spectrogram settings, because the only thing that you should be able to do is select a preset to detect with.  There should not really be any performance difference, so I'll look into that.

   Anyway,  last time I checked, the batch detection does work, and %SOUND_NAME% is also correctly interpreted in the generated log name...  You can also just specify a log name directly if you want to.  The reason the the sound name was not included by default is that the computed log name will include the sound name anyway, so you will see something that looks redundant in the 'Logs' info listbox.

Thanks,
Matt

[Mike Pitzrick]

Hi Matt,

I've confirmed that when I run IBWO vocal detections with identical spectrogram settings (FFT Size: 512, Advance: 0.25, Type: Hann, Length: 1), the results are not the same when the detections are run through the XBAT Palette compared to detections run through the Data Template. The detections run through the XBAT Palette run in about half the time, and result in fewer events than detections run through the Data Template. Based on past results using XBAT_SVN_1157, the run time for detections run through the Data Template are about what I'd expect, and detections run through the XBAT Palette take about half the expected time.

With both types of detections, I confirmed settings in the Spectrogram Palette before and after the detection. In the case of detections run through the XBAT Palette, that entailed:

1. opening the appropriate XBAT Sound window
2. entering the settings in the Spectrogram Palette
3. closing the XBAT Sound window
4. initiating the detection by right-clicking on the sound stream in the XBAT Palette, selecting "Action"-"Detect", and in the Detect Palette selecting "Data Template", the appropriate preset, and "OK".
5. after the detection, opening the appropriate XBAT Sound Window and checking the settings in the Spectrogram Palette.

I have also confirmed that IBWO double knock detections, which are run at different spectrogram settings (FFT Size: 1024, Advance: 0.25, Type: Kaiser, Parameter: 20, Length: 1) take the same amount of time to run from the XBAT Palette and the Data Template. In 2 or 3 hours I will know whether or not the resulting logs are identical.

-Mike

[Mike Pitzrick]

Hi Matt,

I've re-run 2 IBWO double knock detections using the data template instead of the XBAT palette. As expected the run times were about the same, but the resulting logs were not identical. The logs resulting from detections run using the Data Template were 10% larger in both cases. I examined logs from one soundstream and found that running the detection through the Data Template identified 6 events not found through the XBAT Palette method, and the XBAT Palette method found 3 events not found by the Data Template method. One event was found by both methods but by different templates in the preset.

This would suggest that if XBAT is not noticing the FFT size I set  when running detections through the XBAT Palette, as I suspect, that there is something else going on. Perhaps XBAT is not noticing one or more of the other spectrogram parameters I have set.

-Mike

[Matt Robbins]

Hi Mike,

   I looked at this a little harder, and you're basically right:  the state of the sound is not being stored correctly and is only loaded again when the sound is opened in a browser.  The spectrogram settings are being reverted to their defaults.

   We are quite busy working on other things at the moment, but one of us ought to be able to spend some time fixing this (it shouldn't be that hard).

-Matt

[Mike Pitzrick]

Hi Matt,

Cool. Today I tried running several detections from the XBAT Palette with the sound open to see if it would remember the spectrogram settings. The predicted detection times looked right at the beginning of the detection, but in no case did I get a usable log.

-Mike

[Matt Robbins]

Mike,
 

The predicted detection times looked right at the beginning of the detection, but in no case did I get a usable log.

What do you mean by 'usable' log?
 

-Matt

[Mike Pitzrick]

Hi Matt,

I'm sorry for being vague!

By 'usable', I mean capable of being opened with events appearing on the event palette. Some of the  logs reported having events in them when I queried the log using the "Logs" part of the XBAT Palette, but when I tried to open them using the Log Palette, no events appeared on the Events Palette.

In a couple of cases I noticed that after attempting to open an unusable log, the "Logs" part of the XBAT Palette started showing 0 rather than the previously reported number of events.

I hope that helps!

-Mike

--
Hope has two beautiful daughters. Their names are anger and courage; anger at the way things are, and courage to see that they do not remain the way they are.    --St. Augustine

[Matt Robbins]

Mike,

   Sorry for the inconvenience.  I'll come by to see what's going on today.

-Matt

[Mike Pitzrick]

Hi Matt,

I should be here until at least 4:30 pm. I set up a fresh detection so you could examine the log before I attempted to open it.

Additional symptoms:

Detection starts out with predicted run time around 2 hours, as expected. A few minutes into the detection it speeds up considerably to 60x, and predicts about 23 minutes left in the detection for extended periods. Then it may revert back to a more sedate 13x. No MATLAB errors or warnings.

I'm not sure if it's relevant, but the soundstreams I'm working on seem defective in some sections. At times a see a high frequency of brief (on the order of milliseconds long) blanks in the recording, at the rate of several per second.

-Mike

[Ann Warde]

Hi Mike,

Is DT batch detection working now?

Thanks --

Ann

On Feb 27, 11:57 am, "Mike Pitzrick" <mpitzr...@gmail.com> wrote:
> Hi Matt,
>
> I should be here until at least 4:30 pm. I set up a fresh detection so you
> could examine the log before I attempted to open it.
>
> Additional symptoms:
>
> Detection starts out with predicted run time around 2 hours, as expected. A
> few minutes into the detection it speeds up considerably to 60x, and
> predicts about 23 minutes left in the detection for extended periods. Then
> it may revert back to a more sedate 13x. No MATLAB errors or warnings.
>
> I'm not sure if it's relevant, but the soundstreams I'm working on seem
> defective in some sections. At times a see a high frequency of brief (on the
> order of milliseconds long) blanks in the recording, at the rate of several
> per second.
>
> -Mike
>

> On 2/27/07, Matt Robbins <matthew.robb...@gmail.com> wrote:
>
>
>
>
>
> > Mike,
>
> > Sorry for the inconvenience. I'll come by to see what's going on
> > today.
>
> > -Matt
>

> > On 2/26/07, Mike Pitzrick < mpitzr...@gmail.com> wrote:
>
> > > Hi Matt,
>
> > > I'm sorry for being vague!
>
> > > By 'usable', I mean capable of being opened with events appearing on the
> > > event palette. Some of the logs reported having events in them when I
> > > queried the log using the "Logs" part of the XBAT Palette, but when I tried
> > > to open them using the Log Palette, no events appeared on the Events
> > > Palette.
>
> > > In a couple of cases I noticed that after attempting to open an unusable
> > > log, the "Logs" part of the XBAT Palette started showing 0 rather than the
> > > previously reported number of events.
>
> > > I hope that helps!
>
> > > -Mike
>

[Mike Pitzrick]

Hi Ann,

I don't think so. Though I haven't checked for a couple of days now.

-Mike

[Harold Figueroa]

Hello,

You might want to check. Last I heard from Matt, was that he had found the problem. It was related to the sound being scanned (and its corresponding spectrogram state) not being loaded at the right time from the file. It think it may be resolved, or at least deserves a round of testing.

Harold

[Mike Pitzrick]

Hi Guys!

On 27 March I updated all my XBAT installations and ran some tests on XBAT batch detections.

For preset "dk_f" and spectrogram parameters (1024, 0.25, Kaiser, 20, 1) I got 97 events using the green Data Template and 1 event using batch detect for one sound stream. Using a second sound stream and similar parameters I got 31 events with the green Data Palette and 0 events with batch detection.

For preset "Kent_2007_CORE" and spectrogram parameters (512, 0.25, Hann, 1) I got 491 events with the green Data Palette and 130 events using batch detection for one sound stream, and 374 vs 12 events for a second sound stream. The batch detections only took half as long for the green Data Palette detections for these spectrogram parameters (I forgot to check run times for preset "dk_f").

I hope you find this useful!

BTW - I'm confused. Aren't we supposed to be using "XBAT-users" Google group for this kind of discussion?

-Mike

--
"We, the people are the rightful masters of both Congress and the courts, not to overthrow the Constitution, but to overthrow men who pervert the Constitution."     --Abraham Lincoln

[Matt Robbins]

Thanks for the info, Mike.

   I just ran some tests on a file stream of black-capped vireo data.  I used essentially the same parameters (non-default) that you did:  spectrogram (256, 0.25, Kaiser, 20, 1) and did a detector scan using both methods.   Both the number of events and the start times and durations of the events are identical between the logs.  My current hypothesis as to what is going on concerns the time-stamps that you are using and I am not. 

   I might suggest (as a temporary workaround) that you remove or disable the time-stamp attribute for the sounds while running the batch detections.  I will have to do some more testing to be sure, but it seems like  it's pretty likely that there is something funny going on regarding time-stamps.  The internal mechanism for both types of detections is identical, it's the same piece of code, so it's hard to see what else could really be going wrong.

Thanks,  I'll let you know when I know more.

-Matt

Also, I've copied this thread over to the users group.  I think (as you suggest) that that would probably be a better place...

[Harold Figueroa]

Just when you thought the time-stamps were dead once and for all they come back to kick our ass.

Scotland is OK, it's old, the old buildings and castles are pretty cool. It's kind of expensive and until today (when the conference started) I really did not get to talk to too many people ... I also had preparing some slides hanging over my head ... Not much of an update but at the university where I am staying they are charging me 10L for 3hrs of access.

Maybe I will try to storm the castle and have fun while I am doing it.

See you soon,

Harold

[Matt Robbins]

I think this problem actually has been fixed in the branch, I just patched it in the trunk as you will notice from my message to the group.