Speed for Segmenting Video step

[Julien Foucaud]

Hello all,

First, thanks to the author for this new version of idtrackerai, it is much improved since last time I used it (2021).

I have a simple question : I find the 'segmenting video' step very long (I have thousands of small_sized video to process) and in the version I last used, speed was very much improved at this step with the parameters SAVE_PIXELS = 'RAM' and SAVE_SEGMENTATION_IMAGE = 'RAM' in my (then) local_settings.py file

I coudn't find the same parameters in your documentation or example .toml files

Is that still possible to enforce in the latest release ?

Thanks a lot, and once again, great job.

Julien Foucaud

[idtrackerai]

Hi Julien,

I'm glad you find the new version more useful than previous ones.

We removed the  "SAVE_PIXELS" parameter since we no longer work with blob's pixels (area) but their contours (perimeter). I can't imagine a tracking setup where this doesn't bring an enormous speedup.

And regarding "SAVE_SEGMENTATION_IMAGE", we removed this tunable parameter hardcoding it to "DISK". Reducing RAM usage and simplify usage was a priority and we didn't see any downside apart from small time penalties (maybe big penalties in your setups?).

If you encounter big slowdowns compared to previous versions, I would like to take a look at your idtrackerai's logs. Or an example of the videos you work on. I would be open to reconsider reimplementing "SAVE_SEGMENTATION_IMAGE" if it's worth it.

Jordi

El dl., 19 de febr. 2024, 14:50, Julien Foucaud <julien....@gmail.com> va escriure:

--
You received this message because you are subscribed to the Google Groups "idtracker.ai users group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to idtrackerai_us...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/idtrackerai_users/9cc77653-c14d-40a4-9b46-6a81f39e4731n%40googlegroups.com.

[Julien Foucaud]

Hi Jordi,

Thanks for your answers. It seems I'm in a very peculiar case where this means a lot to be able to have the segmentation image in the RAM.

I have run a test, consisting of a batch of 288 small-size videos in each of which only one animal needs to be tracked (see the attached log file for 1 of these videos) : this takes 36h hours overall and mainly because of disk access during this segmentation step. It means around 7.5 min per video in average, unlike the ~2 min per video with the older parameters. From the log file I understand it would probably be back to 2min per video with a reasonable segmentation step.

The thing is I need to process 24 batches like this every 3 weeks for 6 months, so this amount of speed up would mean I'd be able not to lag behind my data accumulation.

Thanks a lot for your efforts in idtrackerai,

Julien

[idtrackerai]

For the records, this issue has been fixed by adding the parameter BOUNDING_BOX_IMAGES_IN_RAM (check documentation).