Animal detection issues

[Calum Knight]

Hello,

My partner is doing camera trapping work for her Honours project at university and I am helping her figure out how to use Camelot for filtering out the photos without animals. She has many hundreds of thousands of photos in total, so doing this work manually is not feasible.

We have mostly worked out how to use Camelot, have been uploading photos, and watching as they are in turn uploaded in batches for animal detection. However, the majority of the folders have no animal detections at all out of ~20k photos, yet there are plenty of animals visible, and others are incorrectly detecting inanimate objects in photos (e.g. fence posts, shadows, and bait) as animals.

I have a couple of questions:

1) Does the 'Unprocessed' checkbox in the library refer to photos that have not been uploaded for animal detection, or simply photos that have not been manually catalogued by the user?

2) Are there any particular steps we should take to fix the detection issues described above? The confidence threshold is 0.8. Are there any other parameters that deal with animal detection which we could change?

Kind regards,

Calum Knight

[Juan Vargas]

Hi Calum,

The processed images are the ones that you have manually catalogued. Even when the photos have passed through the detector they are not yet processed, you still have to go through them manually. The detector does make this a lot faster.

Regarding the 'wrong' classifications, the detector is trained to find three different categories: animal/person/vehicle. When you select the 'has animal' button it shows you all these categories I think. A way to avoid this would be to use the query box, you can select only the photos that have animals using this query: suggestion-id:animal.

I don't know of any other parameters you can modify.

I hope this helps,

Juan

[Juan Vargas]

Sorry, the query would actually be "suggestion-key:animal", my mistake.

[Chris Mann]

Hi Calum,

Further to Juan's response, you might find 0.8 too low as a confidence threshold.  By default this is 0.9, which is still quite lenient.

It can take some time for all images to be uploaded and processed and you can track the status of this.  You can use the search "media-detection-completed" to determine how many images in your dataset have actually passed through the animal detection, regardless of whether or not they have animals identified by it.

-Chris

--
You received this message because you are subscribed to the Google Groups "Camelot Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camelot-proje...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/camelot-project/b908b138-7196-43ee-a939-2d70f7063ecbn%40googlegroups.com.

[Calum Knight]

Hi Chris and Juan,

Thank you for your help!

We changed the confidence threshold back to 0.9, so hopefully should see some better results from the detection.

However, when we perform the search 'media-detection-completed', it says that none of our photos have passed through the animal detection, even though many have come back with animals detected in them. Do you know why this might be the case? We did just change the confidence interval and restart the software before doing this, so don't know what it would have said before we changed anything.

Furthermore, changing the keyword from 'animal' to 'person' or 'vehicle' results in no photos coming up in the search, and when the keyword is 'animal', every detection made is returned, including inanimate objects such as bait and fence posts. We can deal with false positives, as sifting through the subset of photos with detections made is still much faster than sifting through all the photos for animals. However, I am wondering, how frequently does the detection software return a false negative, i.e. fail to detect an animal in a photo when there is one present? We have seen sheep and people in photos that ought to have been run through the detection software by now, yet when we tick 'Has animal' on that camera, the filter does not return any photos for that camera.

Kind regards,

Calum Knight

[Chris Mann]

Hi Calum,

My apologies - the correct search should be: "media-detection-completed:true"

On the topic of false negatives, you may be interested in an ongoing analysis of the model we have here: http://monitoring.camelotproject.org/d/tlRZ5jZGk/batch_processing-model-analysis?orgId=1&refresh=1m&from=now-7d&to=now.  That is, with the data we have the recall for the 0.9 confidence threshold is around 80% (i.e., there's a 20% false-negative rate), while the false positives are quite low generally (though keeping in mind the data here is skewed based on the images processed).

-Chris

To view this discussion on the web, visit https://groups.google.com/d/msgid/camelot-project/41f8b13a-3ad1-459b-9f0f-6667e068116cn%40googlegroups.com.