Would Fish AI community benefits from an improved Mega Fish Detector?

[Asta Audzi]

Ideas from Dan Morris, posted here for easier discussion. You can reply to here or email Dan directly 

• MegaDetector is a general-purpose animal detection model we built for camera traps (finds animals and people in camera trap images, doesn't do any species classification) (it doesn't know about fish).
• MegaFishDetector is a model built by Woods Hole that aims to do something similar for fish (finds fish in images, doesn't do any species classification).  I haven't tried this model.  They appear to know what they're doing, but I don't have any evidence that this is better or worse than other models.

--

There are a few questions I'd like to help the community answer:

1. Would a general-purpose fish detector be useful?  The idea is that such a detector could (a) eliminate empty frames from videos and (b) make it easier to train and run species classifiers (by cropping individual fish out of a busy scene).  But I don't know this world as well as I know the camera trap world, so I don't actually know whether this is a useful concept for fish.
2. If a general-purpose fish detector is useful in principle, has someone already built one that works well enough to be useful for a variety of ecosystems?  The only two models I know of are MegaFishDetector and KakaduFishAI (which Andrew talked about at the meeting).  I have not tried either.
3. If the idea is useful but existing models don't do the job, how do we get the right model trained?

--

I'm not super-well-equipped to answer question (1), since I'm pretty new to this area.  I welcome input from the community on this.

The next steps I would propose to answer questions (2) and (3) are:

1. Writing reasonably consistent sample code to work with all the available public datasets (e.g. from this list).  That's basically making this page (which is about drone/aerial images) for fish.  This is probably a day or two of work.  I may or may not find time to do this myself, but I would be happy to advise and to be involved.
2. Trying both of the available models (and any others that I'm missing) on all of those datasets (with the caveat that many of those datasets were used to train MegaFishDetector).

I think if we do those two things, we'll have a good sense for where we're at, i.e. how close we are to a generalizable fish detector.  This could be a nice paper for a motivated student who is pretty well-versed in Python; if you know a student that fits the bill, I'm happy to guide them along.  Also once we've done (1), it's almost trivial to train new models on all the existing data.

 

[th...@aeria.ai]

Thanks Dan for instigating the idea to work towards a general purpose fish detector. I think such detector will be useful in variety of contexts. Like the camera trap MegaDetector it may take a little before practitioners 1) start to get used to the idea to use models for image analysis and 2) adopt using a particular general model instead of build many specific ones. 

Hopefully the experience from MegaDetector can help to speed this up in the fish world. (Or who knows I completely miss a huge ML fish movement....).

I am interested to help push this forward and learn more about how this general models work (but not a well-versed Python coder).

Cheers,

thor

[Luyen Chou]

Google just introduced a new version of Bard with multi-modal input (image). Whoa:

Gonna play more, but pretty impressive!

--
You received this message because you are subscribed to the Google Groups "AI for fish" group.
To unsubscribe from this group and stop receiving emails from it, send an email to AI_for_fish...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/AI_for_fish/8dff7acb-3149-4836-be50-acaa630e510fn%40googlegroups.com.

[Jimmy Freese]

Very interesting. I like how it also gives the other possible species. 

Please keep sharing. I'm curious to see it try with images with poor lighting, funny angle, water on the lens and occlusion and how well it performs. 

Jimmy

To view this discussion on the web visit https://groups.google.com/d/msgid/AI_for_fish/CAAtFm_DYfPw%2BKFHsmzg0WaQvtk4yqzg%3DL9mhF%2BZQ-03j7wcAhw%40mail.gmail.com.

[Luyen Chou]

A few more experiments. 

What’s interesting is that it’s been trained not to process photos with people in it (for obvious reasons), but this limits it’s effectiveness for rec angling purposes. That said, very impressive out of the box, and probably could be the basis for further training and specific applications. 

Here it pushes back on picture with person:

Here it gets it wrong…

… but learns from its mistake:

And here it reveals its limitations trying to gauge size based on references:

[Jimmy Freese]

Thanks for that Luyen. Interesting. I had heard that it's not ready to handle people yet. 

Good you threw it a couple of curve balls.