MultiTracking: animals

[Vincent Prevosto]

Hi everyone,
It seems that some Bonsai users are doing multi-tracking, and I know Bonsai is also being used for tracking animal location. Is there anyone using Bonsai to track multiple animals on a video feed?
I have labmates who study social interaction in mice, and they are interested to use Bonsai for that purpose. If anyone has experience with it, could you share some tips on how to do it, and perhaps suggest typical workflow? (We've already played with the ColorSegmentation example, which is a great start).
Thanks

[Gonçalo Lopes]

Hi Vincent,

There are currently two main approaches to track multiple individuals in a video feed. The first is using different segmentation methods (e.g. can be color or just brightness contrast in the case of black and white mice) to distinguish each individual. The second one is using a marker-based approach with the ArUco library. ArUco uses the structure of known fiducials to individually recognize markers as well as recover their full 3D position and orientation in space relative to the camera. The disadvantage is of course that markers have to be used.

To use this latter approach you need to install the Bonsai ArUco package (using the package manager). Unfortunately there a couple of tricky steps in the approach, since you need to calibrate the intrinsic parameters of your camera and provide the DetectMarkers node with a file containing these parameters. In the future I will write a detailed tutorial on how to do this, but for now if you want to give it a quick try I'm attaching a calibration file for a 640x480 webcam, as well as a marker file. If you start Bonsai, feed your camera data to the DetectMarkers node, point it to the calibration parameters file and show it a physical print of the marker, you can at least get a feel for what it looks like.

There are other computer-vision based approaches to track multiple objects that I would love to integrate with Bonsai. If by any chance anyone comes across any open-source libraries that can do markerless model-based, feature-based or template-matching tracking, let me know and I will gladly take a look at them. I have my own approaches to try in the future but didn't get around to implement them yet.

Cheers,

[Niccolò Bonacchi]

Hi Vincent,

In my experience the ArUco lib is very good, however it becomes very important to use the proper yml configuration, and tweak your setup depending on the resolution of the camera, the distance to the marker and the marker size. If your yml config is not configured for your specific camera it will work but you might loose the marker if it moves fast. I ended up using simpler image segmentation processes like color (as suggested), or motion, size, shape, ROI's, etc... The cool thing is that in bonsai you can potentially use one or all of them at once and then construct a node to manage the information from all these different processes to give you an x,y position and an ID. 

Best, 

--
N

On Sat, Dec 6, 2014 at 9:31 AM, Gonçalo Lopes <goncal...@neuro.fchampalimaud.org> wrote:

Hi Vincent,

There are currently two main approaches to track multiple individuals in a video feed. The first is using different segmentation methods (e.g. can be color or just brightness contrast in the case of black and white mice) to distinguish each individual. The second one is using a marker-based approach with the ArUco library. ArUco uses the structure of known fiducials to individually recognize markers as well as recover their full 3D position and orientation in space relative to the camera. The disadvantage is of course that markers have to be used.

To use this latter approach you need to install the Bonsai ArUco package (using the package manager). Unfortunately there a couple of tricky steps in the approach, since you need to calibrate the intrinsic parameters of your camera and provide the DetectMarkers node with a file containing these parameters. In the future I will write a detailed tutorial on how to do this, but for now if you want to give it a quick try I'm attaching a calibration file for a 640x480 webcam, as well as a marker file. If you start Bonsai, feed your camera data to the DetectMarkers node, point it to the calibration parameters file and show it a physical print of the marker, you can at least get a feel for what it looks like.

There are other computer-vision based approaches to track multiple animals that I would love to integrate with Bonsai. If by any chance anyone comes across any open-source libraries that can do markerless model-based, feature-based or template-matching tracking, let me know and I will gladly take a look at them. I have my own approaches to try in the future but didn't get around to implement them yet.

Cheers,

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users...@googlegroups.com.
Visit this group at http://groups.google.com/group/bonsai-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/75582ae0-ce17-4368-ae6a-e9a2bc3b038e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.