Fitting a circle - dealing with occlusion

[Nikolas Karalis]

I am trying to use Bonsai for tracking the size of the pupil of the eye.
So far, I am thresholding and detecting the black pupil in a grey/light background and using the LargestBinary Region the diameter and other parameters are calculated by the ellipse that is fit.

However, the problem is that there are some reflections that are constantly on parts of the video (bright white). When the pupil overlaps with the reflection, the detected shape is not circular any more but rather crescent-shaped (see screenshots). This throws the ellipse fitting off so the resulting calculated diameter is wrong.

The way that I thought this problem could be solved would be to use the prior knowledge that the shape has to be circular so instead of fitting an ellipse, try to fit a circle, under some minimum size constraints.

Is this implemented in Bonsai? Any suggestions on how to proceed?

Thanks in advance!

Best,
Nikolas

Typical image


Occluded image

[Nikolas Karalis]

From what I see, using the OpenCV function minEnclosingCircle would do the job.
However I am not sure how I can access this function inside a Python Transform node (Bonsai keeps complaining about not finding it).

[Gonçalo Lopes]

Hi Nikolas,

That is indeed a cool use of MinEnclosingCircle, it works really well on the image you provided. There is a trick to using it in IronPython because of the multiple return parameters, but something like this works:

import clr
clr.AddReference("OpenCV.Net")
from OpenCV.Net import *
from System import Tuple

@returns(Tuple[bool,Point2f,float])
def process(value):
  ret,center,radius = CV.MinEnclosingCircle(value.Contour)
  return Tuple.Create(ret,center,float(radius))

The returned parameters are:

  - true/false indicating whether all points are contained in the circle or not

  - the center of the circle

  - the radius

Thanks, I may consider adding a node with this to the new version coming out, since it can indeed be very useful.

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/bonsai-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/85a59556-c917-4a42-80c2-3e4741d77ec8%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Nikolas Karalis]

Hi Gonzalo,
thanks a lot for the help.
Indeed the multiple outputs were the problem.

For overlaying the output to the video, from what I understand I need to use a similar approach as here: link
Do I need to use the tuple to read the multiple items of the output?

Best,
Nikolas

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users...@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

[Gonçalo Lopes]

Yes, something like that would work.

You probably need to Zip the result of MinEnclosingCircle with the image and then run the PythonTransform. Then you would have something like the following to extract all relevant inputs (the order of the first tuple may be swapped depending on the order you Zip):

image = value.Item1

center = value.Item2.Item2

radius = value.Item2.Item3

A bit awkward, but should work.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/5d199b82-2e03-4cb2-b57d-db43a82be824%40googlegroups.com.

[Nikolas Karalis]

Thanks, as before, the problem was the tuples (and how to extract the items).

For anybody who might stumble upon this thread, to plot a circle on the video, the Python code is:

import clr
clr.AddReference("OpenCV.Net")
from OpenCV.Net import *

color = Scalar.Rgb(255,255,255) # White

@returns(IplImage)
def process(value):
 
  center = value.Item1.Item2
  radius = value.Item1.Item3
  image = value.Item2

  output = image.Clone()
  CV.Circle(output,Point(center),radius,color,1)

  return output

Cheers,
Nikolas

[Nikolas Karalis]

The code discussed above seems to work fine, but rarely it crashes.
The error is:
Value cannot be null.
Parameter name: SafeHandle cannot be null.

I think this happens when the Largest Binary Region doesn't return anything.

How can I check this in the Python code to avoid the crashing?
I tried  using:
if value != None:

but this doesn't seem to work.

Thanks in advance!

Best,
Nikolas

[Gonçalo Lopes]

Hi Nikolas,

Sorry for the late reply. Indeed, the LargestBinaryRegion always returns something, that's why the code didn't work.

The trick is that when there is no binary region on the screen, the Area of the largest returned region will be zero (and other parameters will have invalid values).

So the correct guard would be:

if value.Area > 0:
  # do something

Hope this helps,

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/200af05a-5649-4341-8507-483c524c61a5%40googlegroups.com.

[Lorenzo Grilli]

Hi guys, 

I found this post very interesting, it might help me too. I have the same problem with reflecting light onto the pupil, and I wanna get rid of it because I would like to extract centroid and pupil diameter.

This is just my first approach to Bonsai and I don't have coding experience, so I'm struggling in the implementation of your suggestion.  Where I should add the PythonTransform embedding the MinimumEnclosinCircle transform? And the same for the circle drawing function. Could you help me, please?

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai...@googlegroups.com.

Visit this group at https://groups.google.com/group/bonsai-users.

[Mikail Weston]

Why don't you try a polarising filter on the lens to reduce the light reflection?

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/8669e86c-7fda-4786-9e09-96974e0aa216%40googlegroups.com.

[Lorenzo Grilli]

I will in the future, but at the moment I have access only at these data, so I need to get rid of the problem with software implementation.

Il giorno venerdì 24 maggio 2019 10:55:29 UTC+2, Mikail Weston ha scritto:

Why don't you try a polarising filter on the lens to reduce the light reflection?

[Adam Lowet]

Hi all,

I'm afraid I'm a novice when it comes to Bonsai, and like Lorenzo I'm having a very hard time figuring out the correct way to incorporate the MinimumEnclosingCircle node. I usually get errors like "No method overload found for the given arguments," which is hard to decipher. The video quality is generally quite good -- contrast is extremely high, albeit with some flicker, and I'll generally only have to deal with very small occlusions (e.g. whiskers; attached .png) -- so I doubt too much processing is necessary here.

My overall strategy is to read in the video stream, threshold it, find contours, and fit the minEnclosingCircle to the largest binary region, but I either get an error, or the minEnclosingCircle node is empty (see attached). Can someone please help? Thanks in advance, and sorry for such a basic question.

[Gonçalo Lopes]

Hi Adam and welcome to the forums,

I guess the MinEnclosingCircle operator could be made a bit easier to use. There are generally two approaches: either you extract the Contour of the LargestBinaryRegion, like so:

Or you can calculate the enclosing circles for all contours, and then look for the one with the largest radius, like so:

Where the property for OrderByDescending is Radius, and inside SelectMany:

The second option at least should deal with the case where no contours at all are detected (e.g. on blinks), but is indeed a bit annoying. Consider raising an issue to improve it for next version.

Hope this helps.

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/28590562-b682-4c61-ab88-2b7b1e9fe704%40googlegroups.com.

[Adam Lowet]

Hi Goncalo,

Thank you so much for the detailed answer! This appears to give me exactly what I was looking for. The second solution is indeed quite elegant, and not something I would have come up with on my own.

To unsubscribe from this group and stop receiving emails from it, send an email to bonsai...@googlegroups.com.