thread synchronisation
Brad
I'm seeing some strange behaviour with my Prosilica camera though and
wondered if I'm doing anything wrong... might just be because I'm new
to threading?
I want to display an image on the screen and then take a picture of
it. So my loop looks like:
for i in range(N):
window.display(img[i])
data[i] = camera.capture()
Where I subclassed the pydc1394.Camera class to form my own class with
a capture() method that looks something like this:
self.new_image.acquire()
self.new_image.wait()
img = self.current_image.copy()
self.new_image.release()
When I call capture() I do not want the currently exposing image that
extends back in time to before I displayed the new picture on the
screen. I want to wait until the next exposure starts, then wait for
that one to finish, and then return that image as my result.
This was causing weird thread starvation problems (I think) - the main
loop would work for a while but then randomly freeze when trying to
acquire the condition object lock. It seemed to be somehow dependent
on both the shutter speed and the amount of time it took for my image
processing inside the loop to complete.
I had a look at the new_image property and it seems to be synchronised
using the new_image condition object, but inside the
CamAcquisitionThread.run method it uses the _condition condition
object instead. So I changed my capture method to acquire and wait on
the self._worker._condition object (instead of self.new_image) and now
it appears to be working better.
The documentation for the Camera class says to use new_image, so I'm
just wondering why the worker thread uses a different one?
Is there some other way to safely grab the next (completely exposed)
frame (without buffering)?
SirVer
> Thanks for posting this project online, it's been very helpful to me.
> I'm seeing some strange behaviour with my Prosilica camera though and
> wondered if I'm doing anything wrong... might just be because I'm new
> to threading?
> I want to display an image on the screen and then take a picture of
> it. So my loop looks like:
>
> for i in range(N):
> window.display(img[i])
> data[i] = camera.capture()
>
> Where I subclassed the pydc1394.Camera class to form my own class with
> a capture() method that looks something like this:
> self.new_image.acquire()
> self.new_image.wait()
> img = self.current_image.copy()
> self.new_image.release()
acquires the new image condition too. Note that you likely run into
some problems if your framerate is high due to the copy: obviously
depending on your resolution and framerate you will not get every
frame with this.
> When I call capture() I do not want the currently exposing image that
> extends back in time to before I displayed the new picture on the
> screen. I want to wait until the next exposure starts, then wait for
> that one to finish, and then return that image as my result.
>
> This was causing weird thread starvation problems (I think) - the main
> loop would work for a while but then randomly freeze when trying to
> acquire the condition object lock. It seemed to be somehow dependent
> on both the shutter speed and the amount of time it took for my image
> processing inside the loop to complete.
regard. However there should be no deadlock regarding the _new_image
condition as it is used as stated in the Python documentation for the
threading module.
> I had a look at the new_image property and it seems to be synchronised
> using the new_image condition object, but inside the
> CamAcquisitionThread.run method it uses the _condition condition
> object instead. So I changed my capture method to acquire and wait on
> the self._worker._condition object (instead of self.new_image) and now
> it appears to be working better.
assert: assert(self._worker._condition == self._new_image).
> The documentation for the Camera class says to use new_image, so I'm
> just wondering why the worker thread uses a different one?
it's constructor.
> Is there some other way to safely grab the next (completely exposed)
> frame (without buffering)?
you, make sure to let us know, so we can fix it.
Cheers,
Holger
Brad Atcheson
On Thu, Sep 2, 2010 at 6:33 AM, SirVer <sir...@gmx.de> wrote:
> Hi Brad,
>
>> Thanks for posting this project online, it's been very helpful to me.
> thanks. May I ask what you use it for?
My research project involves projecting sequences of patterns on a
surface and recording them with a camera to study the light transport
within the scene.
>
>> I'm seeing some strange behaviour with my Prosilica camera though and
>> wondered if I'm doing anything wrong... might just be because I'm new
>> to threading?
>> I want to display an image on the screen and then take a picture of
>> it. So my loop looks like:
>>
>> for i in range(N):
>> window.display(img[i])
>> data[i] = camera.capture()
>>
>> Where I subclassed the pydc1394.Camera class to form my own class with
>> a capture() method that looks something like this:
>> self.new_image.acquire()
>> self.new_image.wait()
>> img = self.current_image.copy()
>> self.new_image.release()
> a simple self.current_image.copy() will be enough as current_image
> acquires the new image condition too. Note that you likely run into
> some problems if your framerate is high due to the copy: obviously
> depending on your resolution and framerate you will not get every
> frame with this.
My shutter speeds are quite high: about 100ms currently. If the image
processing loop is slower than the shutterspeed then I do want to drop
frames, so I'm using the interactive=True mode.
>
>> When I call capture() I do not want the currently exposing image that
>> extends back in time to before I displayed the new picture on the
>> screen. I want to wait until the next exposure starts, then wait for
>> that one to finish, and then return that image as my result.
> This should happen whenever you access self.current_image
>
Unfortunately that is not the behaviour I'm seeing with my Prosilica
EC1350 camera. To test this I used the following program (pj.Camera is
derived from pydc1394.Camera and just handles all the initialisation,
but doesn't modify the capture process in any way):
cam = pj.Camera()
cam.shutter.val = 4000
print("starting camera")
cam.start(bufsize=4, interactive=True)
raw_input("Press enter to capture and save image")
img = cam.current_image.copy()
print("finished copying")
cv.SaveImage("junk.jpg", pj.numpy2opencv(img))
print("finished saving")
cam.stop()
cam.close()
Now I place my hand in front of the camera while the program sits at
the input prompt, for some random amount of time. I then quickly
remove my hand and press enter to allow it to grab the current image.
What I would like to see is an image with no hand present, and
sometimes this is the case. However, I often see a partially exposed
hand instead, and the "finished copying" message often prints very
soon after I press enter - much too soon to allow for a full 4 second
exposure to be recorded. So it definitely looks like the current_image
property returns an image that started exposing before I made the call
to copy it.
>>
>> This was causing weird thread starvation problems (I think) - the main
>> loop would work for a while but then randomly freeze when trying to
>> acquire the condition object lock. It seemed to be somehow dependent
>> on both the shutter speed and the amount of time it took for my image
>> processing inside the loop to complete.
> Sounds like a deadlock. I am not sure if pydc1394 is bug free in this
> regard. However there should be no deadlock regarding the _new_image
> condition as it is used as stated in the Python documentation for the
> threading module.
>
>
>> I had a look at the new_image property and it seems to be synchronised
>> using the new_image condition object, but inside the
>> CamAcquisitionThread.run method it uses the _condition condition
>> object instead. So I changed my capture method to acquire and wait on
>> the self._worker._condition object (instead of self.new_image) and now
>> it appears to be working better.
> those two objects should be idential, you can check this with an
> assert: assert(self._worker._condition == self._new_image).
You're right, I didn't notice that before. Which also means that my
"fix" of using _condition instead of new_image is incorrect. It only
looked like it was working to me before because the original problem
occurred randomly and I just happened to not see it after making the
change.
>
>> The documentation for the Camera class says to use new_image, so I'm
>> just wondering why the worker thread uses a different one?
> It doesn't. The condition object is passed to the worker thread via
> it's constructor.
>
>> Is there some other way to safely grab the next (completely exposed)
>> frame (without buffering)?
> no, this is the canonical way. If you find what causes the errors for
> you, make sure to let us know, so we can fix it.
Right now I removed all the lock acquisition from own code and put in
a call to time.sleep(cam.shutter.val/1000.0 + epsilon) before calling
cam.current_image. So far this seems like the only way to work around
the fact that the camera doesn't appear to have a software trigger.
I just found it confusing that there was no "get_snapshot()" function,
so I had assumed that the current_image property would do that. But it
makes more sense to think of it as just pulling the next frame to
finish exposing off the camera.
Thanks for the help.
- brad
>
> Cheers,
> Holger
>
>
SirVer
> >> Thanks for posting this project online, it's been very helpful to me.
> > thanks. May I ask what you use it for?
>
> My research project involves projecting sequences of patterns on a
> surface and recording them with a camera to study the light transport
> within the scene.
reflecting objects in the scene :).
camera delivers. As the acquisition might already by started, you
might get a partial image. Something like this:
cam.current_image
i = cam.current_image.copy()
will get you the actual fresh acquired image.
> >> This was causing weird thread starvation problems (I think) - the main
> >> loop would work for a while but then randomly freeze when trying to
> >> acquire the condition object lock. It seemed to be somehow dependent
> >> on both the shutter speed and the amount of time it took for my image
> >> processing inside the loop to complete.
> > Sounds like a deadlock. I am not sure if pydc1394 is bug free in this
> > regard. However there should be no deadlock regarding the _new_image
> > condition as it is used as stated in the Python documentation for the
> > threading module.
>
> >> I had a look at the new_image property and it seems to be synchronised
> >> using the new_image condition object, but inside the
> >> CamAcquisitionThread.run method it uses the _condition condition
> >> object instead. So I changed my capture method to acquire and wait on
> >> the self._worker._condition object (instead of self.new_image) and now
> >> it appears to be working better.
> > those two objects should be idential, you can check this with an
> > assert: assert(self._worker._condition == self._new_image).
>
> You're right, I didn't notice that before. Which also means that my
> "fix" of using _condition instead of new_image is incorrect. It only
> looked like it was working to me before because the original problem
> occurred randomly and I just happened to not see it after making the
> change.
on my system occasionally, but cannot reproduce it reliably. So I feel
that I will not be able to fix it atm.
> >> The documentation for the Camera class says to use new_image, so I'm
> >> just wondering why the worker thread uses a different one?
> > It doesn't. The condition object is passed to the worker thread via
> > it's constructor.
>
> >> Is there some other way to safely grab the next (completely exposed)
> >> frame (without buffering)?
> > no, this is the canonical way. If you find what causes the errors for
> > you, make sure to let us know, so we can fix it.
>
> Right now I removed all the lock acquisition from own code and put in
> a call to time.sleep(cam.shutter.val/1000.0 + epsilon) before calling
> cam.current_image. So far this seems like the only way to work around
> the fact that the camera doesn't appear to have a software trigger.
> I just found it confusing that there was no "get_snapshot()" function,
> so I had assumed that the current_image property would do that. But it
> makes more sense to think of it as just pulling the next frame to
> finish exposing off the camera.
functionality; so I didn't design for it in the python wrapper. There
is a nearly complete rewrite of the library by Robert Jordens[1], and
I think it contains snapshot functionality. Maybe you find this one
prettier to use. You could also suggest a API for snapshot
functionality and try to implement it yourself. However, I played
around with the functionality but it is not terrible well documented
in the dc1394 documentation (the C API) and the way I thought it
should work does not on my camera on a first try (tough my camera
claims to have the one-shot feature).
Note that you will not be able to able to mix freerunning
(interactive=True) and one-shot feature, no matter what.
[1] lp:~jordens/pydc1394/work
Cheers,
Holger