Python and webcam capture delay?
jack catcher (nick)
I'm thinking of using Python for capturing and showing live webcam
stream simultaneously between two computers via local area network.
Operating system is Windows. I'm going to begin with VideoCapture
extension, no ideas about other implementation yet. Do you have any
suggestions on how short delay I should hope to achieve in showing the
video? This would be part of a psychological experiment, so I would need
to deliver the video stream with a reasonable delay (say, below 100ms).
Tim Roberts
You need to do the math on this. Remember that a full 640x480 RGB stream
at 30 frames per second runs 28 megabytes per second. That's more than
twice what a 100 megabit network can pump.
You can probably use Python to oversee this, but you might want to consider
using lower-level code to control the actual hardware. If you are
targeting Windows, for example, you could write a DirectShow graph to pump
into a renderer that transmits out to a network, then another graph to
receive from the network and display it.
You can manage the network latency by adding a delays in the local graph.
--
Tim Roberts, ti...@probo.com
Providenza & Boekelheide, Inc.
jack catcher (nick)
Thanks Tim, you're correct about the math. What is your main point about
DirectShow: that it is generally faster and more reliable than doing the
job high-level, or that one could use coding/decoding in DirectShow to
speed up the transmission? I think the latter would be a great idea if
the latency were tolerable. On the other hand, I'd like to keep things
simple and do all the programming in Python. I've got no experience with
DirectShow, but I guess the filters need to be programmed in C++ and
called from Python?
Another option might be to use resolution 320x240@15fps.
Stef Mientki
I couldn't get it working, ...
... it seems to start ok
... but moving the captured window,
... or sometimes even just moving the mouse hangs the program :-(
So I'm still looking for a good / open source workiing video capture in
Python.
I can make a good working capture in delphi,
make a DLL or ActiveX from that and use it in Python,
but I'm not allowed to ditribute these.
cheers,
Stef
Rhodri James
wrote:
Does the webcam just deliver frames, or are you getting frames out of
a decoder layer? If it's the latter, you want to distribute the encoded
video, which should be much lower bandwidth. Exactly how you do that
depends a bit on what format the webcam claims to deliver.
--
Rhodri James *-* Wildebeest Herder to the Masses
jack catcher (nick)
Well, getting already encoded video from the webcam sounds almost like a
free lunch (which it probably is not). At least I wouldn't want to get
too long a delay because of the encoding.
I haven't got the webcam(s) yet, and I guess I can basically purchase
any ones I find suitable for getting the job done. Any recommendations?
Nobody
>> Does the webcam just deliver frames, or are you getting frames out of
>> a decoder layer? If it's the latter, you want to distribute the encoded
>> video, which should be much lower bandwidth. Exactly how you do that
>> depends a bit on what format the webcam claims to deliver.
>
> Well, getting already encoded video from the webcam sounds almost like a
> free lunch (which it probably is not). At least I wouldn't want to get
> too long a delay because of the encoding.
>
> I haven't got the webcam(s) yet, and I guess I can basically purchase
> any ones I find suitable for getting the job done. Any recommendations?
The webcam is bound to do some encoding; most of them use USB "full speed"
(12Mbit/sec), which isn't enough for raw 640x480x24bpp@30fps data.
Chroma subsampling and JPEG compression will both reduce the bandwidth
without introducing noticable latency (the compression time will be no
more than one frame).
Temporal compression will reduce the bandwidth further. Using B-frames
(frames which contain the differences from a predicted frame which is
based upon both previous and subsequent frames) will provide more
compression, but increases the latency significantly. For this reason, the
compression built into video cameras normally only uses P-frames (frames
which contain the differences from a predicted frame which is based only
upon previous frames).
The biggest issue is likely to be finding latency specifications, followed
by finding a camera which meets your latency requirement.
Also, any "read frame, write frame" loops will add an additional frame of
latency, as you can't start writing the first byte of the frame until
after you've read the last byte of the frame. Video APIs which let you get
rows as they're decoded are rare.
Rhodri James
wrote:
> Rhodri James kirjoitti:
>> Does the webcam just deliver frames, or are you getting frames out of
>> a decoder layer? If it's the latter, you want to distribute the encoded
>> video, which should be much lower bandwidth. Exactly how you do that
>> depends a bit on what format the webcam claims to deliver.
>
> Well, getting already encoded video from the webcam sounds almost like a
> free lunch (which it probably is not). At least I wouldn't want to get
> too long a delay because of the encoding.
Not so unlikely as you might think, since there are very basic M-JPEG and
MPEG-2 on-chip encoders available, and the webcam will have to do some
compression to get the video data into the computer. USB is not as fast
as it would like to pretend.
> I haven't got the webcam(s) yet, and I guess I can basically purchase
> any ones I find suitable for getting the job done. Any recommendations?
Sorry, no. I'm used to getting my video feeds down fibre-optic cables :-)
jack catcher (nick)
> On Mon, 06 Jul 2009 20:41:03 +0300, jack catcher (nick) wrote:
>
>>> Does the webcam just deliver frames, or are you getting frames out of
>>> a decoder layer? If it's the latter, you want to distribute the encoded
>>> video, which should be much lower bandwidth. Exactly how you do that
>>> depends a bit on what format the webcam claims to deliver.
>> Well, getting already encoded video from the webcam sounds almost like a
>> free lunch (which it probably is not). At least I wouldn't want to get
>> too long a delay because of the encoding.
>>
>> I haven't got the webcam(s) yet, and I guess I can basically purchase
>> any ones I find suitable for getting the job done. Any recommendations?
>
> The webcam is bound to do some encoding; most of them use USB "full speed"
> (12Mbit/sec), which isn't enough for raw 640x480x24bpp@30fps data.
>
> Chroma subsampling and JPEG compression will both reduce the bandwidth
> without introducing noticable latency (the compression time will be no
> more than one frame).
>
> Temporal compression will reduce the bandwidth further. Using B-frames
> (frames which contain the differences from a predicted frame which is
> based upon both previous and subsequent frames) will provide more
> compression, but increases the latency significantly. For this reason, the
> compression built into video cameras normally only uses P-frames (frames
> which contain the differences from a predicted frame which is based only
> upon previous frames).
>
> The biggest issue is likely to be finding latency specifications, followed
> by finding a camera which meets your latency requirement.
Thanks for the comments. Unfortunately, such specifications aren't easy
to find, even in reviews. Fortunately several newer webcams seem at
least to use usb2.
Regarding Python, I'll have to check whether the ImageCapture extension
works at all, as someone suggested earlier, and the possibility of using
DirectShow. I'm still considering Matlab as an option for Python here.
Nobody
> Thanks for the comments. Unfortunately, such specifications aren't easy
> to find, even in reviews. Fortunately several newer webcams seem at
> least to use usb2.
Supporting USB-2 doesn't mean that the camera necessarily uses high-speed
(480Mbit/sec).
AFAIK, the only real difference between "USB-1 conformant" and "USB-2
conformant" is that the latter actually passed a test of its ability to
say "no, I can't do high-speed", while the former didn't. The check for
high-speed capability was designed such that it shouldn't cause problems
for USB-1 devices, but individual USB-1 devices weren't actually tested
for this.
Tim Roberts
>
>The webcam is bound to do some encoding; most of them use USB "full speed"
>(12Mbit/sec), which isn't enough for raw 640x480x24bpp@30fps data.
That's not true. Most of the web cams made in the last 5 years or so run
at high speed, 480 Mbps. Full speed only gets you 1 fps at 640x480
uncompressed, so it's really only useful for the most primitive video
conference cams.
Nobody
> On Tue, 07 Jul 2009 23:37:43 +0100, Nobody <nob...@nowhere.com>
> declaimed the following in gmane.comp.python.general:
>
>> AFAIK, the only real difference between "USB-1 conformant" and "USB-2
>> conformant" is that the latter actually passed a test of its ability to
>> say "no, I can't do high-speed", while the former didn't. The check for
>> high-speed capability was designed such that it shouldn't cause problems
>> for USB-1 devices, but individual USB-1 devices weren't actually tested
>> for this.
>
> USB-1 or USB-1.1? As I recall the latter is capable of 11Mbps
> whereas the original USB spec was much lower... (of course, USB 2 at top
> is 480Mbps)
USB 1.0 supports 1.5Mbps (low-speed) and 12MBps (full-speed). Hosts and
hubs must support both, functions (devices) can use either. USB 1.1 was
a bug-fix release which solved some interoperability issues arising from
ambiguities in the USB 1.0 standard.
Nobody
>>The webcam is bound to do some encoding; most of them use USB "full speed"
>>(12Mbit/sec), which isn't enough for raw 640x480x24bpp@30fps data.
>
> That's not true. Most of the web cams made in the last 5 years or so run
> at high speed, 480 Mbps. Full speed only gets you 1 fps at 640x480
> uncompressed, so it's really only useful for the most primitive video
> conference cams.
The very earliest models typically only did 320x200, while later USB-1
webcams used onboard compression to get a decent framerate.
For internet use, 12Mbps isn't that much of an obstacle; the internet
connection's upload speed is more likely to be the limiting factor. Faster
speeds are more useful for things like LAN-based CCTV systems.