Multiple UVC Cameras, One USB 2.0 Bus, Linux

[Matt Baker]

Hey guys, did anyone come with a working solution to getting linux to reduce the camera USB bandwidth allocation?  This is kind of a rehash of GitHub issue #127 (https://github.com/openpnp/openpnp/issues/127).  I'm using two ELP cameras.

The laptop I wanted to use has only one root hub shared by its three USB ports.  A second root hub is listed at hardware enumeration, but I can only assume it is dedicated to the docking station expander -- my quick fix backup plan is to just get one of the old docking stations on eBay to relieve this issue (and de-clutter my operating station).

Longer term, I'd like to go fully IP for isolation, and am investigating whether a travel router (cheapest consumer electronics with ethernet+usb controllers) can be used as a USB to IP camera converter.  This is a candidate: https://www.amazon.com/HooToo-Wireless-Performance--TripMate-Hotspot/dp/B00HZWOQZ6/ .

[Cri S]

The router works.
You must upgrade it to openwrt including installing the motion module.

[Cri S]

For Linux, there exist a patch for v4l that ignore be, it works on 60%.
Camera change need 3-7 second, still picture takes 1 to 3 seconds.

[Matt Baker]

Hmm, I'll look into patches a little bit I guess.  One user said he overrode the allocation code.  Considering that it doesn't reduce bandwidth when you reduce framerate, it should not hit an actual throughput problem.

The camera change seems slow, but I suppose after initial setup the down facing camera may not need to be particularly active, and could reduce the number of switches.  I need more experience running OpenPnP.

[Jason von Nieda]

Matt,

There is an unfinished camera implementation in https://github.com/openpnp/openpnp/tree/feature/uvc-camera that should make it possible to solve the problem. If you are Java savvy, you could have a play with that. 

Overall, I feel like camera switching is a dead end, as I noted in the original issue. It can probably be made to work, but I think it would be better to get the UVC camera working and stable. I think the only thing really left to do is get all the packaging and distribution of libuvc going, but if you already have it installed the camera should work. 

In fact, with that in mind, I will see about getting that code merged into main and just pop a dialog to users letting them know they will need to install libuvc to use it.

Jason

On Wed, Sep 7, 2016 at 10:19 AM Matt Baker <baker....@gmail.com> wrote:

Hmm, I'll look into patches a little bit I guess.  One user said he overrode the allocation code.  Considering that it doesn't reduce bandwidth when you reduce framerate, it should not hit an actual throughput problem.

The camera change seems slow, but I suppose after initial setup the down facing camera may not need to be particularly active, and could reduce the number of switches.  I need more experience running OpenPnP.

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/CALn7eW_eANg%2BRDFumNZF0_HeaNK4cyRUp_Z4_zTsfYAhWT-kKA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Jason von Nieda]

Oh, one thing I forgot to mention: The real limitation isn't frame rate, it's compression method. To use multiple ELP cams on one root you have to use them in MJPEG mode instead of YUV mode. 

[Matt Baker]

Interesting, thanks Jason.  I'll play around with the implementation in the branch when I get a chance.  It is on my list to get the openpnp project setup directly from the github for the future.  My Java is rusty but it is similar enough to C.

It seems we're brushing on two issues: the maximum bandwidth required for modern webcams is approaching the USB2 bus bandwidth, and either the camera itself or the Linux UVC driver is overly aggressive with bandwidth allocation.  I guess it depends whether that bandwidth is necessary or not during the bursting of a single camera frame.

libuvc seems like the way to go, and would alleviate the bandwidth constraint either by enabling MJPEG compression as you suggest or possibly/additionally by more accurately dialing back the required bandwidth in lower frame rate modes.

No idea what kind of latency is involved, but it seems like for the upfacing camera, even being able to just snap still frames may be all that is necessary.  Though, it seems that in the drivers the camera has to be pre-connected to, and might reserve the bandwidth anyway.  May not be worth the effort if we can enable two cameras in video mode on one controller anyway.  That would enable a dual camera machine on a single host controller PC, which from my single sample suggests it may be relatively standard for laptops and may be common on other mini PCs as well (might be worth investigating some mini-ITX boards and Intel NUC to see how common this constraint is).  Should probably just add more USB controllers to the PC when possible though, as it would be cheaper than the labor to fully optimize the software.  Supporting two cameras per controller seems like a reasonable target though.

[Cri S]

Libusb dont resolve anything until still picture is used to get image and then in memory jpeg parse from opencv.
As root:
rmmod uvcvideo
modprobe uvcvideo quirks=640
echo -1 > /sys/module/usbcore/parameters/autosuspend
echo 0x400 > /sys/module/uvcvideo/parameters/trace

Now tray running both cameras with mjpeg and check kernel logss for band with allocation usually 48 to 52% is allocated by default.
Generally running 4 cameras at 15 fps is no problem.
Last resort is module recompilation and patching it manually.

[Cri S]

In case 640 works, try 128.
You can make it permanently as option inside etc.

With
4l2-ctl --list-formats

You can list formats and compressions.
Example
v4l2-ctl --set-fmt-video=width=640,height=480,pixelformat=1

[Jason von Nieda]

Maybe that's the case w/ Linux - I haven't checked, but on my Mac I can use libuvc with my LibuvcCamera implementation to run multiple cameras using MJPEG. Still captures are not required - streaming works fine. Code is here: https://github.com/openpnp/openpnp/blob/feature/uvc-camera/src/main/java/org/openpnp/machine/reference/camera/LibuvcCamera.java

Jason

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/871168ab-7121-4637-9808-66d40c284744%40googlegroups.com.

[Matt Baker]

I tried quirks=128 last night messing around.  It didn't drop the bandwidth for any of the intermediate resolution steps until it was about about 1/3 the full resolution.  I could not get the frame rate to reduce the bandwidth at all.  I'll play around some more tonight, but I already have the docking station on order so I don't have to compromise in the short term.

[Cri S]

Camera can be switched to mjpg using several command line tools.
Problem is, camera reserves 48% on all systems regards real used bandwith if using full resolution. Usb2rs232 driver reserve 20% .
Mouse and keabord reserve 2% each.
Now if one have only one USB root hub, there is not enough bandwith without hacking parameters, not on Linux and not on win, on Mac I presume its same if you use the same root switch.

[Jason von Nieda]

Cri,

I don't think that's correct, at least on Mac. Simple test I did:

1. Open two cameras on a USB hub (one USB root) at full resolution using OpenCV (YUV): Locks up.

2. Open two cameras on a USB hub (one USB root) at full resolution using libuvc (MJPEG): Works fine.

As I said, I have not tested this in Linux, only on Mac, but I was present when someone else tested it on Linux and it also behaved this way. That's why I started writing the LibuvcCamera - because it seemed like it would solve the problem.

I did not check what bandwidth it reserved - not sure how to check that on Mac. But it did work, which was all I cared about :)

Jason

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/d782aed6-7a74-4b35-bc64-ea4f322d5e91%40googlegroups.com.

[Cri S]

Same on Linux, but if additional smoothie/arduino/.. Is present on same root hub, it don't work a anymore without tweak.

[Jason von Nieda]

Ah, I see. I don't remember if I tested that or not.

Jason

On Wed, Sep 7, 2016 at 12:37 PM Cri S <phon...@gmail.com> wrote:

Same on Linux, but if additional smoothie/arduino/.. Is present on same root hub, it don't work a anymore without tweak.

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/64b5455e-2036-4de2-8278-dc8550583f86%40googlegroups.com.

[Matt Baker]

Problems with low speed devices could be in the hub controller rather than the root hub.  Would like to see that kind of test run with a proper Multi-TT USB hub that can handle multiple low speed devices.  I would expect a hub controller inside any decent PC would be proper multi tt but you never know.  I did have my smoothie plugged in and on the same root hub during my camera testing so it could be a contributing factor to my experience.

On Wed, Sep 7, 2016 at 12:55 PM, Jason von Nieda <ja...@vonnieda.org> wrote:

Ah, I see. I don't remember if I tested that or not.

Jason

On Wed, Sep 7, 2016 at 12:37 PM Cri S <phon...@gmail.com> wrote:

Same on Linux, but if additional smoothie/arduino/.. Is present on same root hub, it don't work a anymore without tweak.

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.

To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+unsubscribe@googlegroups.com.

To post to this group, send email to ope...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/64b5455e-2036-4de2-8278-dc8550583f86%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "OpenPnP" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openpnp/gSSFehc1cwI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openpnp+unsubscribe@googlegroups.com.

To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/CA%2BQw0jxd3R%2BwDcVk2FOm7qHvsRHobKJiw_%2B%3D_s%3DM0KuOs6GPsw%40mail.gmail.com.

[Jason von Nieda]

Alright, I was going to go ahead and merge this code in so people could play with it easier, but I had forgotten it depends on another library I have not finished yet. until I do that, and deploy it, there's not much point in merging this as it will just break things. I'm putting this at the top of my TODO list for when I am settled in my new house which will be around the end of September. At that time I will get this all finished up and deployed so people can start using it. 

Jason

On Wed, Sep 7, 2016 at 1:42 PM Matt Baker <baker....@gmail.com> wrote:

Problems with low speed devices could be in the hub controller rather than the root hub.  Would like to see that kind of test run with a proper Multi-TT USB hub that can handle multiple low speed devices.  I would expect a hub controller inside any decent PC would be proper multi tt but you never know.  I did have my smoothie plugged in and on the same root hub during my camera testing so it could be a contributing factor to my experience.

On Wed, Sep 7, 2016 at 12:55 PM, Jason von Nieda <ja...@vonnieda.org> wrote:

Ah, I see. I don't remember if I tested that or not.

Jason

On Wed, Sep 7, 2016 at 12:37 PM Cri S <phon...@gmail.com> wrote:

Same on Linux, but if additional smoothie/arduino/.. Is present on same root hub, it don't work a anymore without tweak.

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.

To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.

To post to this group, send email to ope...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/64b5455e-2036-4de2-8278-dc8550583f86%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "OpenPnP" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openpnp/gSSFehc1cwI/unsubscribe.

To unsubscribe from this group and all its topics, send an email to openpnp+u...@googlegroups.com.

To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/CA%2BQw0jxd3R%2BwDcVk2FOm7qHvsRHobKJiw_%2B%3D_s%3DM0KuOs6GPsw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "OpenPnP" group.

To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.

To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/CALn7eW_-2rjc9Hv4KhoTm-inn-QmmUSpw0Tb2-iX2jadaF8NNA%40mail.gmail.com.