VGA camera capture

[João M. S. Silva]

We are not able to capture VGA (640 x 480) video or frames with the BBB.

We've experimented with mplayer, luvcview, guvcview, cheese, opencv, simplecv, v4l, etc. and the behaviour seems consistent.

Lower resolutions like 320 x 240 work, but 640 x 480 doesn't.

We've tested with Ubuntu, Fedora and Arch Linux BBB images.

With a Logitech camera which is able of doing up to 1280 x 1024, although 1280 x 1024 works, 640 x 480 still does not work.

The cameras work OK in other computers.

Should this be a bug in the hardware or software?

[Matthew Witherwax]

João,

I am in the process of writing this up on my blog; however, I hope this information helps you.

I have recently run into the same issue on the BBB with a PS3Eye and this is what I have found.

Initially I was unable to capture any images at 640x480, but I could capture at 320x240.
I found this project v4l2grab and made changes to it to allow me to set the frame rate.

With the ability to set the frame rate, I discovered I could capture images at 640x480 if the frame rate was no more than 15 fps.
I could also capture 320x240 at up to 60 fps.

Doing some quick calculation we see that 15 * 640 * 480 * 24 = 30 * 320 * 240 * 24
where 24 is the bits per pixel (8 * 3) in the uncompressed image from the PS3Eye.

This works out to just about 13.2 MB/s.

I unloaded the usb driver for the audio from the PS3Eye, ensured there were no other usb devices connected over usb, and connected the PS3Eye directly to the board to ensure it would have the maximum amount of bandwidth.  Long story short, it seems 13.2  MB/s is all it will handle.

I did some further testing with the same program and running off a Bodhi live cd and found that a laptop I had with an Intel 5 series 3400 usb controller could support a max of 30 fps at 640x480 while one with an Intel 6 series c200 was able to achieve the full 60 fps at 640x480 that the PS3eye is capable of.

If your camera supports compression, enabling it should allow you to capture larger images and/or at a faster frame rate.

[Don Miller]

You guys should check out Derek Molley's blog.  He was able to capture high quality images at reasonable frame rates if he used a camera that did H264 encoding internally.
He was able to stream at 1920x1080, 25fps with the BBB @ 12% CPU utilization.

You need to put a little more money in the camera to do this.

http://derekmolloy.ie/beaglebone-images-video-and-opencv/
http://derekmolloy.ie/streaming-video-using-rtp-on-the-beaglebone-black/

Don

[Matthew Witherwax]

Don,

Thanks for the links.  As I said, if your camera supports compression, enabling it should allow you to capture large images and/or at a faster frame rate.  I was simply detailing why cameras weren't working so people can understand why they need to spend more money.  I didn't want to just say spend more money and it will work.

--
For more options, visit http://beagleboard.org/discuss
---
You received this message because you are subscribed to a topic in the Google Groups "BeagleBoard" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/beagleboard/2NO62mGcSvA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to beagleboard...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Don Miller]

There is something interesting in the OP:  the fact that with his camera, 1280x1024 actually works, but 640x480 does *not*.

Do you suppose that his Logitech camera is H264 capable, but maybe it doesn't use H264 for the lower resolutions?

[Matthew Witherwax]

The command v4l2-ctl --list-formats-ext will list all available pixel formats and resolutions.  When I execute that with my Logitech C920, it shows it is capable of H264, YUYV, and MJPG.  I can pull single images up to 1920x1080 off of it using either H264, MJPG, YUYV (uncompressed) when the frame rate is 30fps.  I have successfully (in the sense that I got video with some dropped frames) captured video at 1920x1080 in both compressed formats as well, but I get select timeouts when uncompressed.

The PS3Eye plays a little differently.  It transfers data in bulk mode as opposed to isochronous like most other web cams and seems to really tax the bus.  Bulk guarantees delivery but not speed and isochronous is the opposite.  Cameras using bulk transfers need to use compressed images even for low frame rates or the data takes too long.  Judging by the dropped frames
with isochronous, relatively large captures can be sent - frames just get lost.

For the OP, I would suspect the camera is sending YUYV at 640x480. I have noticed when using OpenCV with Python, it doesn't matter what pixel format I set using v4l2-ctl, OpenCV will switch to YUYV when I start a capture.

 

On Tue, Aug 27, 2013 at 6:33 PM, Don Miller <greenca...@gmail.com> wrote:

There is something interesting in the OP:  the fact that with his camera, 1280x1024 actually works, but 640x480 does *not*.

Do you suppose that his Logitech camera is H264 capable, but maybe it doesn't use H264 for the lower resolutions?

--

[Matthew Witherwax]

Just wanted to clarify on OpenCV and YUYV - I gave it as an example of a program that will change the pixel format to YUYV without explicitly asking it to.  I didn't mean to suggest the other programs OP listed do the same.  One interesting side effect I have noticed when using OpenCV is after I do a capture with OpenCV, if I open another vid capture program, it will be set to YUYV as well since v4l2 is used to control the pixel format and it keeps the last set pixel format (resolution and fps as well) across programs.

To unsubscribe from this group and all its topics, send an email to beagleboard+unsubscribe@googlegroups.com.

[james....@btinternet.com]

So will my YUYV cam always work slow? 

I'm running the moustache placer software from here: http://beagleboard.org/project/stache

It produces a window that is probably 640x320, but I get what looks like a 320 feed and two underneath, one smaller that the other...

There are some settings in the file I haven't managed to get to work to lower the resolution yet. It runs at 100% cpu, so all progress is slow.

But if I could benefit from a camera with compression, i'd be happy to get one, except I only need 320x240 resolution... So I can process opencv functioning.

Btw cheese works extremely well with my cam at 640x320, drops a few frames, but you can move your hand it front of it and see it move in real time, 1-2sec delay. Its not bad.

To unsubscribe from this group and all its topics, send an email to beagleboard...@googlegroups.com.

[João M. S. Silva]

Thanks Matthew, Don.

Here is my follow up:

1. Matthew, what is you blog address?

2. I compiled v4l2grab and used the -I switch to adjust the fps. I found out that with the Logitech camera I can capture 640x480 at up to 6 fps. With -I 7 or above, it hangs. With another cheap camera, even -I 1 hangs!

3. Is 13.2 MB/s somehow BBB's limit? Why?

4. My cheap cameras are YUYV capable and one of them is MJPEG capable. The Logitech is both YUYV and MJPEG capable.

5. As I've read elsewhere UVC does not allow to get still images, so video performance issues affect still image capturing, right? is there a way to capture pure still images (bandwidth limitations should be irrelevant in that case)?

6. Overall, this is an issue from the BBB, right? All of this works in my laptop and in BBXM. Is it a bug from the BBB USB implementation?

7. I also found BBB's USB is not really plug-and-play. Sometimes devices don't get recognized and a reboot is needed.

Thanks.

[Matthew Witherwax]

João,

1.  Glad to hear you are making progress.  My blog is at blog.lemoneerlabs.com.  Unfortunately I have yet to post my write up on the USB cameras, but I will get it done soon.  I can tell you with the Logitech C920 it is possible to capture 1920x1080 frames with the FPS set to 30 in both MJPEG and YUYV format.

2.  v4l2grab sets the format to YUYV before grabbing the frames.  It then converts the captured frame to a jpeg and saves it.  I will post a version to my blog this week that uses MJPEG instead.  Typical MJPEG implementations encode each video frame as a jpeg.  The code I wrote puts the webcam in MJPEG mode for capture and saves the returned frame.  In the case of the Logitech C920, the returned frame is a jpeg image.  It seems by going this route the camera actually creates the jpeg sparing the BBB from having to do it.  In addition, the jpeg is considerably smaller than the YUYV frame resulting in less load on the USB.  I would appreciate you testing the output when I post the code as it is entirely possible that not all cameras handle MJPG the same, and the result might not be a valid jpeg.

3.  13.2 MB/s is the limit I was able to reach through testing.  You should note the webcam I tested with for that number only did bulk transfers which guarantee delivery.  It may be slightly higher for isochronous transfers as frames can (and seem to) get dropped.  I am not sure why that is the limit.  It maybe a hardware issue or a USB driver implementation issue.  During further testing on several laptops, the newer ones could reach a higher throughput even though all had a USB 2 bus which would lead me to believe it is hardware related, but it is a question for the hardware guys.

4.  I will post my code this week.  Please test it with your MJPEG capable cameras, and let me know the results.

5.  The UVC implementation in Linux does not support still image capture.  This means tools like v4l2grab set the camera to record in a mode that uses no compression or intra-frame compression (http://en.wikipedia.org/wiki/Intra-frame) and then grab frames from the stream.  This is why the format and frame rate affect the capture.  The stream has to be open and the data pulled before we can grab an image.

6.  Possibly, see 3.

7.  For my purposes, I have not tested plug-and-play so I cannot say much about it.

--

[James Richins]

Did you register anything on CPU usage. I know running moustache placing cv code it will run at 100% CPU on a standard webcam. Is pre processing less cpu intensive or will cv code always push the bb to the max?

I didn't think I'd rush to buy a h264 hardware encoding cam if its the software and running at 320 is just as efficient as preencoded hd video. 

[Matthew Witherwax]

James,

I have not done any cpu testing other than while saving single frames.  In MJPEG format, capturing a single frame at 1920x1080 30 fps and saving it was taking about 6% cpu.  Doing the same in YUYV at a reduced resolution and converting to jpeg was taking about 70%.  I would not rush out to buy a h264 camera.  First, I am not sure OpenCV decodes h264 streams.  Second, if it did, I am not sure you would see much improvement because of the need to decode the h264 stream.  If the BBB does this in hardware is likely wont be bad, but if it is a software implementation we are just shifting the burden.  I am currently working through several tradeoffs.  YUYV should give the truest image free of compression artifacts, but will be larger and require more resources to capture and process.  MJPEG may suffer from compression artifacts, but images can be captured more quickly and at higher resolutions.  One has to decide what combination of frame rate, resolution, and fidelity are required.  I am currently working through the permutations to see what suits my application.

An option I have open is to send the captured images over a wifi link to a computer that runs all the OpenCV code and pipes back the data I am after.  This leaves the BBB mostly free to do other work.  I will update as I progress.

Matthew Witherwax

[James Richins]

Matthew,

It does remind of something on Derek Molloy's site in that he adds coding to deal with the h264 codec for taking stills with opencv. I hesitated at the idea. Interestingly even using an additional bbb or raspberry pi might not even help. I'm looking at camera tracking objects/object recognition and servo control. 

I might find that by not displaying the video on the monitor, that the tracking and servo control can work less than 100% CPU.

I have yet to implement the cv code for that, it's a real struggle to find good code. And not get errors. I have managed servo control through python and manual entering numbers so its ultimately possible.

I'm not sure if my camera does anything other than yuyv. I'd be interested in connecting an Ethernet cam to the Ethernet port and read camera data from there, but you'd probably run into the same problem. Ethernet cam to pc processing cv code, wifi to beaglebone servo control might be good. But its probably a slow process and too complicated for me.

Anyway good luck with your project.

James

[Matthew Witherwax]

Getting another BBB or raspberry pi probably wont help, but a U2 from HardKernel here http://www.hardkernel.com/renewal_2011/products/prdt_info.php?g_code=G135341370451 probably would.  You can use the BBB to do all IO and use the processing power of the U2 to handle compute intensive tasks.  That is actually my end goal.  The cpu use I stated earlier was without showing the image.  Displaying the image will increase cpu use.

I am also working on a tracking application with the webcam mounted on a servo.  For my purposes, I do not need to see the image on the BBB and can push it over wifi to my laptop if I want to view it.

As far as cv code, what are you looking for?  I have posted a tool on my blog here http://blog.lemoneerlabs.com/post/shades-of-red to help you find HSV values from images to allow you to to find values to threshold target colors.  I will post another one on using HSV ranges to threshold an image to isolate things like a red colored ball.  I have been implementing OpenCV code using python right now for experimentation, but you should be able to translate it to C++ or your language of choice.

The command v4l2-ctl --list-formats-ext will tell you the pixel formats and resolutions supported by your camera.

Hope this helps,

Matthew Witherwax

[James Richins]

Thanks Matthew,

I'll recheck the camera to see what t produces,

I have seen the U2 before. I was hoping not to spend much more... I might. The development is for a pc less version of 
http://projectsentrygun.rudolphlabs.com/

Indeed I won't need to see the video stream directly, similarly a wifi module would be nice to capture images of tracked

Objects. I'm only planning on using water pistols though, nothing as beefy as some people.

The red ball tracking would be a reasonable start. I'm not not well versed in opencv. The pc/java version uses a particular

Form of tracking. I wouldn't mind a adding a colour detector to avoid my cat, or whatever colour collar she has on.

Ingeniously your colour detector code might be just the thing to detect a particular range on a specific item as such.

Cheers

James

[Matthew Witherwax]

James,

I have posted the application and write up for thresholding a target color here http://blog.lemoneerlabs.com/post/hsv-threshold

Capture code to follow soon.

Matthew Witherwax

[James Richins]

Thanks Matthew,

I did send you a message through the site. I don't know if that works, it basically a question as to whether I have opencv setup correctly and setting up the pythonpath. Which actually it might be well worth you posting on the site so anyone can have a go.

Regards,

James

Keep up the good work. I look forward to giving the code a scent whirl.

[Matthew Witherwax]

James,

I received your email. To install OpenCV and Python, I first installed Python then OpenCV using Arch's package manager.  What distro are you using?

[James Richins]

Ah,

Just read things through a bit more. Well I'll probably give Arch Linux a go rather than angstrom 3.8.13

I'm no longer sure how I have especially installed python and opencv. It's python 2.7. 

I can run c++ program's but python eludes me apart from the adafruit setup but they have a load of setup files..

James

Sent from my iPad

[Matthew Witherwax]

James,

I wanted to let you know I posted the code I am using to capture images and time webcams on my BBB here http://blog.lemoneerlabs.com/post/BBB-webcams

I will take a look at what is required to run OpenCV with Python on Angstrom when I have a free moment.

[João M. S. Silva]

My follow-up on this subject:

1. In attachment, a table with experiments on the cameras I have. Entries are in the format "laptop / BBB".

2. To test the USB bandwidth, I experimented playing some video files on a USB pen connected to the BBB with mplayer and redirecting X to my laptop (connecting with ssh -Y). The movie files were:

  a) RIFF (little-endian) data, AVI, 608 x 256, 23.98 fps, video: XviD, audio: MPEG-1 Layer 3 (stereo, 48000 Hz)
  b) RIFF (little-endian) data, AVI, 720 x 404, ~30 fps, video: DivX 5, audio: MPEG-1 Layer 3 (stereo, 48000 Hz)
  c) ISO Media, MPEG v4 system, version 1

Results are: a) played OK, b) played a little slow and c) played really slowly, almost stopped.

3. I have a theory on why the Logitech camera works and the others don't. The Logitech is the only one which is able of fps = 5, 10, 15, 20, 25 and 30. The others are 30 fps (YUYV) and one of them is 60 fps (MJPG). This goes along with what Matthew (?) said: fps must be reduced in order to be able to capture. This reduction is probably made automatically with the Logitech, while the others are not able to go below 30 or 60 fps.

4. I've been experimenting with this v4l2 code:

    fd_set fds;
    FD_ZERO(&fds);
    FD_SET(fd, &fds);
    struct timeval tv = {0};
    tv.tv_sec = 0;

    if(-1 == xioctl(fd, VIDIOC_STREAMON, &buf.type))
    {
        perror("Start Capture");
        return 1;
    }
 
/*    int r = select(fd+1, &fds, NULL, NULL, &tv);
    if(-1 == r)
    {
        perror("Waiting for Frame");
        return 1;
    }*/
 
    if(-1 == xioctl(fd, VIDIOC_DQBUF, &buf))
    {
        perror("Retrieving Frame");
        return 1;
    }
   
    if(-1 == xioctl(fd, VIDIOC_STREAMOFF, &buf.type))
    {
        perror("Stop Capture");
        return 1;
    }

to see if I could reduce the interval during which the stream is open (and data is transferred through the bus). This code is more or less the same in Derek Molloy's examples, v4l2grab, etc. but I'm using it from capturev4l2. I had to add the VIDIOC_STREAMOFF, otherwise my laptop would crash with kernel panic. I moved the FD_* code out of the interval and commented the call to select, but it does not change the result.

[Matthew Witherwax]

João,

Please see my post here http://blog.lemoneerlabs.com/post/BBB-webcams

There you will find some timing results for the PS3Eye and the Logitech C920 as well as an example application that will allow you to change the frame rate.

--

[James Richins]

Yes I successfully used capture.c etc, then found I couldn't find a suitable imageviewer program for angstrom. Gimp always crashes. 

Btw Matthew, with the moustache placer code there's a little script which is supossed to be able to tweet you your photo. That might be useful to look at.

It is certainly an interesting set of results. Basically opencv can be very limiting. But it's excellent to see how much data is passed by the camera in one go. 

I did look at the narcissus website yesterday which handles angstrom package building. However it looks like the stripped down version can provide less usb functionality. But I still feel its all in the software used for streaming that the cpu power is used, so I'm not sure if distributions can be managed. One potential for me would have been to lower the CPU MHz and reduce the power consumption, so I could run on solar/battery.

Keep up the could work. I only really need to be able to tell python to be able to find the opencv modules. 

If I get a chance I might try and go for a step by step attempt with C++.

James

--

For more options, visit http://beagleboard.org/discuss
---
You received this message because you are subscribed to a topic in the Google Groups "BeagleBoard" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/beagleboard/2NO62mGcSvA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to beagleboard...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

<cameras.pdf>

[João M. S. Silva]

Matthew,

When you say:

I can tell you with the Logitech C920 it is possible to capture 1920x1080 frames with the FPS set to 30 in both MJPEG and YUYV format.

you mean in BBB?

[Matthew Witherwax]

Correct.  On the BBB, you can still capture frames from the C920 with the FPS set to 30; however, you will only get about 12 FPS in performance.  Because of the way the PS3Eye sends the data, you cannot receive any frames when the FPS is set to 30 and the resolution is 640x480.

--

[James Richins]

Actually I was experiencing a thing with my camera where I think it was set to 640x480 but it gave me 3 streams fitted into the 640 window none of them being 640x480 but it had tried to fit stream data to the window.

A bit odd, but makes some sense, but interestingly it may have forced the stream into the window.

Regards,

James

Sent from my iPad

[James Richins]

Matthew,

If opencv is correctly in your pythonpath can you post what it says.

For me in the python console.

Import sys

Print sys.path

Tells me which folders are findable by python. I can get the opencv folders into the pythonpath within the python console, but back into the command line python will not find the folders.

If I recheck they are no longer listed. So it's not making a permanent change. It may be just one file.
Sent from my iPad

[Matthew Witherwax]

James,

Here is the output

['/root', '/usr/lib/python27.zip', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-linux2', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/lib/python2.7/site-packages']

In the screenshot, you can see OpenCV (cv) is in /usr/lib/python2.7/site-packages on my BBB.

[james....@btinternet.com]

Hi Matthew,

Thanks for the image as it shows exactly what I'm missing. Whatever I do with configuring python I'm not going to get a cv.py and cv.so without building it specifically. So adding opencv literally to the folder wasn't going to work. Build instructions for opencv for python a probably a must then. I don't think I have to worry about it too much.

I managed to spend the weekend reading up on neon optimizations. Writing assembly for neon proves to be a whole lot faster and I think that just using neon flags doesn't do a lot without code optimization. http://computer-vision-talks.com/2012/07/opencv-tutorial-part-4/

 

Shamefully there is one book that deals with neon optimizations but its seriously written for iphone/ipad. So the code examples would probably need some debugging to work which you can be really stranded if you don't know. Uncannycv reports some excellent optimizations. 

The issue being then is if opencv can run that much faster if optimized, is it worth taking the neon coding route, but the way the guy in the example has written his tutorial it still works as an addendum to the c++ code.  

[Tinashe Mudavanhu]

Firstly i would like to say i came across your discussion looking for select timeout error solution but as i went through it i didnt notice if you found a solution, but if you have it now, i would appreciate it. The select timeout error still seems to be in existence even on BBB Rev C with kernel  3.8.13-bone79. I am working on an Iris recognition system that initially has to track eyes in a live video stream (from there captures cropped eye images that will be processed). I have installed different OpenCV versions, 2.4.9, 2.4.11 and 3.0.0 and in all of them i am getting the same error. I am working with a Logitech, Inc. QuickCam E 3500 webcam. I am accessing the BBB Desktop using Tightvnc client on my PC running Ubuntu 14.04 LTS.

What is surprising is that it is capable of video streaming with opencv installed by apt-get (apt-get install python-opencv) but the limitation with this version is that it has very old cv bindings and documentation on some functions for Histogram equalisation is not available online. I am stuck, i need your help.

[Matthew Witherwax]

Hi Tinashe,

Please see my article here http://blog.lemoneerlabs.com/post/BBB-webcams

There is a version of framegrabber.c linked to it that allows you to specify the frame rate with the command line parameter -I.  If reducing the frame rate works for you, then the code for framegrabber should provide a starting point for accomplishing the same thing in your own program.

To view this discussion on the web visit https://groups.google.com/d/msgid/beagleboard/ecbcf781-d04a-4a40-b6d3-1a33e5e26267%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tinashe Mudavanhu]

Hi Matthew,

I looked into the article as i went through your discussion but did not try the framegrabber.c, will test it though. I finally got a solution to the problem from links listed below. It kind of made sense to me (lack of memory in ARM systems) because running the same code on my PC worked perfectly well. Running this command `sysctl vm.overcommit_memory=1` worked for me after a long struggle. What i'm not really sure are the implications (being it on Hardware or Software) if there comes a state when large size memory is really needed.


  [1]: https://www.raspberrypi.org/forums/viewtopic.php?f=31&t=17773
  [2]: https://tequals0.wordpress.com/2014/03/24/libv4l2-error-allocation-conversion-buffer-using-opencv-on-a-pi/

To view this discussion on the web visit https://groups.google.com/d/msgid/beagleboard/CAD6ZcAVYeJGAbrsrC2PK34S-5suznkhj53%3DoK%2Bg-YRX6O27G-w%40mail.gmail.com.

[Matthew Witherwax]

The issue causing the select time out detailed in my article has to do with how much data is being sent over the USB and how it is sent. Reducing the frame rate reduces the load on the USB. Looking at the articles you linked, one says it solved an issue but not the select timeout, and the other shows an error message that says it could not allocate memory.  Neither one of these are the cause of the select timeout I addressed.

The select timeout occurs when the select times out. Looks like in the cases in your links a previous call to allocate memory failed followed by select failing. I have never had an issue with memory allocation. All my troubles had to do with too much data on the USB. You might want to confirm what your actual problem is.

---

From: Tinashe Mudavanhu
Sent: ‎5/‎23/‎2016 4:58 PM
To: beagl...@googlegroups.com
Subject: Re: [beagleboard] Re: VGA camera capture

[The entire original message is not included.]

[Tinashe Mudavanhu]

Hi Matthew,

I'm a rookie in this linux/opencv area i wouldn't really know what it means, only learning from you. I posted a question https://groups.google.com/forum/#!category-topic/beagleboard/debian/VFuvveM_8Gc looking for a solution because it always happened when i plugged in the webcam on BBB. Running the command i previously mentioned ended my woes.

To view this discussion on the web visit https://groups.google.com/d/msgid/beagleboard/57438c64.87c30d0a.b3918.2ced%40mx.google.com.

[Tinashe Mudavanhu]

The timeout error resurfaced again, i followed you blog post on  http://blog.lemoneerlabs.com/3rdParty/Darling_BBB_30fps_DRAFT.html but got the following errors in compiling libjpeg-turbo ;

root@beaglebone:~/libjpeg-turbo-1.3.0/build# make
make  all-recursive
make[1]: Entering directory `/root/libjpeg-turbo-1.3.0/build'
Making all in java
make[2]: Entering directory `/root/libjpeg-turbo-1.3.0/build/java'
make[2]: Nothing to be done for `all'.
make[2]: Leaving directory `/root/libjpeg-turbo-1.3.0/build/java'
Making all in simd
make[2]: Entering directory `/root/libjpeg-turbo-1.3.0/build/simd'
make  all-am
make[3]: Entering directory `/root/libjpeg-turbo-1.3.0/build/simd'
make[3]: Nothing to be done for `all-am'.
make[3]: Leaving directory `/root/libjpeg-turbo-1.3.0/build/simd'
make[2]: Leaving directory `/root/libjpeg-turbo-1.3.0/build/simd'
Making all in md5
make[2]: Entering directory `/root/libjpeg-turbo-1.3.0/build/md5'
make[2]: Nothing to be done for `all'.
make[2]: Leaving directory `/root/libjpeg-turbo-1.3.0/build/md5'
make[2]: Entering directory `/root/libjpeg-turbo-1.3.0/build'
make[2]: Leaving directory `/root/libjpeg-turbo-1.3.0/build'
make[1]: Leaving directory `/root/libjpeg-turbo-1.3.0/build'
root@beaglebone:~/libjpeg-turbo-1.3.0/build#

Before compiling libjpeg-turbo i tested my code to see what will be happening on the resolution and frame rate. I am working with a Logitech QuickCam E3500 which has support for both YUVY and MJPEG. I set the resolution to 320x240 with a 30fps frame rate but after running the code, got select timeout errors and the resolution had changed to 640x480 YUVY with a 15fps frame rate;

root@beaglebone:~/libjpeg-turbo-1.3.0/build# v4l2-ctl -V
Format Video Capture:
    Width/Height  : 640/480
    Pixel Format  : 'YUYV'
    Field         : None
    Bytes per Line: 1280
    Size Image    : 614400
    Colorspace    : SRGB
root@beaglebone:~/libjpeg-turbo-1.3.0/build#

[João M. S. Silva]

Hi Tinashe,

My two cents:

> root@beaglebone:~/libjpeg-turbo-1.3.0/build# make
> make all-recursive
> make[1]: Entering directory `/root/libjpeg-turbo-1.3.0/build'
> Making all in java
> make[2]: Entering directory `/root/libjpeg-turbo-1.3.0/build/java'
> make[2]: Nothing to be done for `all'.
> make[2]: Leaving directory `/root/libjpeg-turbo-1.3.0/build/java'
> Making all in simd
> make[2]: Entering directory `/root/libjpeg-turbo-1.3.0/build/simd'
> make all-am
> make[3]: Entering directory `/root/libjpeg-turbo-1.3.0/build/simd'
> make[3]: Nothing to be done for `all-am'.
> make[3]: Leaving directory `/root/libjpeg-turbo-1.3.0/build/simd'
> make[2]: Leaving directory `/root/libjpeg-turbo-1.3.0/build/simd'
> Making all in md5
> make[2]: Entering directory `/root/libjpeg-turbo-1.3.0/build/md5'
> make[2]: Nothing to be done for `all'.
> make[2]: Leaving directory `/root/libjpeg-turbo-1.3.0/build/md5'
> make[2]: Entering directory `/root/libjpeg-turbo-1.3.0/build'
> make[2]: Leaving directory `/root/libjpeg-turbo-1.3.0/build'
> make[1]: Leaving directory `/root/libjpeg-turbo-1.3.0/build'
> root@beaglebone:~/libjpeg-turbo-1.3.0/build#

There is no error above.

> Before compiling libjpeg-turbo i tested my code to see what will be
> happening on the resolution and frame rate. I am working with a Logitech
> QuickCam E3500 which has support for both YUVY and MJPEG. I set the
> resolution to 320x240 with a 30fps frame rate but after running the
> code, got select timeout errors and the resolution had changed to
> 640x480 YUVY with a 15fps frame rate;

Due to this, among other reasons, we gave up on the BeagleBone Black and
are now using an A20-OLinuXino-LIME.

João M. S. Silva

[Matthew Witherwax]

There are no actual errors in the output you posted.

Do you have access to any other webcams?  I have tested this with the Logitech C920, the Logitech C270, and the PS3Eye.  Without access to your webcam, it is hard for me to look into what is going on.

[Tinashe Mudavanhu]

Hi,

The same is happening on the other webcam i have (not sure if its openCV changing resolution). I bought it on AliExpress and has the following discription; HD 1.3 megapixel 960p 2.8-12mm varifocal lens Aptina AR0130 android external usb camera module with IR cut ELP-USB130W01MT-FV. The supplier can be found on http://www.elpcctv.com.

Since there are no errors, what could be the problem? Is there some dependencies i have to install?

I tested with your framebrabber.c code and was getting this (see below) with the Logitech webcam,

root@beaglebone:~# time ./framegrabber -f mjpeg -H 480 -W 640 -c 1000 -I 30
Startup took 0.010000 seconds
Captured 1000 frames in 0.220000 seconds
Shutdown took 0.000000 seconds


real    0m42.068s
user    0m0.025s
sys    0m0.223s

The no name webcam was giving the following results;
root@beaglebone:~# time ./framegrabber -f mjpeg -H 480 -W 640 -c 1000 -I 30
Startup took 0.020000 seconds
Captured 1000 frames in 0.260000 seconds
Shutdown took 0.000000 seconds


real    2m10.965s
user    0m0.033s
sys    0m0.262s
root@beaglebone:~#

To view this discussion on the web visit https://groups.google.com/d/msgid/beagleboard/a0d097f6-2657-436a-9551-f8c6d4fb48c2%40googlegroups.com.

[Tinashe Mudavanhu]

Silva,

There has to be way. Dereck Molloy https://www.youtube.com/watch?v=8QouvYMfmQo was able to get a video stream then its possible to video stream with BBB. I don't have much time to look for another development board because the Project is due next week.

[Tinashe Mudavanhu]

I have flashed another Debian Wheezy image on my BBB. It comes with preinstalled opencv (version 2.3.1) but with very old cv bindings. I posted this in googlegroups as a solution which i mention or i dont see as a solution because i just avoided the problem. I am able to stream a video with both cameras. Whichever resolution i set the Logitech camera to, it converts to the one shown below;
root@beaglebone:~/Project/py_servo_facetracker# v4l2-ctl -V
Format Video Capture:
    Width/Height  : 160/120

    Pixel Format  : 'YUYV'
    Field         : None

    Bytes per Line: 320
    Size Image    : 38400
    Colorspace    : SRGB
root@beaglebone:~/Project/py_servo_facetracker#

The other webcam is set to a 1280x720 resolution with the pixel format set as YUYV.

I am suspecting it has something to do with the opencv itself when it grabs frames from the webcam. Is there a way i can set opencv to maintain the camera resolution i set it to?

[João M. S. Silva]

If the camera has a good compression mechanism (e.g. MJPEG), it may be
softer on the USB bus.

In our board, we are using OpenCV but we don't capture images with the
OpenCV instruction. Instead, we use something like this:

https://linuxtv.org/downloads/v4l-dvb-apis/v4l2grab-example.html

For video capturing there is also an example:

https://linuxtv.org/downloads/v4l-dvb-apis/capture-example.html

Note that OpenCV uses v4l underneath.

On 25/05/2016 16:51, Tinashe Mudavanhu wrote:
> Silva,
>
> There has to be way. Dereck Molloy
> https://www.youtube.com/watch?v=8QouvYMfmQo was able to get a video
> stream then its possible to video stream with BBB. I don't have much
> time to look for another development board because the Project is due
> next week.
>
> On Wed, May 25, 2016 at 5:48 PM, Tinashe Mudavanhu <tina...@gmail.com
> <mailto:tina...@gmail.com>> wrote:
>
> Hi,
>
> The same is happening on the other webcam i have (not sure if its
> openCV changing resolution). I bought it on AliExpress and has the
> following discription; HD 1.3 megapixel 960p 2.8-12mm varifocal lens
> Aptina AR0130 android external usb camera module with IR cut
> ELP-USB130W01MT-FV

> <http://www.aliexpress.com/item/HD-1-3-megapixel-2-8-12mm-varifocal-lens-android-external-usb-camera-module-with-IR/32295825258.html>.

> ------------------------------------------------------------------------

>>> <http://blog.lemoneerlabs.com>.

> <https://groups.google.com/d/msgid/beagleboard/ecbcf781-d04a-4a40-b6d3-1a33e5e26267%40googlegroups.com?utm_medium=email&utm_source=footer>.

> <https://groups.google.com/d/msgid/beagleboard/57438c64.87c30d0a.b3918.2ced%40mx.google.com?utm_medium=email&utm_source=footer>.

>
> For more options, visit
> https://groups.google.com/d/optout.
>
>
>
> --
> For more options, visit http://beagleboard.org/discuss
> ---
> You received this message because you are subscribed to a topic
> in the Google Groups "BeagleBoard" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/beagleboard/2NO62mGcSvA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email

> to beagleboard...@googlegroups.com
> <mailto:beagleboard...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/beagleboard/a0d097f6-2657-436a-9551-f8c6d4fb48c2%40googlegroups.com
> <https://groups.google.com/d/msgid/beagleboard/a0d097f6-2657-436a-9551-f8c6d4fb48c2%40googlegroups.com?utm_medium=email&utm_source=footer>.

>
> For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> For more options, visit http://beagleboard.org/discuss
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "BeagleBoard" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/beagleboard/2NO62mGcSvA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

> beagleboard...@googlegroups.com
> <mailto:beagleboard...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/beagleboard/CADw4j_vDXfQ-jHZop-Uk_Twx1wbZXKKPYYR5_zdGfUfcHLsabQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/beagleboard/CADw4j_vDXfQ-jHZop-Uk_Twx1wbZXKKPYYR5_zdGfUfcHLsabQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[Matthew Witherwax]

Looks like framegrabber was able to capture images based on the output you provided. Did you look at the images?

If it works for you then the code for the subscriber here http://blog.lemoneerlabs.com/post/bbb-mjpeg-streaming will show you how to load it into opencv.

---

From: João M. S. Silva
Sent: ‎5/‎25/‎2016 4:13 PM

[Tinashe Mudavanhu]

The 160x120 video stream had no latency but the other camera with 1280x720 was slow, video was displayed as images being flipped like book pages.

I need to understand something from your files. Does it mean the rest of my iris recognition code has to be in the framegrabber.py client file and also this file can also run on BBB? Do i need the zhelper.h file when my PC is running Ubuntu?

Here is my setup like. I have a PC running Ubuntu connected to the BBB via usb and also sharing internet via the same cable (there is no wired connection, only the PC is connected to wifi). I want to see the video stream on my BBB for debugging purposes and once it is working fine it doesn't have to show the video stream. I am accessing the BBB Desktop/GUI via a tightvnc client running on my Ubuntu PC. Thats when i will move to the hub for a wired connection so that everyone connected to the hub can access web pages served by the BBB (running lighttpd webserver).

I was successful in the part of running BBB as a webserver (without an video stream to show on the site because i don't need it) but then the first part of getting a video stream to show was not possible because of select timeout errors.

--

For more options, visit http://beagleboard.org/discuss
---
You received this message because you are subscribed to a topic in the Google Groups "BeagleBoard" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/beagleboard/2NO62mGcSvA/unsubscribe.

To unsubscribe from this group and all its topics, send an email to beagleboard...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beagleboard/574628d3.8847810a.511bf.33ec%40mx.google.com.

[Tinashe Mudavanhu]

Silva,

It does have MJPEG but opencv is converting the format to YUYV. And also does your board have I/O pins digital and analogue?

[Tinashe Mudavanhu]

I have finally managed to get a video stream (at least i can say for now) by changing the frame resolution inside the python opencv code using instructions on http://stackoverflow.com/questions/11420748/setting-camera-parameters-in-opencv-python. The maximum resolution i can set it to was 640x480 with a 5fps frame rate, increasing the frame rate will result in a select timeout message. At 320x240 i can set it  up to 30fps and running htop the CPU utilization will be fluctuating around 86-90%. The python code was taking about 12% and most of the chunk was on tightvnc about 60% (maybe because of the frame window being displayed).

This is the code i was running;

import cv2


#capture from camera at location 0
cap = cv2.VideoCapture(0)
#set the width and height, and UNSUCCESSFULLY set the exposure time
cap.set(3,320)
cap.set(4,240)
cap.set(5, 30)

while True:
    ret, img = cap.read()
    cv2.imshow("input", img)
    #cv2.imshow("thresholded", imgray*thresh2)

    key = cv2.waitKey(10)
    if key == 27:
        break


cv2.destroyAllWindows()
cap.release()

And also the resolution i forced the camera to in the opencv code is the one displayed after running v4l2-ctl -V;
root@beaglebone:~/Project/Testing# v4l2-ctl -V
Format Video Capture:
    Width/Height  : 320/240

    Pixel Format  : 'YUYV'
    Field         : None

    Bytes per Line: 640
    Size Image    : 153600
    Colorspace    : SRGB
root@beaglebone:~/Project/Testing#

What you figured out about bandwidth is really a serious issue. I have a friend who did not experience this problem on their RPi (only much higher resolutions resulted in select timeout messages, message because i don't think it is an error)  maybe because it has many USB ports meaning higher bandwidth compared to BBB with only one.

I wonder what will happen when connecting the BBB wirelessly, with a wifi dongle and camera on a USB hub.

[João M. S. Silva]

I suppose you were using OpenCV in C++. For setting the desired
resolution in C++ you have to do as you did in the Python code.
Something like: http://stackoverflow.com/a/24867771

You should also set it to MJPEG, if possible:
http://docs.opencv.org/2.4/modules/highgui/doc/reading_and_writing_images_and_video.html?highlight=cv_cap_prop_frame_width#videocapture-set

Please be more structured and succinct in what your current issue is so
that it's easier to help. Have you considered the code from
https://linuxtv.org/downloads/v4l-dvb-apis/capture-example.html?

Is your code in Python or C++?

In order to solve the larger problem you have to break it down into
smaller ones.

João M. S. Silva

> ------------------------------------------------------------------------
> From: João M. S. Silva <mailto:joao.m.sa...@gmail.com>

> Sent: ‎5/‎25/‎2016 4:13 PM
> To: beagl...@googlegroups.com

> <mailto:beagl...@googlegroups.com>

>
> Subject: Re: [beagleboard] Re: VGA camera capture
>
> If the camera has a good compression mechanism (e.g. MJPEG),
> it may be
> softer on the USB bus.
>
> In our board, we are using OpenCV but we don't capture
> images with the
> OpenCV instruction. Instead, we use something like this:
>
> https://linuxtv.org/downloads/v4l-dvb-apis/v4l2grab-example.html
>
> For video capturing there is also an example:
>
> https://linuxtv.org/downloads/v4l-dvb-apis/capture-example.html
>
> Note that OpenCV uses v4l underneath.
>
> On 25/05/2016 16:51, Tinashe Mudavanhu wrote:
> > Silva,
> >
> > There has to be way. Dereck Molloy
> > https://www.youtube.com/watch?v=8QouvYMfmQo was able to
> get a video
> > stream then its possible to video stream with BBB. I
> don't have much
> > time to look for another development board because the
> Project is due
> > next week.
> >
> > On Wed, May 25, 2016 at 5:48 PM, Tinashe Mudavanhu
> <tina...@gmail.com <mailto:tina...@gmail.com>

> > <mailto:tina...@gmail.com <mailto:tina...@gmail.com>>>

> <mailto:tina...@gmail.com>> wrote:
> >
> > Hi Matthew,
> >
> > I'm a rookie in this linux/opencv area i
> wouldn't really
> > know what it means, only learning from
> you. I posted a
> > question
> >
> https://groups.google.com/forum/#!category-topic/beagleboard/debian/VFuvveM_8Gc
> > looking for a solution because it always
> happened when i
> > plugged in the webcam on BBB. Running the
> command i
> > previously mentioned ended my woes.
> >
> > On Tue, May 24, 2016 at 1:04 AM, Matthew
> Witherwax
> > <able...@gmail.com

> <mailto:beagl...@googlegroups.com>

> <mailto:able...@gmail.com>> wrote:
> >
> > Hi Tinashe,
> >
> > Please see my article here
> > http://blog.lemoneerlabs.com/post/BBB-webcams
> > There is a version of
> framegrabber.c linked to
> > it that allows you to specify the
> frame rate
> > with the command line parameter
> -I. If reducing
> > the frame rate works for you,
> then the code for
> > framegrabber should provide a
> starting point for
> > accomplishing the same thing in
> your own program.
> >
> > On Mon, May 23, 2016 at 2:43 PM,
> Tinashe
> > Mudavanhu <tina...@gmail.com

> <james....@btinternet.com <mailto:james....@btinternet.com>>

> <james....@btinternet.com <mailto:james....@btinternet.com>>

> wrote:
> >>
> >>
> >>
> >>
>
> [The entire original message is not included.]
>
> --
> For more options, visit http://beagleboard.org/discuss
> ---
> You received this message because you are subscribed to a
> topic in the Google Groups "BeagleBoard" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/beagleboard/2NO62mGcSvA/unsubscribe.
> To unsubscribe from this group and all its topics, send an
> email to beagleboard...@googlegroups.com

> <mailto:beagleboard...@googlegroups.com>.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/beagleboard/574628d3.8847810a.511bf.33ec%40mx.google.com

> <https://groups.google.com/d/msgid/beagleboard/574628d3.8847810a.511bf.33ec%40mx.google.com?utm_medium=email&utm_source=footer>.

>
> For more options, visit https://groups.google.com/d/optout.
>
>
>
>

> --
> For more options, visit http://beagleboard.org/discuss
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "BeagleBoard" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/beagleboard/2NO62mGcSvA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> beagleboard...@googlegroups.com

> <mailto:beagleboard...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/beagleboard/CADw4j_twhZ1AYUFrXKmF2sgNomRgCoKZ%3D1xyKUxzBu0CoFxnag%40mail.gmail.com
> <https://groups.google.com/d/msgid/beagleboard/CADw4j_twhZ1AYUFrXKmF2sgNomRgCoKZ%3D1xyKUxzBu0CoFxnag%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[João M. S. Silva]

> It does have MJPEG but opencv is converting the format to YUYV. And also
> does your board have I/O pins digital and analogue?

It has lots of GPIO's, but I don't know if any of them are analog:
https://www.olimex.com/Products/OLinuXino/A20/A20-OLinuXino-LIME/open-source-hardware

João M. S. Silva