UVC camera YUYV pixel format support in android applications

[Nikko Torcita]

Hi all,

I've been trying to support generic UVC cameras on android and have been successful so far with getting the native camera app to display preview images until I tested my code with skype video call and got stuck with a buffer memory problem. To break it down,

1. I'm using the yuv422i-yuyv format since it's the only pixel format supported by the uvc camera (Vimicro chipset) that i'm testing. I have also set CameraService::Client::registerPreviewBuffers() to support HAL_PIXEL_FORMAT_YCbCr_422_I in CameraService.cpp.

2. I set the size of my preview heap to 'PreviewHeap = width x height x 2' since YUV422 takes up 4 bytes per 2 pixels.  

From logcat, I get this: 

D/CameraHardware(28343): PREVIEW SIZE: w=320 h=240 framerate=30

E/CameraHardware(28343): virtual android::status_t android::CameraHardware::startPreview() :

D/V4L2Camera(28343): USB Camera now streaming...

D/libEGL  (28734): loaded /system/lib/egl/libGLES_android.so

D/libEGL  (28734): loaded /vendor/lib/egl/libEGL_POWERVR_SGX540_120.so

D/libEGL  (28734): loaded /vendor/lib/egl/libGLESv1_CM_POWERVR_SGX540_120.so

D/libEGL  (28734): loaded /vendor/lib/egl/libGLESv2_POWERVR_SGX540_120.so

E/Camera-JNI(28734): Manually set buffer was too small! Expected 153600 bytes, but got 115201!

E/Camera-JNI(28734): Manually set buffer was too small! Expected 153600 bytes, but got 115201!

E/Camera-JNI(28734): Manually set buffer was too small! Expected 153600 bytes, but got 115201!

W/com.skype.je(28734): not showing

I traced it down to the camera jni and it seems like somewhere above the HAL, a buffer was set to accommodate only up to 'width x height x 1.5'. Hence, 320x240x1.5 = 115200 instead of 320x240x2 = 153600. Could it be that the skype application only provided support for NV21 (YUV420) which takes up 'width x height x 1.5' bytes of data?

Thank you in advance.

Nikko Torcita

[Alexy]

Hi,

have you tried color converting and giving 420SP (NV21) buffers? Its
possible that Skype expects the camera to be giving frames in NV21
format.

-Alexy

On May 2, 7:55 am, Nikko Torcita <nikkotorc...@gmail.com> wrote:
> Hi all,
>
> I've been trying to support generic UVC cameras on android and have been
> successful so far with getting the native camera app to display preview
> images until I tested my code with skype video call and got stuck with a
> buffer memory problem. To break it down,
>
> 1. I'm using the yuv422i-yuyv format since it's the only pixel format
> supported by the uvc camera (Vimicro chipset) that i'm testing. I have also
> set CameraService::Client::registerPreviewBuffers() to support
> HAL_PIXEL_FORMAT_YCbCr_422_I in CameraService.cpp.
>
> 2. I set the size of my preview heap to 'PreviewHeap = width x height x 2'
> since YUV422 takes up 4 bytes per 2 pixels.
>
> From logcat, I get this:
>
> D/CameraHardware(28343): PREVIEW SIZE: w=320 h=240 framerate=30
> E/CameraHardware(28343): virtual android::status_t
> android::CameraHardware::startPreview() :
> D/V4L2Camera(28343): USB Camera now streaming...
> D/libEGL  (28734): loaded /system/lib/egl/libGLES_android.so
> D/libEGL  (28734): loaded /vendor/lib/egl/libEGL_POWERVR_SGX540_120.so
> D/libEGL  (28734): loaded /vendor/lib/egl/libGLESv1_CM_POWERVR_SGX540_120.so
> D/libEGL  (28734): loaded /vendor/lib/egl/libGLESv2_POWERVR_SGX540_120.so

> *E/Camera-JNI(28734): Manually set buffer was too small! Expected 153600
> bytes, but got 115201!*
> *E/Camera-JNI(28734): Manually set buffer was too small! Expected 153600
> bytes, but got 115201!*
> *E/Camera-JNI(28734): Manually set buffer was too small! Expected 153600
> bytes, but got 115201!*

[Vishal Bhoj]

Hi Nikko,

The color conversion needs to be done before passing the frames through the callback(CAMERA_MSG_PREVIEW_FRAME) as the yuv420sp is the default color format supported by android application while accessing preview frames from application layer.

Have a look at lines from 209-215 in

http://android.git.linaro.org/gitweb?p=hardware/linaro/common.git;a=blob;f=libcamera/CameraHardware.cpp;h=8c90fa4a591b2c0a43d0cbe42f656937785708ee;hb=1ad0e928edf6f046572b3f32eb5ee23d8d7f3f79

here is  the camera hal which I have tested with skype,linphone,google hangouts kind of applications,you can look at it as reference.
http://android.git.linaro.org/gitweb?p=hardware/linaro/common.git;a=summary

Regards,
Vishal

Nikko Torcita

--
unsubscribe: android-porti...@googlegroups.com
website: http://groups.google.com/group/android-porting

[Nikko Torcita]

Hi guys,

Thanks for all the inputs. I converted my YUYV frames to YUV420SP right before passing it to the preview callback and it worked! I am now able to use the usb camera for skype video calls. I just find it useless for google to put other pixel formats in the CameraService.cpp if they're only going to support YUV420SP in the first place.

Now that I have usb camera support, I'm faced with another challenge. The device that I'm developing for has a built-in camera and uses the samsung camera interface unit (cpu is Samsung S5PV210). Is it possible for the CameraService to support two HAL implementations? One for the usb camera and one for the built-in camera?

[Eino-Ville Talvala]

This isn't a question of Google support; a third-party application has to specifically code in support for a given YUV format; Skype chose to use NV21, and configured the interface accordingly (here, by leaving the preview format as the default, which is NV21).  There's no way for Skype to automatically handle YUY2, especially when it asked for NV21 and was given YUY2 instead. Writing YUV processing code is fairly tedious, so nobody wants to support more formats than they have to.

Since HC, a camera HAL is required to support both NV21 and YV12 (these are the API-level names; yuv420sp and yuv420p are the HAL-level names, IIRC) for preview callbacks; any other formats are optional. As a result, most 3rd-party apps should pick one of those two formats to maximize cross-device compatibility, and I wouldn't expect many apps to be able to deal with YUY2.

To answer your question, currently only one HAL module per subsystem can exist.  You'd have to create a new subsystem class (like 'usb_camera') and then teach CameraService how to access it separately from the main camera HAL module. In theory, it's not terribly difficult since you want to keep the same interface to both; but you'd probably have to duplicate/refactor a lot of code in the camera service to do so.  Or, you can add the USB support directly into the main camera HAL module, listing the USB devices after the fixed cameras.

The main difficulty is that USB cameras are likely to be dynamic, which isn't a concept the current camera API really has facilities for dealing with. You can probably get something working well enough as long as you don't unplug/plug cameras in while a camera-using app is running, but a solid solution would require new app-level API support, I think.

Regards,

Eddy Talvala

Nikko Torcita

--

unsubscribe: android-porting+unsubscribe@googlegroups.com
website: http://groups.google.com/group/android-porting

[mrAlmond]

Hi everyone,

I'm trying to do the same thing for a tablet running Gingerbread 2.3.1.
With this Android version is not possible to port that mods to do the conversion as the CameraHal interface is different (probably an older version).

For example there are no methods like mRequestMemory and types like camera_memory_t :

camera_memory_t* picture = mRequestMemory(-1, framesize, 1, NULL);

Can someone give me some help to perform the colorspace conversion the previewThread method for my Android version?
Thank you very much!

mrA

[Benjamin Bui]

Hi Eddy,

Has been long time since your post. We have been success to support uvc webcam in our board but facing problem with hot unplug while camera app is opening as you pointed out here. Could you suggest more detail how to solve this problem? How we can modify camera API to support this? Or at least, how we can make everything work again after user close camera application.

Many thanks.

BR.

[Marco Iacobelli]

Hi, this is my first post.

I'm trying a camera porting on android 2.3 . I've customize CameraHardware.cpp and v4l2camera.cpp following this example https://gitorious.org/0xdroid/hardware_omap3_camera/source/b564bbeee68a33c92013955e11deba7605385a15: but nothing happens.

Can anyone help me whit this porting? my ideas are so confused

tnxs

Nikko Torcita

--

unsubscribe: android-porti...@googlegroups.com
website: http://groups.google.com/group/android-porting