Camera preview on Android using pyjnius

5,341 views
Skip to first unread message

Gully

unread,
Apr 10, 2013, 2:12:25 PM4/10/13
to kivy-...@googlegroups.com
Hi all,

I'm trying to run a simple camera preview application on Android. What I'm trying to do is basically this:

1. Create a widget based on the Image widget, with a new texture
2. Use the android's Camera class to open the camera
3. Create an android SurfaceTexture using my widget's texture ID
4. Set the SurfaceTexture as the preview texture using the Camera's setPreviewTexture

It took a while, but now I have a stable app, can start preview. Using the logs, I can see the camera object is updating frames, but nothing is displayed...

I've tried to call canvas.ask_update() from different locations but to no avail.

Also, tried to implement (using pyjnius) the camera's preview callback (although many examples say I don't have to) but that didn't work either. When I try to set the callback I get the following error (or at least I think it's related):
"W/dalvikvm(26697): Invalid indirect reference 0x5ded791c in decodeIndirectRef"

All of this is the first step before using the MediaRecorder class, since that requires even more tinkering, and it seems to me preview is the first thing I should be worried about...

Also, its worth mentioning the rest of the application is working well, but for now I've limited it to a button over the preview widget to start/stop the preview.

See below for the code.

Any ideas?

Thanks!


class CameraPreview(Image):
    play
= BooleanProperty(False)
    _camera
= None
    _previewCallback
= None
    _previewTexture
= None

   
def __init__(self, **kwargs):
       
super(CameraPreview, self).__init__(**kwargs)
       
self.texture = Texture.create(size=(480, 640))
       
self.texture_size = (480, 640)
       
self._camera = Camera.open()
       
self._previewTexture = SurfaceTexture(int(self.texture.id))
       
self._camera.setPreviewTexture(self._previewTexture)
       
# Tried using a preview callback (see jnius class further down) - I get a crash...
       
# self._previewCallback = CameraPreviewCallback(self._previewTexture)
       
# self._camera.setPreviewCallback(self._previewCallback)

   
def on_play(self, instance, value):
       
if not self._camera:
           
return
       
if value:
           
self._camera.startPreview()
       
else:
           
self._camera.stopPreview()

# pyjnius-wrapped class that implements the camera's preview callback interface: doesn't work :(
class CameraPreviewCallback(PythonJavaClass):
    __javainterfaces__
= ['android/hardware/Camera$PreviewCallback']
    preview_texture
= None

   
def __init__(self, preview_texture):
       
self.preview_texture = preview_texture

   
@java_method('(B[]Landroid/hardware/Camera;)V')
   
def onPreviewFrame(self, data, camera):
       
self.preview_texture.updateTexImage()




Gabriel Pettier

unread,
Apr 11, 2013, 6:04:29 AM4/11/13
to kivy-...@googlegroups.com
Hi

I actually tried the same route a few days ago, and got the same issue
of lack of display, turns out the kind of texture android expect there
is different from the one used in kivy internally, so you probably need
to find a way to extract the pixels from android texture to
create/update kivy texture, or find a different way to share the texture
between both. I didn't have time to research either of these direction
yet, but i'm interested in this, i'll try to get back to it in some
time, but if you find before i get to it, i'd love to see updates on it.

good luck :)
> --
> You received this message because you are subscribed to the Google Groups "Kivy users support" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kivy-users+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

Gully

unread,
Apr 11, 2013, 6:22:49 AM4/11/13
to kivy-...@googlegroups.com

Yeah, I figured out that android (or rather, the SurfaceTexture class) uses the TEXTURE_EXTERNAL_OES target, while Kivy uses the TEXTURE_2D target.

There are probably three options:
  1. Make Kivy use the TEXTURE_EXTERNAL_OES target - I don't think it's supported and I have no idea how to do that. Maybe use glGenTexture and glBindTexture explicitly instead of letting Kivy do it? will glBindTexture accept
    TEXTURE_EXTERNAL_OES or its enum number? (glGetIntegerv for the same value returns a key error - I have no idea why would Kivy validate this parameter...)
  2. Use the Camera class' PreviewCallback to grab the preview data, convert it and blit it to Kivy's texture (or simply blit it and use a shader) - can't get pyjnius to work on that one
  3. Per frame, find the active texture unit (using some java callback? don't know which as well as can't get callbacks to work), pass the ID to a shader and use it to display the data - this seems like the fastest option, but I have no idea if it can be done :(
I'd appreciate thoughts on how to pursue any of these directions... :)

Gully

unread,
Apr 11, 2013, 9:43:21 AM4/11/13
to kivy-...@googlegroups.com
Progress!

I've decided to take the first route, and now I can get a stable (although not ideal) preview.
Still working on the image orientation, stretching etc.

I'd appreciate any comments on the implementation or the code. I'm still working on improving and optimizing it, but as my knowledge in Kivy and OpenGL is sketchy, I'm sure there might be better ways of doing what I do.
Eventually, I'd like to package it in a new Kivy Camera implementation for android, as well as extend my code to function as a video recorder.

In a nutshell, I've done the following (not in order, see the comments in the code below)
  1. Created texture manually, with a GL_TEXTURE_EXTERNAL_OES (36197) target and a new texture ID. Note this is an OpenGL extension that is not supported in Windows (as far as I know).
  2. Use a RenderContext as canvas, with a fragment shader that simply samples the new texture.
  3. Add a BindTexture instruction to the canvas and set a canvas parameter to the shader so the shader will have access to the new texture
  4. Add a Callback instruction to the canvas.before so that we will have an entry point before something is rendered. We will use this to make sure the SurfaceTexture actually updates the texture since this has to be done from the right OpenGL context.
  5. Schedule a clock callback to repeatedly ask for canvas updates. This is required since Kivy doesn't know the SurfaceTexture is ready with a new frame, and we have to poke it so our canvas.before callback will be called. This is ugly and should be replaced with the SurfaceTexture's onFrameAvailable callback, but that's still not working :)
  6. Create the camera and attach a new SurfaceTexutre initialized using the new texture ID.

from jnius import autoclass, PythonJavaClass, java_method
Camera = autoclass('android.hardware.Camera')
SurfaceTexture = autoclass('android.graphics.SurfaceTexture')

GL_TEXTURE_EXTERNAL_OES
= 36197


class CameraPreview(Image):
    play
= BooleanProperty(False)

    resolution
= ListProperty([640, 480])

    _camera
= None
    _previewCallback
= None
    _previewTexture
= None

    secondary_texture
= None

   
def __init__(self, **kwargs):
       
self.canvas = RenderContext()
       
super(CameraPreview, self).__init__(**kwargs)
       
self.bind(size=self.size_changed)

       
# (2)
       
self.canvas.shader.fs = '''
            #extension GL_OES_EGL_image_external : require
            #ifdef GL_ES
                precision highp float;
            #endif

            /* Outputs from the vertex shader */
            varying vec4 frag_color;
            varying vec2 tex_coord0;

            /* uniform texture samplers */
            uniform sampler2D texture0;
            uniform samplerExternalOES texture1;

            void main()
            {
                gl_FragColor = texture2D(texture1, tex_coord0);
            }
        '''

       
# This is needed for the default vertex shader.
       
self.canvas['projection_mat'] = Window.render_context['projection_mat']

       
with self.canvas.before:
           
# (4)
           
Callback(self.draw_callback)
       
with self.canvas:
           
# (3)
           
BindTexture(texture=self.secondary_texture, index=1)
       
self.canvas['secondary_texture'] = 1

       
# (1)
        tex_id
= kivy.graphics.opengl.glGenTextures(1)[0]
        kivy
.graphics.opengl.glBindTexture(GL_TEXTURE_EXTERNAL_OES, tex_id)
        width
, height = self.resolution
       
self.secondary_texture = Texture(width=width, height=height, target=GL_TEXTURE_EXTERNAL_OES, texid=int(tex_id), colorfmt='rgba')
       
# (6)
       
self._camera = Camera.open()
       
self._previewTexture = SurfaceTexture(int(tex_id))
        self._camera.setPreviewTexture(self._previewTexture)

   
def draw_callback(self, instr):
       
if self._previewTexture:
           
self._previewTexture.updateTexImage()

   
def config_camera(self, surface):
       
self._camera.setPreviewTexture(surface)

   
def update_canvas(self, dt):
       
self.canvas.ask_update()

   
def size_changed(self, *largs):
       
pass


   
def on_play(self, instance, value):
       
if not self._camera:
           
return
       
if value:
           
self._camera.startPreview()

           
# (5)
           
Clock.schedule_interval(self.update_canvas, 1.0/30)
       
else:
           
Clock.unschedule(self.update_canvas)
           
self._camera.stopPreview()

Mathieu Virbel

unread,
Apr 12, 2013, 5:10:52 AM4/12/13
to kivy-...@googlegroups.com
\o/

Last time i've checked, i was telling to gabriel that the last piece missing was to change a shader. You just did it, and it works.

The implementation / approach is the right way to do it. Creating a new rendercontext and set the shader is the right way. Initialization and attachment of the camera is what gabriel and i did last time we checked.

Some remarqs:
 - you don't need to seperate canvas.before/canvas, put the callback in the canvas
- you can generate texture id directly with tex=Texture(..., target=GL_TEXTURE_EXTERNAL_OES). Then bind() the texture (the texture will be created internally, and set few options), then you can use tex.id for retreving the texture id.

All the rest is great. Be careful when you'll try to get the onFrameAvailable, it might not be the same thread as the main/ui thread. You might also stress / lock the GIL more than needed (if you have 60 camera FPS, but your app is running at lower FPS (slow), the java part will still call your python callback. The GIL will act as a lock for both thread.) Scheduling is OK for now, but you can still compare the perfs with both approach.

Good job,

Mathieu


Gully

unread,
Apr 12, 2013, 6:47:29 AM4/12/13
to kivy-...@googlegroups.com
Thanks :)

I'm currently working on the orientation - it seems the Camera's setDisplayOrientation and Parameters.setRotation do not affect the preview results. It might be that when dumping data directly to a SurfaceTexture no processing is made - I'll update if UI have any results.

Gully

unread,
Apr 12, 2013, 6:53:48 AM4/12/13
to kivy-...@googlegroups.com
By the way, as for rotation - what would be the best way to rotate the texture by 90 degrees before coordinates reach the fragment shader? I've tried applying a different projection matrix but didn't get anywhere...

Davide Rosa

unread,
May 9, 2013, 2:13:21 AM5/9/13
to kivy-...@googlegroups.com
Hi Gully,

I'm developing an application for android with Kivy. I'm interested in this "widget" that provides camera functionalities. Have you some updates? Is it possible also to store one picture from camera?

Thank you.

Gully

unread,
May 9, 2013, 4:36:04 AM5/9/13
to kivy-...@googlegroups.com
Hi Davide,

Sure. I've been a bit busy, so I haven't been able to update - I'll ask one of the guys here to strip down our latest code and send an example for a media recorder and camera preview. We haven't tried capturing a single image, but I'm sure it's quite simple once you have a working example.

Mathieu Virbel

unread,
May 9, 2013, 4:39:42 AM5/9/13
to kivy-...@googlegroups.com
Capturing a single image should'nt use a full camera provider, but the
Intent ACTION_IMAGE_CAPTURE instead.

http://developer.android.com/reference/android/provider/MediaStore.html#ACTION_IMAGE_CAPTURE



Le 09/05/2013 10:36, Gully a �crit :

Davide Rosa

unread,
May 9, 2013, 5:34:11 AM5/9/13
to kivy-...@googlegroups.com
Ok, right. The standard Intent method should be the best way, so probably the integration inside a peview widget could be the wrong way. But I've tried a lot to capture a picture with the intent, and I had different problems:
   1. The application freezes for long time after the end of the Intent task;
   2. It does not return any value in extra data;
   3. There is no possibility to make changes in the intent in order to customize the view.

So I would like to use the preview widget. :-)


Il giorno giovedì 9 maggio 2013 10:39:42 UTC+2, Mathieu Virbel ha scritto:
Capturing a single image should'nt use a full camera provider, but the
Intent ACTION_IMAGE_CAPTURE instead.

http://developer.android.com/reference/android/provider/MediaStore.html#ACTION_IMAGE_CAPTURE



Le 09/05/2013 10:36, Gully a �crit :

Chen

unread,
May 9, 2013, 1:42:39 PM5/9/13
to kivy-...@googlegroups.com
Here is the AndroidCameraPreview widget Gully and I are working on. I believe it should be easy enough to use the camera object (_camera) to take pictures (e.g. with takePicture). Note it assumes a vertical orientation. You might want to set "keep_ratio: True" in the kv file so it will keep ratio. Also be sure to set the camera ID before playing.

import kivy
kivy
.require('1.6.0')

from kivy.app import App
from kivy.clock import Clock
from kivy.uix.image import Image
from kivy.properties import BooleanProperty, ListProperty, NumericProperty, StringProperty
from kivy.graphics import RenderContext, Callback, BindTexture
from kivy.graphics.texture import Texture
from kivy.core.window import Window
from kivy.logger import Logger
import sys

GL_TEXTURE_EXTERNAL_OES
= 36197

IsAndroid = sys.platform == 'linux3'  

if IsAndroid:
   
from jnius import autoclass, cast

   
# MediaRecorder Java classes

   
Camera = autoclass('android.hardware.Camera')

   
Surface = autoclass('android.view.Surface')

   
SurfaceTexture = autoclass('android.graphics.SurfaceTexture')

   
MediaRecorder = autoclass('android.media.MediaRecorder')
   
AudioSource = autoclass('android.media.MediaRecorder$AudioSource')
   
VideoSource = autoclass('android.media.MediaRecorder$VideoSource')
   
OutputFormat = autoclass('android.media.MediaRecorder$OutputFormat')
   
AudioEncoder = autoclass('android.media.MediaRecorder$AudioEncoder')
   
VideoEncoder = autoclass('android.media.MediaRecorder$VideoEncoder')
   
CamcorderProfile = autoclass('android.media.CamcorderProfile')

class AndroidCameraPreview(Image):
   
"""
    Android camera preview widget. Extends Image with a custom shader that
    displays textures of target GL_TEXTURE_EXTERNAL_OES. Note, this requires
    Android API 11 because of it's use of SurfaceTexture.
    """

    play
= BooleanProperty(False)
    resolution
= ListProperty([720, 720])
    camera_id
= NumericProperty(-1)


    _camera
= None
    _previewCallback
= None
    _previewTexture
= None

    _previewSurface
= None
    _secondary_texture
= None


   
def __init__(self, **kwargs):
       
self.canvas = RenderContext()

       
super(AndroidCameraPreview, self).__init__(**kwargs)
       
self.bind(resolution=self._resolution_changed)
       
self.bind(camera_id=self._camera_id_changed)
       
self.bind(play=self._play_changed)


       
# This is needed for the default vertex shader.
       
self.canvas['projection_mat'] = Window.render_context['projection_mat']


       
with self.canvas:  
           
Callback(self._draw_callback)
           
BindTexture(texture=self._secondary_texture, index=1)
       
self.canvas['secondary_texture'] = 1
       
self._init_texture(self.resolution)

   
def start(self, play=True):
       
self._init_camera(self.camera_id)
       
self.play = play

   
def stop(self):
       
self._release_camera()

   
def _init_texture(self, resolution):
        width
, height = resolution
       
# Image looks at self.texture to determine layout, so we create
       
# texture but don't actually display it.
       
self.texture = Texture.create(size=(height, width))
       
self.texture_size = self.texture.size
       
if IsAndroid:
           
self._secondary_texture = Texture(width=height, height=width, target=GL_TEXTURE_EXTERNAL_OES, colorfmt='rgba')
           
self._secondary_texture.bind()
           
self._previewTexture = SurfaceTexture(int(self._secondary_texture.id))

   
def _init_camera(self, camera_id):
       
Logger.info('Init camera %d' % camera_id)
       
if self._camera and self.camera_id == camera_id:
           
return
       
self._release_camera()
       
if IsAndroid:
           
self._camera = Camera.open(camera_id)
            parameters
= self._camera.getParameters()            

           
#print parameters.flatten()
           
if parameters is None:
               
Logger.warning('Can''t read parameters')
               
return

            supportedSizes
= parameters.getSupportedVideoSizes()
           
if supportedSizes is None:
                supportedSizes
= parameters.getSupportedPreviewSizes()
           
            sizes
= []
           
if supportedSizes is not None:
                  iterator
= supportedSizes.iterator()
                 
while iterator.hasNext():
                      cameraSize
= iterator.next()
                      sizes
.append((cameraSize.width, cameraSize.height))

            pickedSize
= self._pick_optimal_size(sizes)

           
# Update texture according to picked size
           
self.resolution = list(pickedSize)

            parameters
.setPreviewSize(*pickedSize)
           
self._camera.setParameters(parameters)
           
self._camera.setPreviewTexture(self._previewTexture)

   
def get_camera(self):
       
return self._camera

   
def _resolution_changed(self, instance, value):
       
Logger.info('Resolution changed to ' + str(value))
       
self._init_texture(value)

   
def _camera_id_changed(self, instance, value):
       
Logger.info('Changed camera %d' % value)
       
if not IsAndroid:
           
return

       
# Transform orientation based on selected camera
       
if self.camera_id == 0:
            sampleVec
= ('1.0-tex_coord0.y','1.0-tex_coord0.x')
       
elif self.camera_id == 1:
            sampleVec
= ('tex_coord0.y','1.0-tex_coord0.x')
       
else:
           
raise Exception('Invalid camera id')

       
self.canvas.shader.fs = '''

            #extension GL_OES_EGL_image_external : require
            #ifdef GL_ES
                precision highp float;
            #endif

            /* Outputs from the vertex shader */
            varying vec4 frag_color;
            varying vec2 tex_coord0;

            /* uniform texture samplers */
            uniform sampler2D texture0;
            uniform samplerExternalOES texture1;

            void main()
            {
                // Flip & Mirror coordinates to rotate source texture by 90 degress
                gl_FragColor = texture2D(texture1, vec2(%s,%s));
            }
        '''
% (sampleVec[0], sampleVec[1])

   
def _play_changed(self, instance, value):
       
Logger.info('camera %d _play_changed %d' % (self.camera_id, value))
       
if not IsAndroid:
           
return

       
if value:
           
if not self._camera:
               
self._init_camera(self.camera_id)        
           
self._camera.startPreview()
           
Clock.schedule_interval(self._update_canvas, 1.0/30)
       
else:
           
if self._camera:
               
Clock.unschedule(self._update_canvas)
               
self._camera.stopPreview()

   
def _release_camera(self):
       
self.play = False
       
if self._camera:
           
self._camera.release()
           
self._camera = None

   
def _pick_optimal_size(self, sizes):        
       
# Now just returns the one guaranteed support size
       
return sizes[0]

   
def _draw_callback(self, instr):
       
if self._previewTexture:
           
self._previewTexture.updateTexImage()

   
def _update_canvas(self, dt):
       
self.canvas.ask_update()


Enjoy!

Davide Rosa

unread,
May 9, 2013, 2:31:09 PM5/9/13
to kivy-...@googlegroups.com

Hi Chen, thank you a lot for the code. I will test it. ;-)

You received this message because you are subscribed to a topic in the Google Groups "Kivy users support" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kivy-users/hiyAJqjhFjo/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to kivy-users+...@googlegroups.com.

Mathieu Virbel

unread,
May 9, 2013, 3:50:37 PM5/9/13
to kivy-...@googlegroups.com

Do you think you could create a garden package for it with doc?

Akshay Arora

unread,
May 9, 2013, 4:12:28 PM5/9/13
to kivy-...@googlegroups.com
Just in case you are not aware of it Kivy-garden is a new section of Kivy. Details about it can be found here http://kivy-garden.github.io

Best Regards

Adam Griffiths

unread,
May 13, 2013, 8:41:32 AM5/13/13
to kivy-...@googlegroups.com
I've just come across the 'camera not working' on Android myself.
Requesting specific resolutions also crashes the application. Ie, Camera is broken on Android.

Is there any reason this isn't being fixed in Kivy itself?

Cheers,
Adam

Davide Rosa

unread,
May 14, 2013, 4:15:16 AM5/14/13
to kivy-...@googlegroups.com
Hi Chen and Gully,

I have tested the new widget and it works, the code quality is very good. There is only one think that should be fixed shorly, about the preview size. Actually it is used the zero index size available for the Android Camera class, and this causes (for Nexus7 device) a very small preview (under 100x100 pixels). I temporarly fixed this calculating the requested preview area and setting the most near available preview area size. 

And I have a question. How can I do to make the preview rendered inside a scatter widget, allowing the user to move, rotate and stretch the live image? This is possible with the standard Camera widget.

Thank you.

Chen

unread,
May 15, 2013, 3:06:06 PM5/15/13
to kivy-...@googlegroups.com
Yes you are correct about the preview size, we just extend and override _pick_optimal_size with a heuristic function fitting our specific needs.

Your question is an important one, and it relates to an issue I raised in a separate topic (Updating a Widget's transformation when rendering with RenderContext). Mathieu was quick to provide a good solution, but I haven't gotten around to test it yet...

Remy

unread,
May 16, 2013, 8:40:54 AM5/16/13
to kivy-...@googlegroups.com

Hello Chen,

I am currently working on a project where I need to access the android camera to compare two photos.  My idea was to take pictures one or two seconds apart and compare them.
To do this, I have to be able to take pictures using Kivy, but I'm having some trouble getting it to work.  I have been looking at the "takePicture" function to take pictures, however I'm not getting it to work at the moment.
From the Android API reference I saw that this function requires a ShutterCallback, RAW PictureCallback and a JPG PictureCallback.  I can't import these using the autoclass function, so I'm not sure what I should be doing here..  I am fairly sure that like 2 hours ago I somehow had these callbacks imported in my Python file, but I for the life of me can't get it to work again..

 When calling takepicture like this:

            cam = AndroidCameraPreview(camera_id=0)
            cam.play=True
            cam._camera.takePicture(None, None, self.PictureCallback())
            cam.stop()

 ...

 self.PictureCallback(self):
     return

Nothing seems to happen, the app doesn't crash nor does it actually take a picture. The ADB logs show:

V/CameraWrapper(  163): camera_preview_enabled
V/CameraWrapper(  163): camera_preview_enabled->41E086C8->41E08720
V/CameraWrapper(  163): camera_preview_enabled
V/CameraWrapper(  163): camera_preview_enabled->41E086C8->41E08720
V/CameraWrapper(  163): camera_set_preview_window->41E086C8->41E08720
V/CameraWrapper(  163): camera_start_preview
V/CameraWrapper(  163): camera_start_preview->41E086C8->41E08720
I/python  (16972): [DEBUG  ] Taking picture
V/CameraWrapper(  163): camera_enable_msg_type->41E086C8->41E08720
V/CameraWrapper(  163): camera_enable_msg_type
V/CameraWrapper(  163): camera_take_picture
V/CameraWrapper(  163): camera_take_picture->41E086C8->41E08720
D/NvOsDebugPrintf(  163): [CAM] [RAWCHIP] rawchip initialize.
D/NvOsDebugPrintf(  163): [CAM] [RAWCHIP] rawchip initialize done.
D/NvOsDebugPrintf(  163): [CAM] NvCameraSensorPowerOn: --
D/NvOsDebugPrintf(  163): [CAM]NvCameraPowerOnThread: --
D/NvOsDebugPrintf(  163): NvIspAfGetConvergeStatus, return status=2
D/NvOsDebugPrintf(  163): ENC_T3:Encoder_Open ++
I/ActivityManager(  552): Process com.endomondo.android (pid 16901) has died.
I/ActivityManager(  552): Process com.google.android.inputmethod.latin.dictionarypack (pid 16776) has died.
I/ActivityManager(  552): Process com.android.keychain (pid 16806) has died.
D/NvOsDebugPrintf(  163): ENC_T3:OutputThread ++
D/NvOsDebugPrintf(  163): ENC_T3:OutputThread Before Semaphore Wait
D/NvOsDebugPrintf(  163): ENC_T3:Encoder_Open --
D/NvOsDebugPrintf(  163): ENC_T3:FeedInputThread ++
D/NvOsDebugPrintf(  163): [CAM] StartStillCapture, mUsingFlash=0, NslNumberOfImages=0
D/NvOsDebugPrintf(  163): [CAM] CameraBlockExtension,StillCaptureActive=1
D/NvOsDebugPrintf(  163): [CAM] Selected Sensor Mode: 3280x2464 FrameRate: 21.0 StreamIndex: 2
I/ActivityManager(  552): Process com.android.voicedialer (pid 16840) has died.
E/Camera  (16972): Unknown message type 32771
I/python  (16972): [DEBUG  ] Stopping camera
V/CameraWrapper(  163): camera_disable_msg_type->41E086C8->41E08720
V/CameraWrapper(  163): camera_disable_msg_type
V/CameraWrapper(  163): camera_disable_msg_type->41E086C8->41E08720
V/CameraWrapper(  163): camera_disable_msg_type
V/CameraWrapper(  163): camera_stop_preview
V/CameraWrapper(  163): camera_stop_preview->41E086C8->41E08720
V/CameraWrapper(  163): camera_cancel_picture
V/CameraWrapper(  163): camera_cancel_picture->41E086C8->41E08720
V/CameraWrapper(  163): camera_release
V/CameraWrapper(  163): camera_release->41E086C8->41E08720
D/NvOsDebugPrintf(  163): Waiting on hOutstandingRequestsServedSema failed 0x5
D/NvOsDebugPrintf(  163): [CAM]NvMMCameraClose: ++
D/NvOsDebugPrintf(  163): [CAM] NvMMCameraBlockPrivateClose NvCameraAp15Cleanup, release camera
D/NvOsDebugPrintf(  163): NvCameraSyncPointThread is terminated.
D/NvOsDebugPrintf(  163): [CAM]NvMMCameraClose: --
D/NvOsDebugPrintf(  163): ENC_T3:Encoder_Close ++
D/NvOsDebugPrintf(  163): ENC_T3:Encoder_Close SW Encoder : 0
D/NvOsDebugPrintf(  163): ENC_T3:OutputThread After Semaphore Wait, CloseEncoder : 1
D/NvOsDebugPrintf(  163): ENC_T3:OutputThread --
D/NvOsDebugPrintf(  163): ENC_T3:FeedInputThread --
D/NvOsDebugPrintf(  163): ENC_T3:Encoder_Close --
V/CameraWrapper(  163): camera_set_preview_window->41E086C8->41E08720
I/CameraClient(  163): Destroying camera 0
V/CameraWrapper(  163): camera_device_close


So.. Could you point me in the right direction of how to handle these callbacks and actually make the "takePicture" function work?
Or if you happen to have an idea on how I can more easily compare two images taken from the camera, I'm all ears.

Thanks,
-Remy
 

Remy

unread,
May 16, 2013, 8:48:38 AM5/16/13
to kivy-...@googlegroups.com
Oh, by the way..
If I don't stop the camera straight away, I get the following lines:

D/NvOsDebugPrintf(  163): NvIspAfGetConvergeStatus, return status=2
D/NvOsDebugPrintf(  163): ENC_T3:OutputThread After Semaphore Wait, CloseEncoder : 0
D/NvOsDebugPrintf(  163): ENC-DBG jpegencoder 0x4137ec78 , ptr for image 0x421d8003, hMem for 2d 0xc1f202e0
D/NvOsDebugPrintf(  163): ENC_T3:OutputThread Before Semaphore Wait
D/NvOsDebugPrintf(  163): ENC_T3:OutputThread After Semaphore Wait, CloseEncoder : 0
W/CameraClient(  163): lockIfMessageWanted(2): dropped unwanted message

D/NvOsDebugPrintf(  163): ENC_T3:OutputThread Before Semaphore Wait

It does seem to return a pointer for the image, so I'm assuming a photo is being made?

Chen

unread,
May 17, 2013, 9:06:28 AM5/17/13
to kivy-...@googlegroups.com
Hi Remy,
I'm not an android Camera expert, but it does seem that you are passing a callback in a wrong way (calling it...). Regardless, from what I understand, pyjnius doesn't support python callbacks at this moment anyway.
So using takePicture is not as straightforward as I imagined it to be. If in fact there is no synchronous way to take still pictures in Android (again, not an expert), then it seems the only way is to create a callback in Java to store the result somewhere, while polling it in python (e.g. using the kivy clock). Calling custom Java code from python is not as scary as it sounds (easiest/hackiest thing is to add it to PythonActivity.java). If anybody else has a better solution, please share.

Davide Rosa

unread,
May 17, 2013, 9:21:48 AM5/17/13
to kivy-...@googlegroups.com
No other solutions. It's the only way as I have seen.

The problem is Android and it's system structure. It's needed the development of a software layer directly over the c/c++ library of the entire system. It's a wall, the only way to pass through it, is using JAVA! damn

Gabriel Pettier

unread,
May 17, 2013, 6:58:03 PM5/17/13
to kivy-...@googlegroups.com
I didn't really look more into the issue, but pyjnius offer the
possibility to implement java interfaces in python, so you can define
your callback in a python implementation of the right java interface,
and have it be called by the java side when needed.

Sergey Kabanov

unread,
Oct 7, 2013, 11:42:13 AM10/7/13
to kivy-...@googlegroups.com
Hi, guys. I like your solution very much.
Is there a repository or something to get the latest version?

Gully

unread,
Oct 10, 2013, 10:41:57 AM10/10/13
to kivy-...@googlegroups.com
Hi,

Not yet - we're kind of under a deadline here to get things finished, but I am planning on posting a complete example when I'm done...

Andy Betsworth

unread,
Jun 7, 2014, 7:01:38 AM6/7/14
to kivy-...@googlegroups.com
Hi  I'm wondering if you found time to post an example anywhere?  I'm struggling to implement the takePicture using pyjnius and PythonJavaClass.  I'm getting strange errors and the callbacks don't even seem to be called in the correct order.  I'm getting None as the data in the raw picture callback, and the shutter callback somehow seems to come after that!

Any advise would be much appreciated.

Gully

unread,
Jun 7, 2014, 9:29:28 AM6/7/14
to kivy-...@googlegroups.com
Hi Andy,

The raw callback isn't supported on all devices/hardware, so don't get surprised it you get it with data=None, and/or out of order.

I've stripped down some of my code to make it simpler and am attaching a short example for a camera preview class. Not perfect, but I hope it helps.
My code has two possible handlers for the JPEG callback,m one for simply saving it to file, the other for loading it into a Texture - I currently use the first one but left the other one in just in case.

Please note, my example relies on some way of getting a surface holder from the PythonActivity instance in order to do the preview. In my case, I've patched PythonActivity.java as follows:
  • PythonActivity class should implement SurfaceHolder.Callback (implements SurfaceHolder.Callback)
  • In onCreate(), call this function before creating the SDLSurfaceView instance (make sure you add the required member variables to the PythonActivity class):
protected void onCreateBeforeSDLSurface() {
        mFrameLayout = new FrameLayout(this);
        mCameraView = new SurfaceView(this);
        mCameraView.setZOrderOnTop(false);
        mCameraView.setFocusable(false);
        mCameraSurfaceHolder = mCameraView.getHolder();
        mCameraSurfaceHolder.addCallback(this);
        mCameraSurfaceHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
        mFrameLayout.addView(mCameraView);
        mCameraView.setVisibility(View.VISIBLE);
    } 
 
  • Call this function instead of setContentView(mView), pass mView to the function:
    protected void onCreateAfterSurface(SurfaceView mView) {
        mView.setZOrderOnTop(true);    // necessary
        mView.getHolder().setFormat(PixelFormat.TRANSLUCENT);
        mFrameLayout.addView(mView);
        setContentView(mFrameLayout);
    }
  • Implement the SurfaceHolder.Callback interface:
    public void surfaceCreated(SurfaceHolder holder) {
        mCameraSurfaceReady = true;
    }

    public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
    }

    public void surfaceDestroyed(SurfaceHolder holder) {
        mCameraSurfaceReady = false;
    }
  • Add this function: 
    public SurfaceHolder getCameraSurfaceHolder() {
        if (mCameraSurfaceReady)
            return mCameraSurfaceHolder;
        return null;
    }

Hope it helps :)

Let me know if you have any issues or if I missed anything.

camerapreview.py

Julien Coron

unread,
Nov 1, 2014, 1:48:20 PM11/1/14
to kivy-...@googlegroups.com
Hi Gully,

I'm not so experimented to use your code as this. Is there a new sample usage of the live preview camera of an Android phone (using kivy) ?

I've tried "plyer" but it only runs the photo application of the phone to take a picture. I can not interact with what the camera sees (and add a layer over it).


Sébastien Ramage

unread,
Jul 22, 2015, 11:13:16 AM7/22/15
to Kivy users support
Hi !
I'm trying to use your code, but the app crash (I'm using Android 4.4.4)
Maybe your code need update ? How can I help ?

Seb

TomK

unread,
Feb 28, 2017, 11:13:50 AM2/28/17
to Kivy users support
Hi Gulli, hi Chen,

me and a friend of mine are trying to develop an application which takes photos and needs a customized access to the camera options (e.g. for choosing a dedicated directory). Since the standard Camera API for Android does not offer this option, we were very happy to find your solution.

We wrote a simple app, which uses your recent posted solution and which should start the camera, when a button ("Init") is pressed. Unfortunately the application closes instead of starting the camera. Thus we tried to get a clue concerning the reason for this behaviour via the logs and error messages. The result is, that the compiled APK (via buildozer) crashes because the "_imaging.so"-file is not found. Despite the fact, that some threads concerning this issue exist and I invested several hours by now (I updated all required/used libraries and tried several proposals for solutions), I could not fix the problem.
 
I would really appreciate if you could give me a hint on whats the matter behind my problem. Did anyone face this problem too?

For a better understanding of my specific problem, I attached the main script, the corresponding kv file as well as the buildozer.spec-file to this message.

Thanks a lot in advance
Tom
buildozer.spec
main.py
main.kv

julc...@gmail.com

unread,
Dec 27, 2018, 10:42:48 AM12/27/18
to Kivy users support

bump!!! need access to camera from Android......

or alternatively access to the picture gallery

can we make a crowdfound to support this piece of functionality ?



On Wednesday, 10 April 2013 20:12:25 UTC+2, Gully wrote:
Hi all,

I'm trying to run a simple camera preview application on Android. What I'm trying to do is basically this:

1. Create a widget based on the Image widget, with a new texture
2. Use the android's Camera class to open the camera
3. Create an android SurfaceTexture using my widget's texture ID
4. Set the SurfaceTexture as the preview texture using the Camera's setPreviewTexture

It took a while, but now I have a stable app, can start preview. Using the logs, I can see the camera object is updating frames, but nothing is displayed...

I've tried to call canvas.ask_update() from different locations but to no avail.

Also, tried to implement (using pyjnius) the camera's preview callback (although many examples say I don't have to) but that didn't work either. When I try to set the callback I get the following error (or at least I think it's related):
"W/dalvikvm(26697): Invalid indirect reference 0x5ded791c in decodeIndirectRef"

All of this is the first step before using the MediaRecorder class, since that requires even more tinkering, and it seems to me preview is the first thing I should be worried about...

Also, its worth mentioning the rest of the application is working well, but for now I've limited it to a button over the preview widget to start/stop the preview.

See below for the code.

Any ideas?

Thanks!


class CameraPreview(Image):
    play
= BooleanProperty(False)

    _camera
= None
    _previewCallback
= None
    _previewTexture
= None


   
def __init__(self, **kwargs):
       
super(CameraPreview, self).__init__(**kwargs)
       
self.texture = Texture.create(size=(480, 640))
       
self.texture_size = (480, 640)
       
self._camera = Camera.open()
       
self._previewTexture = SurfaceTexture(int(self.texture.id))
       
self._camera.setPreviewTexture(self._previewTexture)
       
# Tried using a preview callback (see jnius class further down) - I get a crash...
       
# self._previewCallback = CameraPreviewCallback(self._previewTexture)
       
# self._camera.setPreviewCallback(self._previewCallback)


   
def on_play(self, instance, value):
       
if not self._camera:
           
return
       
if value:
           
self._camera.startPreview()

       
else:
           
self._camera.stopPreview()

# pyjnius-wrapped class that implements the camera's preview callback interface: doesn't work :(
class CameraPreviewCallback(PythonJavaClass):
    __javainterfaces__
= ['android/hardware/Camera$PreviewCallback']
    preview_texture
= None

   
def __init__(self, preview_texture):
       
self.preview_texture = preview_texture

   
@java_method('(B[]Landroid/hardware/Camera;)V')
   
def onPreviewFrame(self, data, camera):
       
self.preview_texture.updateTexImage()




Reply all
Reply to author
Forward
0 new messages