Android Sample 3 Camera Overlay Problem Revisited

[Dmitry]

Hi,

I'm going to revisit the issue of the third sample not working right.
Basically, calling setPreviewDisplay with null prevents the callback
from being called. I've read many messages about how "some" devices
are not doing the "right thing", but google's api docs clearly state:
* Pass a fully initialized SurfaceHolder to
setPreviewDisplay(SurfaceHolder). Without a surface, the camera will
be unable to start the preview.

I just got a Samsung Galaxy Nexus, which uses stock Android 4.0.2 and
this example will simply not work on it (it does work on an Asus
Transformer (Android 3.2)) . I can't use VideoInput, because I want to
set some parameters for the camera (Android 4.0.2 added a few more
options).

What would be the best way to get this example to work as intended?
I've been trying various "fixes" for the last few days and am out of
ideas.

--Dmitry

[Dmitry]

Semi-answered my own question. Using an idea from another thread:
http://stackoverflow.com/questions/2933882/how-to-draw-an-overlay-on-a-surfaceview-used-by-camera-on-android
. I didn't realize that I could create an xml layout, setContentView
to it and THEN update setZOrderMediaOverlay in the onCreate method
(false for the "fake" preview surface and true from every other view
you want to be on top of it). Anyway, not the cleanest solution, but
works.

[Nexus2k]

I don't get it, I got the same problem that my App doesn't launch
preview because of "NULL ANativeWindow was passed to setPreviewWindow;
Preview not started. Preview in progress flag set."
I basically try to start the face-detection Sample on my Galaxy Nexus.
Could you explain your solution in more detail ?

On 20 Dez., 16:41, Dmitry <dmitry....@gmail.com> wrote:
> Semi-answered my own question. Using an idea from another thread:http://stackoverflow.com/questions/2933882/how-to-draw-an-overlay-on-...

[Dmitry]

Ok, so what you have to do is have a valid SurfaceView for the camera
preview to start, but you can't just hide it after passing it to
setPreviewDisplay() because the surface will detect that and will stop
previewing. So...

I got the inspiration from the following posts:
Create a FrameLayout to overlay two surfaces
http://stackoverflow.com/questions/2933882/how-to-draw-an-overlay-on-a-surfaceview-used-by-camera-on-android

Then change the Z order to put the real surface on top:
http://stackoverflow.com/questions/5778647/how-to-properly-use-setzordermediaoverlay-on-android

Putting it all together.

The "Tutorial 2 Advanced" example in the SDK instantiates the
Sample3View class inside the onCreate function. I moved that out to
layout xml:

main.xml:
<?xml version="1.0" encoding="utf-8"?>

<FrameLayout xmlns:android="http://schemas.android.com/apk/res/
android"
android:layout_width="fill_parent"
android:layout_height="fill_parent">

<!-- the fake surface view has to start on top in order for the
preview to start correctly -->
<SurfaceView android:id="@+id/fakeCameraView"
android:layout_width="wrap_content"
android:layout_height="wrap_content" />

<!-- the actual opencv drawing class -->
<org.opencv.samples.tutorial3.Sample3View
android:id="@+id/cvsurface"
android:layout_width="wrap_content"
android:layout_height="wrap_content" />

</FrameLayout>



The following constructors have to be added (ones that accept the
layout xml attributes) to:
Sample3View.java
public Sample3View (Context context, AttributeSet attrs) {
super(context, attrs);
}

and

SampleViewBase.java for this to work:
public SampleViewBase(Context context, AttributeSet attrs) {
super(context, attrs);
mHolder = getHolder();
mHolder.addCallback(this);
Log.i(TAG, "Instantiated new " + this.getClass());
}

Then update Sample3Native onCreate as follows

- setContentView(new Sample3View(this));
+ setContentView(R.layout.main);
+ SurfaceView fakeview = (SurfaceView)
this.findViewById(R.id.fakeCameraView);
+ fakeview.setZOrderMediaOverlay(false); // This is the magic call
that states things may be drawn over this surface
+ SurfaceView trueview = (SurfaceView)
this.findViewById(R.id.cvsurface);
+ trueview.setZOrderMediaOverlay(true); // This is the magic call that
tells this surface it's an overlay

Finally, the SampleViewBase.java needs to use the now hidden preview:
+ SurfaceView fakeview = (SurfaceView) ((View)
getParent()).findViewById(R.id.fakeCameraView);
try {
- mCamera.setPreviewDisplay(null);
+ mCamera.setPreviewDisplay(fakeview.getHolder());

My guess is that this works because setZOrderMediaOverlay s a way to
partially draw over a camera preview canvas for augmented reality
(that's the context of the hints I found), in this case we completely
cover the camera preview SurfaceView.

There are probably more elegant ways to do this where the "fake"
preview canvas is actually a custom class that instantiates the camera
and and then the SampleViewBase.java class just needs to take care of
the drawing. However, this was the quickest way that I could get the
samples to work.

Best,
Dmitry Kit

[Agnés Borràs]

Thanks!!

[Zhou Xue]

Hi

I followed your instructions but the main.xml keeps reporting the cannot find the sampleview class. The app will crush unless I put the trueview before the fakeview.

I use the fakeview holder for display but it never works....

I am stuck with the problems for several days and I hope you may help me.

thank you in advance

[Geomod]

Well finally I managed to run this, based on the Dmitry code modification, I needed to modify some things in order to get it working.


First you need to add in the constructors in the SampleViewBase.java the following line:

mHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);

so the constructors read:

public SampleViewBase(Context context) {
        super(context);
        mHolder = getHolder();
        mHolder.addCallback(this);
        mHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);

        Log.i(TAG, "Instantiated new " + this.getClass());
    }
   
    public SampleViewBase(Context context, AttributeSet attrs) {
        super(context, attrs);
        mHolder = getHolder();
        mHolder.addCallback(this);

        mHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);

        Log.i(TAG, "Instantiated new " + this.getClass());
    }

I also modified this in order to catch the exception throwed if the problem is in the mCamera.startPreview(); metod:

try{
            Log.i(TAG, "start preview");
            mCamera.startPreview();
            Log.i(TAG, "started preview");
           } catch (RuntimeException e){Log.e(TAG, "mCamera.startPreview() failed");}

Then in the run() section I added this line to create the trueview object:

SurfaceView trueview = (SurfaceView) ((View)getParent()).findViewById(R.id.cvsurface);


then you should modify this in order to get a canvas (I was getting null without this):

Canvas canvas = trueview.getHolder().lockCanvas();
             
                if (canvas != null) {
                   
                    canvas.drawBitmap(bmp, (canvas.getWidth() - getFrameWidth()) / 2, (canvas.getHeight() - getFrameHeight()) / 2, null);
                   
                    trueview.getHolder().unlockCanvasAndPost(canvas);

and that's all, the app works just fine.

I'm sure I made a lot of mistakes with this code and if someone could correct that would be nice.

It took around 2 weeks to figure this but well.

I recommend you to use the Log.i(TAG, "whatever"); in oder to know where is your program and what is working and after where it crashed or anything like that.

I managed at first to process all but showing only the stream from the camera, then I realized the problem was with canvas getting null.

Greetings.

[Dmitry]

I haven't really had to deal with this problem since I solved it, but
since then I was told that a MUCH easier way to do this is to do the
following:
"
For other samples you can open SampleViewBase.java and replace
commented lines with the uncommented one:
//try {
//mCamera.setPreviewDisplay(null);
//} catch (IOException e) {
// Log.e(TAG, "mCamera.setPreviewDisplay fails: " + e);
//}
mCamera.setPreviewTexture(new SurfaceTexture(10));
"

Now I haven't personally tried this, but it does makes sense. If
anyone does try it, please update this thread so the entire solution
is preserved here.

Thanks

[Geomod]

Well Ican't try it since SurfaceTExture was added for API 11 (Honeycomb) and I don't have tablet or device with supprt for that API or greater.

BTW

Sometimes for some strange reason the preview doesn't start (something eclipse realted) , then I just switch the place between Surfaces in the main.xml , and then it works.

[Padmanabha]

Hi,

I have been facing similar problem on Honeycomb.

public class MyView extends SurfaceView {
public MyView(Context context) {
super(context);
SurfaceHolder sh=getHolder();
Surface s=sh.getSurface();
Log.d("MyView",s.toString());
}
}

The surface here is coming out to be NULL as seen in below logs:

D/MyView ( 6009): Surface(name=null, identity=-1)
D/Camera ( 6009): app passed NULL surface
D/demoCamera( 6009): setPreviewDisplay
E/CameraHal ( 1584): setPreviewWindow - NULL ANativeWindow passed to
setPreviewWindow
E/CameraHal ( 1584): startPreview - Preview not started. Preview in
progress flag set

Is this a platform specific issue. Why would the constructor fail.

[Jacky Le]

Dear Geomod,
Could you share to me your sample project(already fix problem). In my sample don't have layout and main.xml file.
Tks

[Geomod]

Ok.
I can't post the complete project because I didn't backup it and I started to modify some functions and now it's not working, however I have a backup from the source code which is at the link.

Regarding main.xml, the orginal project doesn't have a main.xml so you must create a file, wich should be inside a folder called "layout" which must be inside the "res" folder so the structure should look like this:
 
       Tutorial 2 Advanced - 1. Add Native OpenCV/res/layout/main.xml

In order to create a subfolder in eclipse just right click on the parent folder and go to new and click folder.

You can download the other source from here and I can upload the apk so you can check if mine runs in your phone.

code: http://www.box.com/s/f5381aa0687dfc364157

app: http://www.box.com/s/0588a69a8646d293da50

Hope it helps and feel free to contact me. If I can help I will do it.

BTW: I just achieved to modify and run that sample because the other ones have some user interfaces things (buttons, menus, etc) and I'm not very good yet in java, android and xml so I couldn't change the other apps.

Also the files have some anotations for my use so some it can throw some errors in adb but are just for info.

If the code that I posted didn't work for you try to switch the lines of the surfaces in the main.xml because sometimes eclipse make some changes that makes the same working code useless.


Greetings.

[Antoine LAURENT]

Thank you very  much !

I've spent the last week trying to make these samples work on an TF201. Your solution works perfectly in my case.

[Putria Febriana]

its doesnt work fine on my project

[Putria Febriana]

can you give me full of source project?thanks

[Putria Febriana]

09-22 17:33:15.593: E/AndroidRuntime(12990): java.lang.RuntimeException: Unable to start activity ComponentInfo{org.opencv.samples.tutorial3/org.opencv.samples.tutorial3.Sample3Native}: android.view.InflateException: Binary XML file line #14: Error inflating class org.opencv.samples.tutorial3.Sample3View
this is your solution error

On Tuesday, December 20, 2011 6:16:25 AM UTC-8, Dmitry wrote:

[Jorge Narvaez]

Sorry for taking so long to answer but I have been really busy.

To Putria: I suggest you to compile the samples before making any change on the code. All the samples should compile (only the native ones could fail if you don't installed all to compile C/C++ code).

To PW: Well thanks for your words, but the work was almost done, I just put the stuff together.

 I think your problem may be on the xml. check that the names of classes on the file match the classes you're using, that happened to me once.

Greetings.

2012/9/30 pw cloete <pwv...@gmail.com>

Oh yeah, I maybe should've put in where the program fails....

It already fails in the onCreate function, just before setContectView(R.layout.main); I also tried swopping the surfaces in the main.xml file like you suggested, although making no difference :(

--