Problem with focus in specific cases

[Alexander Popov]

Hello.

I'm developing a CameraX app that takes pics of different angles of a smartphone. While taking a pic the camera has problems with autofocus. I have already tried to use tap to focus, but it also does not work properly.

For example, when I try to shot the top or bottom part of a smartphone CameraX usually set focus on the wrong object.

Picture 1. My CameraX app example 

On the picture 1 you can see result of tap to focus. The focus point was exact on the smartphone. 

The picture was shot on Pixel 2. Nevertheless, I have this problem on other devices as well.

However, I noticed that focus works better when the smartphone is placed horizontally. But in my case, I need to shoot the smartphone only vertically.

Picture 2. My CameraX app horizontal orientation example

My question is what can I do with the focus to make it work properly? 

I know there is no manual focus in CameraX yet. Is there a plan to add a manual focus in it soon?

Thanks in advance!

[Xi Zhang (张熹)]

Are you using PreviewView? What is the version of your camera-core and camera-view?

--
You received this message because you are subscribed to the Google Groups "Android CameraX Discussion Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camerax-develop...@android.com.
To view this discussion on the web visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/133edba8-d2cc-46a6-a10a-28129f271a22n%40android.com.

[Alexander Popov]

Yes, I'm using PreviewView, camera:camera-core1.0.0-beta08,  camera-view:1.0.0-alpha15". I also tested the last versions beta10 and alpha17 with one device with same result.

[Scott Nien]

Hi Alexander, 

I think the reason why it did not focus well is because the object is too near the camera and it's out of the focus capability.   You can try the built-in camera and see if it is not working either. 

Regarding manual focus,   so far we consider adding it after CameraX reaches RC but no confirmed plan yet. 

To view this discussion on the web visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/90aeaf78-fc11-421b-9f5e-36edaafa09cbn%40android.com.

[Alexander Popov]

Built-in-cameras usually focus better than CameraX does.

For example, this is Google Camera on Pixel 2 with tap to focus during same conditions:

Picture 1. GCam example

The main problem is that even with tap to focus CameraX cannot focus on proper object. And it's not only Pixel, I tested my app on some Samsung, Huawei, Sony and other vendor devices with same result.

And as I mentioned before if a device placed horizontally it works much better.

[Scott Nien]

Got it.  and while we investigate the issue further,  can you try to focus on something farther from the camera and see if focus on the right object ?  We want to know if the coordinate conversion is correct or not. 

[Alexander Popov]

Yes, I've tried to focus on the right and got correct focus:

The white circle is a focus position.

Just in case here is my tapToFocus code:

 private void tapToFocus(PreviewView cameraPreview) {

        cameraPreview.setOnTouchListener((view, motionEvent) -> {

            float x = motionEvent.getX();

            float y = motionEvent.getY();

            if (motionEvent.getAction() == MotionEvent.ACTION_UP) {

                float width = cameraPreview.getWidth();

                float height = cameraPreview.getHeight();

                MeteringPointFactory factory = new SurfaceOrientedMeteringPointFactory(width, height);

                MeteringPoint point = factory.createPoint(x, y);

                FocusMeteringAction action = new FocusMeteringAction.Builder(point, FocusMeteringAction.FLAG_AF)

                        .setAutoCancelDuration(5, TimeUnit.SECONDS)

                        .build();

                CameraControl cameraControl = camera.getCameraControl();

                ListenableFuture future = cameraControl.startFocusAndMetering(action);

                future.addListener( () -> {

                try {

                    FocusMeteringResult result = (FocusMeteringResult) future.get();

                } catch (Exception e) {

                    e.printStackTrace();

                }

            } , ContextCompat.getMainExecutor(requireContext()));

            }

            return true;

        });

}

[Scott Nien]

Found the problem.  You should use previewView.getMeteringPointFactory()  to create MeteringPoint instead of SurfaceOrientedMeteringPointFactory. 

SurfaceOrientedMeteringPointFactory is used to create a point from  a (x, y) in the "Surface coordinate" which aligns with the sensor.  I guess this is the reason why you focus correctly in landscape because in landscape 90 degree,  your view coordinates match the sensor coordinates. 

On the other hand,  previewView#getMeteringPointFactory will map your previewView (X, Y) into sensor coordinate correctly  regardless of the device rotation. 

[Alexander Popov]

Thanks, it helped. 

But unfortunately not for all the devices I have. The problem still remains for some Samsung devices such as S6 (SM-G920F), S6 EDGE (SM-925A),  S7 Edge (SCV-33). It works perfectly on S10 (SM-G973F) though.

[Scott Nien]

Can you try to focus on father objects (real objects like a cup / a book, not the background) on these problematic devices and see if it focuses correctly ?     The coordinates conversion is wrong in your original codes,  so I'd like to confirm  if the previewView#getMeteringPointFactory fixes the coordinates problems or not.  

If we did focus on the correct farther objects,  there is no way we focus incorrectly on near objects.   Then it must be the focus capability issues or the way we focus does differ (currently we trigger auto CONTROL_AF_TRIGGER in AF_AUTO modes,  but some camera apps keep in CONTROL_AF_MODE_CONTINUOUS_PICTURE mode). 

[Alexander Popov]

 I cannot focus to father objects because the camera is already focused on them.

After your words about AF modes I tried to change AF_MODE manually to CONTROL_AF_MODE_MACRO using this code:

Camera2Interop.Extender extender = new Camera2Interop.Extender(builder);
extender.setCaptureRequestOption(CaptureRequest.CONTROL_AF_MODE, CameraMetadata.CONTROL_AF_MODE_MACRO);

And it helped for these problematic devices, except S7 EDGE (SCV33). It seems to me you were right about CONTROL_AF_MODE_CONTINUOUS_PICTURE mode.

Should I use my solution? Is there another way to change AF mode?

[Scott Nien]

I mean you can focus on something else that is farther from the camera.  but it doesn't matter I guess because now the problem is likely the AF_MODE. 

Yes you can change the AF_MODE forcibly by using Camera2Interop API.   The request options you set have higher priority over what is set by CameraX.  But be mindful about :
(1) Not every device supports CONTROL_AF_MODE_CONTINUOUS_PICTURE ,  so you might want to query its availability by CameraCharacteristics.CONTROL_AF_AVAILABLE_MODES and probably fallback to CONTROL_AF_MODE_AUTO or OFF. 

(2) Camera2Interop is a experimental API and is subject to change.  Be prepared that the API could be changed in the future. 

[Alexander Popov]

Could you please provide a code example for getting AF modes by CameraCharacteristics.CONTROL_AF_AVAILABLE_MODES?

[Scott Nien]

ya. this part is tricky. 

You should first get a camera without use cases if you want to get these characteristics before opening camera,  and then use Camera2CameraInfo API to get the camera characteristics.   see below .   

// need to use latest camerax version,  beta10 
val camera = cameraProvider.bindToLifecycle(this, cameraSelector)  // no use cases here. 
val availableAfModes = Camera2CameraInfo.fromCameraInfo(camera!!.cameraInfo).getCameraCharacteristic(CameraCharacteristics.CONTROL_AF_AVAILABLE_MODES)

[Alexander Popov]

Thank you, Scott!

I assume I can't change AF Mode while camera is working, because I got this exception:

Option values conflicts: camera2.captureRequest.option.android.control.afMode, existing value (ALWAYS_OVERRIDE)=0, conflicting (ALWAYS_OVERRIDE)=1

[Scott Nien]

I wonder how you set the AF mode while the  camera is working ?   Current API does not allow that.  

But we will support a new camera2Interop API soon which allows you to set the parameters while the camera is working. 

Another thing you should know is if you override the AF mode to AF_AUTO,  this means it will not focus automatically , users have to manually focus on the objects (using startFocusAndMetering API). 

[Alexander Popov]

I tried to use this code for setting AF mode:

private void setAutoFocusMode(ExtendableBuilder<?> builder, int value) {

Camera2Interop.Extender extender = new Camera2Interop.Extender(builder);

extender.setCaptureRequestOption(CaptureRequest.CONTROL_AF_MODE, value);
}

 

private void startCamera() {

//some code

previewBuilder = new Preview.Builder();

...

}

I used a global variable for Preview.Builder and pass it as a parameter. 

setAutoFocusMode(previewBuilder, CameraCharacteristics.CONTROL_AF_MODE_AUTO);

I also tried to change focus distance for manual focus with the same negative result:

extender.setCaptureRequestOption(CaptureRequest.CONTROL_AF_MODE, CameraMetadata.CONTROL_AF_MODE_OFF);

extender.setCaptureRequestOption(CaptureRequest.LENS_FOCUS_DISTANCE, distance);

Okay, I'll be wating for new API.

That's interesting about AF_AUTO mode. How should I start automatic focus? Using CameraMetadata.CONTROL_AF_TRIGGER_START?

[Scott Nien]

ya, you cannot achieve what you want by setting parameters to the extender.  you have to wait for the new API. 

And the crash is actually caused by setting the same parameter twice.    Currently it is not allowed. 

[Scott Nien]

Give some update on this issue in case someone does not notice yet. 

CameraX now support apps to override any parameters through camera2 interop.  What's even better ,   you can now change the parameters dynamically using Camera2CameraControl API without the needs of unbind/bind again.   

Camera camera =  processCameraProvider.bindToLifecycle(..)  
Camera2CameraInfo camera2CameraInfo = Camera2CameraInfo.from(camera.getCameraInfo());
Camera2CameraControl camera2CameraControl = Camera2CameraInfo.from(camera.getCameraControl());

[Patrick Liao]

Hi everyone,

Sorry for being late to the party :)

In fact I was able to reproduce the same focus bug on my XIAOMI MI 10 Pro (Samsung GN1 CMOS) (As of December when my app was once a photography app)...

This bug ONLY occurs in 4:3 mode but NOT 16:9 ("widescreen") mode and I have no idea why this is happening.

 And I wasn't able to reproduce such bug on other Android devices me and my friends have, and the stock cameras/Google Camera port (Which i guess is still using camera2 under the hood) are working fine even in 4:3. 

Cheers,

Patrick Liao