How to set auto continuous focus?

[江澎涌]

Hello CameraX Group,

When I use CameraX video recode , how can I make it auto continuous focus (AF and AWB).

[Scott Nien]

Hi , 

By default CameraX will set the following parameters if devices support it.  

CONTROL_AF_MODE_CONTINUOUS_PICTURE (or CONTROL_AF_MODE_CONTINUOUS_VIDEO depending on video is enabled or not). 
CONTROL_AE_MODE_ON ( or CONTROL_AE_MODE_ON_ALWAYS_FLASH / CONTROL_AE_MODE_ON_AUTO_FLASH depending on the flash mode)

CONTROL_AWB_MODE_AUTO

On Fri, Dec 24, 2021 at 9:12 AM 江澎涌 <jpyn...@gmail.com> wrote:

Hello CameraX Group,

When I use CameraX video recode , how can I make it auto continuous focus (AF and AWB).

--
You received this message because you are subscribed to the Google Groups "Android CameraX Discussion Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camerax-develop...@android.com.
To view this discussion on the web visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/4bf3b213-8c8d-43e7-863e-b00e9e76b89an%40android.com.

[江澎涌]

Thank you for the reply.

Is there an api to set this parameters? When I use camerax to record, I focus a point and then move the phone slow, but it did't auto continuous focus, cause the preview page to be blurred.

I have three phones (HUAWEI P30 Pro、XiaoMi 6、Samsung S10) . However all of them have this problem. 

[江澎涌]

Hi, Scott.

I use the code to search current af_mode value and the return value is null.

```
val camera2CameraControl = Camera2CameraControl.from(cameraControl)
val captureRequestOptions = camera2CameraControl.captureRequestOptions
val captureRequestOption =captureRequestOptions.getCaptureRequestOption(CaptureRequest.CONTROL_AF_MODE)
// the captureRequestOption value is null
Log.e(CameraFragment.TAG, "captureRequestOption: $captureRequestOption")
```

And then, I try to set `CONTROL_AF_MODE_CONTINUOUS_PICTURE` parameters via this code.

```
val camera2CameraControl = Camera2CameraControl.from(camera!!.cameraControl)
camera2CameraControl.captureRequestOptions = CaptureRequestOptions.Builder()
    .setCaptureRequestOption(
        CaptureRequest.CONTROL_AF_MODE,
        CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE
    )
    .build()
```

It will make the `startFocusAndMetering` api doesn't work, which always show an error like this.

```
java.util.concurrent.ExecutionException: androidx.camera.core.CameraControl$OperationCanceledException: Cancelled by another startFocusAndMetering()
```

And the preview page seem doesn't auto continuous foucs. How can I do? 

[Scott Nien]

That's expected. When you issue a focusAndMetering (CameraControl#startFocusAndMetering),  it will switch to AF_AUTO mode instead of continuous focus ,  set the AF region and trigger a manual scan. 

To fix the issue, one possible solution is to call CameraControl#cancelFocusAndMetering when you detect the device movement. 

On the CameraX side, we will implement this feature in the future. 

And also, using Camera2CameraControl (Camera2CameraInterop) to override AF_MODE will interfere with CameraControl API and cause some of the APIs to fail. 

[Patrick Liao]

Hi Pengyong,

startFocusAndMetering() can be thought of something like "ONE SHOT" focus in DSLR -- that is, when you supply a focus coordinate to it, it will try to focus to the point and exit the routine entirely (i.e. not the kind of "continuous focus" that you are expecting if I understand your question correctly). To achieve servo focusing (i.e. continuously trying to focus on a certain coordinate, the thing you are asking for), you may need to write a loop for sending the same coordinate over and over again to startFocusAndMetering(). Here's my implementation and it worked for me.

Basically, I first send a focus coordinate to startFocusAndMetering() and wait for it to finish focusing (which is done by the ListenableFuture stuff in my code). If I wanted the original coordinate to be a FOCUS_SERVO point, then the looper thread would fire another startFocusAndMetering() using the same coordinate and repeats such routine until I ask my code to switch back to FOCUS_AUTO or pass in a FOCUS_SINGLE focus coordinate.

Hope this helps. 

Best,

Patrick Liao

[Patrick Liao]

And IIRC,  CONTROL_AF_MODE_CONTINUOUS_VIDEO is set instead of  CONTROL_AF_MODE_CONTINUOUS_PICTURE when a Recording session (from the new camera-video library) is active.

On Sunday, December 26, 2021 at 9:19:34 PM UTC-5 Scott Nien wrote:

[Scott Nien]

Thanks Patrick for the sharing.  

You are right about focusAndMetering being a one shot focus.  Do you think that having an option to keep continuous AF mode in the startFocusAndMetering API will be helpful ?  

[Patrick Liao]

Yes and no. Having continuous AF mode built-in would be nice, but it would really complicate the API as continuous AF involves more variables to tweak:

1. How frequent should we run the AE/AWB algorithms to avoid unwanted flickering especially in low light situations?

2. How fast should we track the objects when the focus system isn't reliable (e.g. low light, lack of contrast around the focus coordinate, cameras using IR for focusing instead of PDAF)?

    (My current approach does not take this into account and simply calls startFocusAndMetering() again whenever the previous call finishes, and it would simply go nuts when the AF system can't focus on the same distance consistently in low light)

3. What if the user wishes to manually adjust the focus tracking speed and sensitivity if, say, the user wishes to configure it in a way such that a car/person/spider-man running across the frame does not throw off the AF system? 

    (This feature exists in most modern DSLR/mirrorless cameras)

4. What if the user wants to modify the size of the AF area on the fly for face tracking?

And the list goes on...

So packing all of the stuff from above into startFocusAndMetering() would really complicate things, I guess.

[Scott Nien]

To clarify, 

I was referring to keeping AF_MODE to CONTROL_AF_MODE_CONTINUOUS_PICTURE while AF/AE/AWB regions are set (instead of setting to AF_AUTO, and trigger AF for once) ,  the change is simple and definitely not complicated. 

The difference would be 

For keeping in CONTROL_AF_MODE_CONTINUOUS_PICTURE,  it will keep triggering auto focus on the given coordinates regardless of the focal distance. 

For switching to AF_AUTO and one time AF trigger (current behavior),  it will keep the current AF setting (e.g, focal distance) , so when you move the camera after startFocusAndMetering, only the object of similar distance from the camera will be focused. 

The only problem would be if keeping CONTROL_AF_MODE_CONTINUOUS_PICTURE mode is really a useful feature. What is the benefits for any possible scenarios ?

[Patrick Liao]

Thanks for the clarification.

I think such feature is not useful for still image capture (at least not yet since most phones still can't have very shallow depth of field when the focal distance is higher than 1 meter) but for video recording -- say the user is shooting vlogs and wants to put the focus on the dead center of the frame so that the AF system does not keep looking around the frame for the closest/largest object to focus on (if it is set to AF_AUTO) and always locks focus on the object that the user wants, avoiding unwanted lens effects (i.e. focus breathing --this is becoming more apparent on newer phones featuring bigger aperture and larger CMOS).  

This is a habit many videographers have when they shoot videos on their mirrorless cameras (specifically, putting a large AF zone on the center of the frame).

Hope this helps.

[Scott Nien]

Thanks for sharing. Your experience definitely can help us shape the API to better serve the applications.  

Please let us know if you have any specific suggestions regarding how to change CameraX behavior or adding features to better support video capture to avoid the focus breathing problem.