About startFocusAndMetering()

[Patrick Liao]

Hi,

I am interested in implementing face tracking in my android devices (and always lock the focus on their faces). May I know how frequent could startFocusAndMetering() receive requests and process them properly? I personally haven't test it yet and not sure if Google designed this API to be one-shot AF and not continuous "servo AF" like the ones Canon DSLRs...

Cheers,

Patrick Liao

[Scott Nien]

Hi Patrick, 
For now, startFocusAndMetering will switch to auto-focus mode for one shot auto-focus action (setting CONTROL_AF_TRIGGER to CONTROL_AF_TRIGGER_START). And I think the frequency allowed is varied among different devices and we don't have the measurements for you to reference.  

Hope this helps.  And we are open to any feedback on how to improve the API going forward. 

--
You received this message because you are subscribed to the Google Groups "Android CameraX Discussion Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camerax-develop...@android.com.
To view this discussion on the web visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/0b9e7feb-f871-48aa-8840-0aee333cc7a1n%40android.com.

[Scott Nien]

To be precisely,  I mean switching to CONTROL_AF_MODE_AUTO mode instead of CONTROL_AF_MODE_CONTINUOUS_PICTURE when doing startFocusAndMetering.  And it will go back to CONTROL_AF_MODE_CONTINUOUS_PICTURE when it is cancelled (by default it will be cancelled automatically in 5 seconds).  

[Patrick Liao]

Thank you Scott,

These are very valuable pieces of info to me.

And for implementing startForcusAndMetering(), I guess a closed-loop control system would work well in this situation. That is, allowing developers to continuously feed new points (without too much delay like I implied in the original thread) to startFocusAndMetering(FocusMeteringAction).

Take Google Camera for example: When the user initiates a "tap-to-focus" action, instead of doing a one-shot focus, Google Camera continues to look for the subject being selected and attempts to lock the focus on that subject while showing the user with a circle where the subject is located on screen. I assume such feature is utilzing some kind of closed-loop focus control in camera2.

Another advice is to expose more metering info/options to users. Moving forward, some apps may need "rack focus" or some AF patterns that are designed to work in videos. By allowing developers to read the focus distance and adjust the AF speed/sensitivity, such features would be possible.

Cheers,

Patrick Liao

[Scott Nien]

Hi Patrick , 

As far as I know, Pixel does the trick in the Camera HAL instead of app layer.  It is possible to implement using camera2/cameraX API but I am not certain what the efforts would be. 

As for the video "rack focus" effect, currently official VideoCapture is under development and the existing videoCapture is not stable enough for production yet.  But actually you can set your own camera2 parameters to the capture request via Camera2Interop API.

See example below. 

// bind again with use cases to open camera 
mCamera = mCameraProvider.bindToLifecycle(this, mCurrentCameraSelector, preview, imageAnalysis, imageCapture);
Camera2CameraControl camera2CameraControl = Camera2CameraControl.from(camera.getCameraControl());

// dynamically changes the parameters. 
camera2CameraControl.setCaptureRequestOptions(
        new CaptureRequestOptions.Builder()
           .setCaptureRequestOption(
                CaptureRequest.CONTROL_AF_MODE,  CONTROL_AF_MODE_OFF)  // set AF to manual 
           .setCaptureRequestOption( 
                CaptureRequest.LENS_FOCUS_DISTANCE,  focusDistance).build());  

[Scott Nien]

I tried to achieve the "rack focus" effect using LENS_FOCUS_DISTANCE,  it works.  

you should also get the minimum focus distance and make sure the device supports MANUAL_SENSOR

Camera2CameraInfo camera2CameraInfo = Camera2CameraInfo.from(mCamera.getCameraInfo());
minFocusDistance =
        camera2CameraInfo.getCameraCharacteristic(CameraCharacteristics.LENS_INFO_MINIMUM_FOCUS_DISTANCE);

[Patrick Liao]

Hi Scott,

Thanks for the tips. I would definitely try it out.

And is there a way to query the current focal distance as set by the AF algorithm? That is, is it possible to get focal distance info when AF is trying to focus? 

Also, is there a way to slow down AF with CaptureRequest? In Video AF, the AF speed shall be slowed down to become more precise and more importantly natural ( see CONTROL_AF_MODE_CONTINUOUS_VIDEO). But when manually set the AF_MODE to be continuous video, the touch-to-focus thing stops responding altogether and I found no way of tweaking the AF speed under CONTINUOUS_PICTURE mode (which is what CameraX uses for touch-to-focus). 

And a quick update regarding the "servo AF" thing, the following code snippet worked well under good lighting conditons except I could not find a way to enlarge the AF area...

https://github.com/LBYPatrick/Phokus/blob/0.1.0/app/src/main/java/com/lbynet/Phokus/backend/CameraControl.java

private static float [] focusPoint = {0,0};

private static void focusToPoint(EventListener listener) {focusToPoint(focusPoint[0],focusPoint[1],true,listener);}

public static void focusToPoint(float x, float y, boolean isContinuous, EventListener listener) {

focusPoint[0] = x;
focusPoint[1] = y;

new Thread(() -> {

isFocusBusy_ = true;
listener.onEventBegan("Start focusing...");

runLater(() -> {

//Start focusing
MeteringPointFactory factory = previewView_.getMeteringPointFactory();
MeteringPoint point = factory.createPoint(focusPoint[0], focusPoint[1]);
FocusMeteringAction action = new FocusMeteringAction.Builder(point).disableAutoCancel().build();
ListenableFuture<FocusMeteringResult> focusResult = camera.getCameraControl().startFocusAndMetering(action);

focusResult.addListener(() -> {

try {

focusResult.get();
listener.onEventUpdated("Focused");
isFocusBusy_ = false;

if(!isCvfStopped_ && isContinuous) focusToPoint(listener);

else if(isContinuous) isCvfStopped_ = false;

} catch (Exception e) {

SAL.print(e);
SAL.print("Focus cancelled");
listener.onEventFinished(false, "Focus cancelled");

}

}, ContextCompat.getMainExecutor(context_));
});

}).start();
}

[Scott Nien]

Hi Patrick, 

You made a good point.  We have a CL which will be merged soon to switch to CONTROL_AF_MODE_CONTINUOUS_VIDEO but we also need some change to make it stay in CONTROL_AF_MODE_CONTINUOUS_VIDEO when doing focusMetering.   We will support this soon. 

Regarding enlarging the AF area,  you can use MeteringPointFactory#createPoint(float, float, float) to specify the size of the area. 

[Patrick Liao]

Thank you Scott,

I will wait for it to come out (And I saw that it has been merged already, yay :)

And I would definitely play around with MeteringPointFactory to see what I can do.

Cheers,

Patrick Liao

[Scott Nien]

Yes,  that CL was merged and welcome any feedback . 

[tobby P]

>  is it possible to get focal distance info when AF is trying to focus?

I would like to know if there is a way to retrieve focus distance value updates as set by AF actions as well.