Recognized faces have negative coordinates while the camera is zoomed in

[Eugene B]

Hello. I am developing a camera application using the CameraX API. On the vendor's tablet, face detection is implemented at the HAL level. Currently, I am trying to resolve an issue where I receive negative coordinates for detected faces when zoom is applied. I debugged the HAL pipeline and confirmed that when zoom is enabled, the coordinates are correctly passed to CameraMetadata* resultMetadata. However, in onCaptureCompleted, result.get(TotalCaptureResult.STATISTICS_FACES) returns a list of detected faces with negative coordinates.
val imageAnalysisExtender = Camera2Interop.Extender(builder)
val captureCallbackAnalysis = object : CameraCaptureSession.CaptureCallback() {
    override fun onCaptureCompleted(
        session: CameraCaptureSession,
        request: CaptureRequest,
        result: TotalCaptureResult
    ) {
        // ...
    }

To set the zoom ratio, we are using camera.cameraControl.setZoomRatio(ratio). I tried manually setting the crop region using:
imageAnalysisExtender.setCaptureRequestOption(
    CaptureRequest.SCALER_CROP_REGION,
    cropRegion
)
This worked, and the coordinates started coming in with correct positive values. However, when I changed the zoom to a different value, the coordinates started coming in as negative again.

Are there any recommendations on how to properly initialize zoom to avoid incorrect coordinate transformation?

[Eugene B]

here what I see in the logs:

what is passed to CameraMetadata* resultMetadata from HAL: android.ha....4-service pid-469  newFaces left: 472 right: 887

what I'm getting in onCaptureCompleted: newFaces: Rect(-1167, -855 - -337, -25)

среда, 26 февраля 2025 г. в 19:41:17 UTC+3, Eugene B:

[Eugene B]

As a reference application, I'm using the legacy Camera. In this application, face recognition works fine because it is based on the Camera ONE API, which defines a repeating preview request with the initiated SCALER_CROP_REGION and resends this request when the zoom is changed:
 
Supplier<Rect> cropRegion = new ZoomedCropRegion(
        cameraCharacteristics.getSensorInfoActiveArraySize(), zoom);
requestTemplate.setParam(CaptureRequest.SCALER_CROP_REGION, cropRegion);

How can I change the SCALER_CROP_REGION in the CameraX API without restarting the camera?

четверг, 27 февраля 2025 г. в 12:07:54 UTC+3, Eugene B:

[Scott Nien]

Hi there, 

I think this is probably an OEM issue that didn't convert the coordinates correctly. 

When using CONTROL_ZOOM_RATIO (which CameraX uses if supported),  the coordinates of faces, 3A regions are effective after-zoom FoV. 

May we know what devices it is ?  

How can I change the SCALER_CROP_REGION in the CameraX API without restarting the camera?

You can do that by using Camera2CameraControl. See the sample codes below

        val camera2CameraControl = Camera2CameraControl.from(camera.cameraControl)
        val options = CaptureRequestOptions.Builder()
                .setCaptureRequestOption(CaptureRequest.SCALER_CROP_REGION, cropRegion)
                .build()
        camera2CameraControl.setCaptureRequestOptions(options);

Scott

--
You received this message because you are subscribed to the Google Groups "Android CameraX Discussion Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camerax-develop...@android.com.
To view this discussion visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/144a1fe6-a777-4a84-9d6e-3fb39ee66b35n%40android.com.

[Eugene B]

Hi Scott,

Thank you very much! This solution really works, and no additional coordinate transformation is required.

понедельник, 3 марта 2025 г. в 06:57:21 UTC+3, scot...@google.com: