Bug or Feature? setTargetResolution needs swapped width and hight for direct match

[Christian Günther]

Hi there,

i implemented my own resolution selection mechanic. Detailed information are already discussed in this thread: https://groups.google.com/a/android.com/g/camerax-developers/c/LpUc-5ro8_w/m/DONqJ85TBAAJ.

Nevertheless, i found out about an strange behavior of setTargetResolution which i want to discuss here in detail. I am acquiring my best resolution and set it by:

```

Camera2CameraInfo camera2Info = Camera2CameraInfo.from(camerax.getCameraInfo());

StreamConfigurationMap map = camera2Info.getCameraCharacteristic(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
Size desiredResolution = OptimalSizeComparator.sorted(map.getOutputSizes(mJpgFallback ? ImageFormat.JPEG : ImageFormat.YUV_420_888))[0];

[...]

captureBuilder.setCaptureMode(ImageCapture.CAPTURE_MODE_MINIMIZE_LATENCY)
   .setTargetRotation(activity.getWindowManager().getDefaultDisplay().getRotation())
   .setTargetResolution(desiredResolution);

```

In most cases camerax selects another resolution and not my desiredResolution. I discovered that i need to swap width and height here for all of my test-devices:

```
captureBuilder.setCaptureMode(ImageCapture.CAPTURE_MODE_MINIMIZE_LATENCY)
   .setTargetRotation(activity.getWindowManager().getDefaultDisplay().getRotation())
   .setTargetResolution(new Size(desiredResolution.getHeight(), desiredResolution.getWidth()));

```

Than i get the exact match between desiredResolution and the camerax selected resoltion for the capture.

So my Question here is, what is going on? Is this a bug or a feature? Is the orientation of the size in setTargetResolution depended on the sensor orientation and we have to some-time switch the width and height depending on the orientation or do we have a constant switched behavior for all devices, that i have to switch width and height for all devices? I am not able to determine it if ti is correct for all devices or only for those with rotated sensor, since all my test-devices have the same getDefaultDisplay().getRotation() of 90°. Why does that happen overall, since it would be nice to have camerax providing an consistently oriented resolution size handling on its interfaces?

[Charcoal Chen]

Hi Christian,

Thanks for the question. The supported sizes retrieved from StreamConfigurationMap might be in different directions against the device's orientation. Those sizes need to be calibrated to the app's coordinate frame by the sensor orientation (retrieved from CameraCharacteristics with CameraCharacteristics.SENSOR_ORIENTATION). This is usually too complex to some camera app developers who are not so familiar with such kinds of details. Therefore, current CameraX's setTargetResolution API design ask the developers to input the resolution, which is in the coordinate frame after rotating the supported sizes by the target rotation. By this approach, developers don't need to know the sensor orientation information and related knowledge. CameraX will internally calibrate the size according to the sensor orientation. The related information has been included in the javadocs of each use case builder's setTargetResolution API. Please refer to the following description extracted from ImageCapture.Builder#setTargetResolution():

"The resolution Size should be expressed in the coordinate frame after rotating the supported sizes by the target rotation. For example, a device with portrait natural orientation in natural target rotation requesting a portrait image may specify 480x640, and the same device, rotated 90 degrees and targeting landscape orientation may specify 640x480."

Therefore, when you select a target resolution from the supported output sizes of StreamConfigurationMap, you will also need to retrieve the sensor orientation value and calibrate the target resolution into your app's coordinate frame.

--
You received this message because you are subscribed to the Google Groups "Android CameraX Discussion Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camerax-develop...@android.com.
To view this discussion on the web visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/f3e97198-631d-4785-990a-86a2d8dc4cc1n%40android.com.

[Christian Günther]

Thanks for prividing the clarification here. This helps alot.