Questions about Aspect Ratios and Resolutions

[Taylor Stephenson]

1. Why was 4:3 chosen as the default rather than attempting to match the aspect ratio to the device screen? Asking because I could check the device aspect ratio and do setTargetAspectRatio, but is there a reason I shouldn't?

2. Does CameraX ever select resolutions higher than what the phone can display? For example, if my device has a 1600x900 screen, is it possible CameraX would capture an image in 1920x1080? The benefit would be these look better when viewed from a desktop or better devices and the drawback is the memory and processing load on the device. If I care more about on-device processing, do I need to use setTargetResolution?

3. I'm porting some old camera2 code to CameraX. I've been getting square images by creating a square PreviewView, but I noticed old camera2 code would try to filter available output sizes to an aspect ratio close to 1:1.

This would be roughly equivalent to setTargetAspectRatio(1:1), except there are only AspectRatio constants for 4:3 and 16:9.

I was just curious for the reasoning behind this. Are there few to no physical camera modules that capture in 1:1 (which would probably mean we ended up using ~4:3 all the time anyway)? Or do they tend to have poor resolutions?

Thanks!

[Charcoal Chen]

Thanks for your questions.

1. Usually, the camera devices have the pixel array of aspect ratio 4:3. Images of aspect ratio 4:3 can contain the full FOV content. Most devices shipped in these years may have 16:9 displays. But it may still depend on apps' design to capture 4:3 (max FOV the camera device can support) or 16:9 (matches the display) images. Apps may also have the corresponding UI layout (view of 4:3 or 16:9) to make sure that the captured/saved JPEG images are the same as what end users see in preview. Selecting the sizes matching the aspect ratio of the device screen as default may cause image content double-crop problems for the apps that have 4:3 preview layout design, if they do not set target aspect ratio explicitly. This is because the 16:9 output images have been cropped (1st time cropping) from the full 4:3 camera sensor pixel array and an additional cropping operation (2nd time cropping) will be done when the images are drawn to the 4:3 preview. Making 4:3 as the default aspect ratio won't have such an issue. If your app is a full screen app, checking the device display aspect ratio to set the target aspect ratio should be able to make the output images nearly the same as preview. But if the device display or the view of preview is not exactly 16:9 or 4:3, a post-cropping process may be needed to make sure they are the same.

2. This question may have different answers for different types of use cases. Please see the max resolution info listed in here. Basically, only Preview use case is limited to be under the display size or 1080p. For ImageCapture or ImageAnalysis, they could possibly have output images larger than the device display. But, which resolution can be selected to use also depends on the camera device capability and the use case combination. Please refer to the guaranteed supported configurations tables listed in here. If the on-device processing means to use ImageAnalysis and your app also needs another ImageCapture and Preview use cases, your app can have output images for ImageAnalysis larger than the device display when running on LIMITED-level above devices. But a trade-off is that your app can only have the RECORD size output image for ImageCapture. And, yes, you need to use setTargetResolution to explicitly set the desired large resolution to ImageAnalysis. Please also see ImageAnalysis.Builder#setTargetResolution(android.util.Size).

3. I can't say it is few, but really some devices do not support 1:1 resolutions. Unlike 4:3 and 16:9 resolutions, they are almost supported on all devices I checked. The best way may still be to set the target aspect ratio or resolution to match your app's UI layout. Setting 4:3 always might still cause a use case to not get the largest resolution it can have. The case is, the device display is 1080p and the preview is full screen. Because the aspect ratio setting is 4:3 and the resolution for preview is limited under 1080p, finally, 1440x1080 might be selected for the preview. After transforming, the preview content is actually drawn from 1440x810 (16:9) image content. But actually, 1080p resolution can be supported for the preview. The output results might be not obviously poor, but it is not the best it could have.