Achieving different resolutions for Preview and VideoCapture

[Javier Gerardo Martinez Salomon]

Hi CameraX folks,

While testing the VideoCapture UseCase I noticed that despite building my Preview UseCase and setting a custom target resolution such as 1080x1920 with `setTargetResolution()` if I bind a VideoCapture UseCase I would have both my Preview and my VideoCapture usecases use the resolution resolved by VideoCapture using the QualitySelector. i.e. if I set the QualitySelector to

QualitySelector.fromOrderedList(listOf(Quality.HD, Quality.SD, Quality.LOWEST)) and the VideoCapture UseCase resolves to Quality.SD the resolution of both Preview and VideoCapture would be 720x1080.

My goal is to make the Preview usecase to support mainly 1080p and fallback to 720p, while the VideoCapture usecase could support down to 480p. Is there a way to use individual resolutions for each usecase? I understand that the stream used for Preview and VideoCapture is shared, but wondering if this is a possibility. I'm using CameraX 1.3.3.

Thanks in advance.

[Charcoal Chen]

Hi,

Thanks for the question.

The Preview's resolution is restricted by the PREVIEW size defined in CameraDevice's regular-capture section. According to the camera device's capabilities, the stream combinations (size + format) listed in the guaranteed supported configurations tables can be used to select the best suitable resolutions for the bound UseCases.

=> PREVIEW refers to the best size match to the device's screen resolution, or to 1080p (1920x1080), whichever is smaller. 

Please check whether the device's display size you tested on is about the size of 720x1280 or not. And you can also run the app on the other device with a larger display to check whether the result will be different or not.

If it is not the root cause, please let me know which device you encountered this issue on? Thanks.

--
You received this message because you are subscribed to the Google Groups "Android CameraX Discussion Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camerax-develop...@android.com.
To view this discussion on the web visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/9382de94-3f55-4055-a292-6a6b75a03d6an%40android.com.

[Javier Gerardo Martinez Salomon]

Hi,

Thanks for your answer, I did a few tests and using a Pixel 6 I found the following:

Using a target resolution of 1080p for Preview and HD(720p) quality for VideoCapture

• Preview resolution is 1080x1920
• VideoCapture resolution is 1280x720

This is what I'd consider the expected behavior.

Using a target resolution of 1080p for Preview, ImageAnalysis and ImageCapture, and using HD(720p) quality for VideoCapture

• Preview resolution is 720x1280
• VideoCapture resolution is 1280x720
• ImageAnalysis resolution is 1080x1920
• ImageCapture resolution is 1080x1920

In this case, the Preview resolution is set to the resolution of the VideoCapture regardless of targeting 1080p. The interesting bit is that if I target FHD quality in VideoCapture I do get all four usecases at 1080p. If I target SD for video both VideoCapture and Preview would have a 480p resolution.

I've included a sample code which makes use of a PreviewView and binds the four usecases, line 118 specifies the quality of the video, It seems to me that for some reason when binding the four usecases the VideoCapture overrides the target resolution of the Preview usecase despite of the device being able to display the preview at the target resolution, and for some reason when only binding the two usecases from the first test this is not a problem.

Currently I do request the Preview to be 1080p or 720p whatever the device can give me, however I'd normally expect the video to be requested at 720p and possibly support 480p without having a side effect on the Preview.

[Charcoal Chen]

Hi,

Thanks for providing the info. It should be a known issue that has been fixed by aosp/2853058. When binding four UseCases, CameraX has a stream sharing mechanism to provide images to Preview and VideoCapture from one input stream. Before aosp/2853058 wes merged, Preview and VideoCapture always had the same resolution. Could you upgrade to use CameraX l.4.0-beta to see whether the issue can be fixed?

[Javier Gerardo Martinez Salomon]

Hi,

Sure, I tried with CameraX "1.4.0-beta02" and repeated the exercise, I compared against the latest stable "1.3.4" and got the following differences:

Using a target resolution of 1080p for Preview, ImageAnalysis and ImageCapture, and using SD(480p) quality for VideoCapture

• In 1.4.0-beta02
  • Preview Resolution 720x1280
  • VideoCapture Resolution 480x720
  • ImageAnalysis Resolution 1920x1080
  • ImageCapture Resolution 1920x1080
• In 1.3.4
  • Preview Resolution 480x720
  • VideoCapture Resolution 720x480
  • ImageAnalysis Resolution 1920x1080
  • ImageCapture Resolution 1920x1080

For the 720p and 1080p cases the results are the same as before.

It seems that there was some improvement as previously I used to get 480p for both Preview and VideoCapture which isn't the same in 1.4.0-beta02, another thing I just noticed and that might be irrelevant is the width and height of the sizes, in 1.3.4 the VideoCapture resolution is represented as 720x480, the same resolution in 1.4.0-beta02 is represented as 480x720. But for the most part when it comes to 720p the results are the same, video and preview match the resolution instead of keeping Preview at 1080p and Preview at 720p. Is this expected?

What would be the recommendation for my implementation, stick to 720p or 1080p video? I'd like to support 480p but I don't want my Preview to look that low resolution-wise.

Thanks

[Charcoal Chen]

Hi,

As I mentioned in my first replyment, camera devices have their capabilities limitation. CameraX refers to the  guaranteed supported configurations tables listed in the CameraDevice's regular-capture section to find the best suitable resolutions for the bound UseCase combination. Actually, the guaranteed supported configurations tables do not include a configuration for four streams for Preview + VideoCapture + ImageAnalysis + ImageCapture combination. For this use case, CameraX uses a stream sharing mechanism to share images from one stream to Preview and VideoCapture UseCases. But this also has some restrictions to avoid some double-cropping image or resolution selection performance problems. There might be too many details there. Please let me know if you are interested in any part.  But, the results you provided for 1.4.0 match our expectations. Here are the explanations for your app’s cases.

Case 1: your app wants the Preview to have 1080p resolution and  VideoCapture to have 480p resolution

Because four UseCases are bound, CameraX needs to find a resolution to share to these VideoCapture and Preview. To avoid the double-cropping issue, CameraX selects a 4:3 resolution to create the stream to share the images. According to the guaranteed tables, only 4:3 resolutions under 1080p or display size can be selected. Finally, maybe 1440x1080 or 1280x960 is selected according to the device's supported output sizes, and the images are cropped and scaled to produce the 1280x720 images to Preview and 720x480 images to VideoCapture.

Case 2: your app wants the Preview to have 1080p resolution and  VideoCapture to have 720p resolution

To avoid the double-cropping issue and have better resolution selection performance, CameraX would select 4:3 resolutions in priority. To make 1080p can be selected for Preview in this case, apps can use ResolutionSelector and implement a ResolutionFilter to force CameraX to select 1080p resolution for Preview. The code looks like:

val preview = Preview.Builder()

   .setResolutionSelector(ResolutionSelector.Builder()

       .setResolutionFilter { supportedSizes, _ ->

           val resultList = mutableListOf<Size>()

           if (supportedSizes.contains(Size(1920, 1080))) {

               resultList.add(Size(1920, 1080))

           }

           resultList

       }

       .build())

   .build()

Please note that not all devices can support the configuration. Therefore, if an IllegalArgumentException is thrown when calling bindToLifecycle method, your app would need to try-catch it and loosen the resolution conditions to make the UseCases bound successfully.