Hello,
I am trying to configure photo capture to use a resolution of 720x1280. However on several devices takePicture() never completes, neither the onError() or onImageSaved() callbacks are called.
On the one device I have that can reproduce it (Motorola Moto e20), I’m not seeing any error logs or exceptions thrown. And even though getOutputSizes() reports that 1280x720 is supported I can’t find a way to massage the configuration to avoid the hang but also get a 720x1280 image. Also the camera2 version of the code does generate 720x1280 images on this device.
The setup code that results in a hang looks something like:
val resolution =
if (cameraPreview.resources.configuration.orientation == Configuration.ORIENTATION_PORTRAIT) {
Size(1_280, 720)
} else {
Size(720, 1_280)
}
imageCapture = ImageCapture.Builder()
.setResolutionSelector(
ResolutionSelector.Builder()
.setResolutionStrategy(
ResolutionStrategy(
resolution,
ResolutionStrategy.FALLBACK_RULE_CLOSEST_HIGHER_THEN_LOWER
)
)
.setAspectRatioStrategy(AspectRatioStrategy.RATIO_16_9_FALLBACK_AUTO_STRATEGY)
.build()
)
.setFlashMode(flashMode).build()
Via logging I am seeing this type of hang on multiple devices, I can’t be certain it’s the same issue as I am seeing on the Moto e20 but I suspect it is. Some of these additional devices are: Infinix X6525, Infinix X669D, RMX3263, RMX3501, RMX3830, RMX3834, SM-A032F, SM-A032M, TECNO BG6, moto e13.
I am on CameraX version 1.4.0.
Any help you can provide would be appreciated, thanks.
--
You received this message because you are subscribed to the Google Groups "Android CameraX Discussion Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camerax-develop...@android.com.
To view this discussion visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/9198cc83-2f10-45d6-853f-17487e8e3c91n%40android.com.
imageCapture = ImageCapture.Builder()
.setResolutionSelector(
ResolutionSelector.Builder()
.setAspectRatioStrategy(AspectRatioStrategy.RATIO_16_9_FALLBACK_AUTO_STRATEGY)
.build()
)
.setFlashMode(flashMode).build()
Hi all, thanks for the responses.
First off, now that the change to remove the resolution parameter has been available to users for a few days the results are looking promising: no hangs and even though it takes longer to take the picture the success rate is over 99% (compared to ~97% for the existing Camera2 implementation, and ~92% for CameraX with the hang bug). Thanks Charcoal for the workaround suggestion, but given this I think it is not needed, although I do want to eventually specify the 720x1280 resolution once a fix is available. But for now it seems like we won’t be blocked on that fix.
And if it helps, I compiled a larger list of impacted devices plus their chipsets:
Infinix X6516 (Unisoc SC9863A1), Infinix X6517 (Unisoc SC9863A1), Infinix X6525 (Unisoc T606), Infinix X6528 (Unisoc T606), Infinix X6528B (Unisoc T606), Infinix X669 (Unisoc T606), Infinix X669D (Unisoc T606), Nokia G21 (Unisoc T606), Nokia G21 (Unisoc ums9230), RMX3231 (Octa Core), RMX3261 (Octa Core), RMX3261 (T610-Unisoc), RMX3263 (T610-Unisoc), RMX3269 (T618-Unisoc), RMX3501 (Unisoc T612), RMX3501 (Unisoc ums9230H), RMX3506 (Unisoc T612), RMX3511 (Unisoc T616), RMX3511 (Unisoc ums9230T), RMX3581 (Unisoc T612), RMX3624 (Unisoc T612), RMX3627 (Unisoc T612), RMX3690 (Unisoc SC9863A), RMX3760 (Unisoc T612), RMX3761 (Unisoc T612), RMX3762 (Unisoc T612), RMX3830 (Unisoc T612), RMX3834 (Unisoc T612), SM-A032F (UNISOC SC9863A), SM-A032F (Unisoc SC9863A), SM-A032M (UNISOC SC9863A), SM-A032M (Unisoc SC9863A), SM-A035F (UNISOC T606), SM-A035F (Unisoc ums9230), SM-A035G (UNISOC T606), SM-A035G (Unisoc ums9230), SM-A035M (UNISOC T606), SM-A035M (Unisoc ums9230), TECNO BF6 (Unisoc SC9863A), TECNO BF6 (Unisoc SC9863A1), TECNO BG6 (Unisoc T606), TECNO KG5k (Unisoc ums9230), TECNO KG5n (Unisoc ums9230), TECNO KI5k (Unisoc T606), TECNO KI5m (Unisoc T606), ZTE 7060 (Unisoc T606), ZTE Blade A54 (Unisoc L3), ZTE Blade A54 (Unisoc L3R), itel A662LM (Unisoc SC9863A), itel A662LM (Unisoc SC9863A1), itel A665L (Unisoc T603), itel S665L (Unisoc T606), moto e13 (Unisoc T606), moto e20 (Unisoc ums9230), moto e32 (MT6765H), moto e32 (Unisoc ums9230)
… basically almost all Unisoc chips are impacted, plus two others (“Octa Core”, and “MT6765H”).
Note that I am not ruling out some sort of problem with my code. But I haven’t been able to find any issues on my side, although the fact that an error isn’t reported does seem to indicate something unexpected is happening inside CameraX.
> Regarding the issue that "Google Pixel 3a XL takes 3-4 seconds", it is abnormal, do you enable ViewPort or use CameraController ?
I am not using CameraController, I am just using a PreviewView. By “enable ViewPort” do you mean calling setViewPort() on the UseCaseGroup? I am not doing that, is it something you would recommend?
I did some proper timings and the slowdown isn’t as bad as I originally stated, it’s only ~33% slower when capturing without specifying 720x1280. The resolution that ends up getting used is 2160x3840, which is significantly larger so the 33% slowdown doesn’t seem way out of line.
I also did try using setViewPort() to see if it might help with the hang, but it can still be reproduced with it added.
-Darren
Ok great, thanks for working on a fix!
Glad you can reproduce the issue. We are not binding an ImageAnalysis, we just bind Preview, ImageCapture and VideoCapture use cases. Let me know if you need me to supply more of our code if that would be helpful in reproducing further.
Also, out of curiosity - does it make sense the Camera2 can capture on these devices at 720x1280? I figured that since CameraX is built on Camera2 that they would behave similarly.
Thanks,
Darren
Hi Darren,
Could you please provide the QualitySelector
setting in your VideoCapture+Recorder?
The default setting prioritizes FHD
if available. I'm curious if it's currently configured to use Quality.HD
since I can only reproduce it when VideoCapture is also 1280x720 (HD).
(Note: issue also happens to me when ImageCapture and VideoCapture are both 1920x1080)
To view this discussion visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/b063c95c-cdda-4c42-8d1e-dcaf24fa509bn%40android.com.
Hi Leo,
Here it is, we are using Quality.HD:
videoCapture = VideoCapture.withOutput(
Recorder.Builder()
.setExecutor(ContextCompat.getMainExecutor(fragment.requireContext()))
.setQualitySelector(
QualitySelector.from(
Quality.HD,
FallbackStrategy.higherQualityOrLowerThan(Quality.HD)
)
).build()
)
Hi Scott,
Not sure if this is what you are looking for, but for camera2 to get the available sizes, we do:
val sizeChoices = cameraManager.getCameraCharacteristics(cameraId)
.get(SCALER_STREAM_CONFIGURATION_MAP)!!
.getOutputSizes(SurfaceHolder::class.java)
We don’t do anything explicitly with a stream configuration for CameraX. I presume since the logic for determining the resolution is done via the setResolutionSelector?
Yeah exactly:
A preview surface created like so:
val previewSize = Size(1280,720)
previewView.setDefaultBufferSize(previewSize.width, previewSize.height)
previewSurface = Surface(previewView.surfaceTexture)
An image reader surface, created something like so:
imageReader = ImageReader.newInstance(previewSize.width, previewSize.height, ImageFormat.JPEG, 1)
imageReaderSurface = imageReader?.surface
A MediaCodec/MediaRecorder surface that is created pretty much like in the sample, as our code is based on that. And we call setVideoSize(1280, 720) for that.
Happy to answer any more questions that you need.
-Darren
Also, out of curiosity - does it make sense the Camera2 can capture on these devices at 720x1280? I figured that since CameraX is built on Camera2 that they would behave similarly.
adb shell dumpsys media.camera | egrep -A 40 "Stream configuration"
To view this discussion visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/3b5adc65-6157-45c1-aeb4-7f33ab7d2b3bn%40android.com.
Ah, interesting. Here it is:
Camera2
Stream configuration:
Operation mode: NORMAL (0)
No input stream.
Stream[0]: Output
Consumer name: SurfaceTexture-0-32448-2
State: 4
Dims: 1280 x 720, format 0x22, dataspace 0x0
Max size: 0
Combined usage: 1125899906842883, max HAL buffers: 8
Frames produced: 175, last timestamp: 1650118263320486 ns
Total buffers: 10, currently dequeued: 8
DequeueBuffer latency histogram: (183) samples
5 10 15 20 25 30 35 40 45 inf (max ms)
99.45 0.55 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 (%)
Stream[1]: Output
Consumer name: GraphicBufferSource
State: 4
Dims: 1280 x 720, format 0x22, dataspace 0x104
Max size: 0
Combined usage: 65539, max HAL buffers: 8
Frames produced: 0, last timestamp: 0 ns
Total buffers: 12, currently dequeued: 0
Stream[2]: Output
Consumer name: ImageReader-1280x720f100m1-32448-0
State: 4
Dims: 1280 x 720, format 0x21, dataspace 0x8c20000
Max size: 1625938
Combined usage: 3, max HAL buffers: 1
Frames produced: 0, last timestamp: 0 ns
Total buffers: 2, currently dequeued: 0
Camera3 Buffer Manager:
Total stream sets: 0
In-flight requests:
Frame 175 | Timestamp: 0, metadata arrived: false, buffers left: 1
Frame 176 | Timestamp: 0, metadata arrived: false, buffers left: 1
Frame 177 | Timestamp: 0, metadata arrived: false, buffers left: 1
Frame 178 | Timestamp: 0, metadata arrived: false, buffers left: 1
Frame 179 | Timestamp: 0, metadata arrived: false, buffers left: 1
Frame 180 | Timestamp: 0, metadata arrived: false, buffers left: 1
Frame 181 | Timestamp: 0, metadata arrived: false, buffers left: 1
Frame 182 | Timestamp: 0, metadata arrived: false, buffers left: 1
CameraX
Stream configuration:
Operation mode: NORMAL (0)
No input stream.
Stream[0]: Output
Consumer name: SurfaceView - com.pinterest.dev/com.pinterest.feature.pin.creation.CreationActivity#0
State: 4
Dims: 1280 x 720, format 0x22, dataspace 0x0
Max size: 0
Combined usage: 1125899906844931, max HAL buffers: 8
Frames produced: 94, last timestamp: 1650210717066959 ns
Total buffers: 9, currently dequeued: 8
DequeueBuffer latency histogram: (102) samples
5 10 15 20 25 30 35 40 45 inf (max ms)
15.69 23.53 30.39 26.47 3.92 0.00 0.00 0.00 0.00 0.00 (%)
Stream[1]: Output
Consumer name: ImageReader-1920x1080f100m4-2067-0
State: 4
Dims: 1920 x 1080, format 0x21, dataspace 0x8c20000
Max size: 3330670
Combined usage: 3, max HAL buffers: 1
Frames produced: 0, last timestamp: 0 ns
Total buffers: 5, currently dequeued: 0
Stream[2]: Output
Consumer name: GraphicBufferSource
State: 4
Dims: 1280 x 720, format 0x22, dataspace 0x104
Max size: 0
Combined usage: 65539, max HAL buffers: 8
Frames produced: 0, last timestamp: 0 ns
Total buffers: 12, currently dequeued: 0
Camera3 Buffer Manager:
Total stream sets: 0
In-flight requests:
Frame 94 | Timestamp: 0, metadata arrived: false, buffers left: 1
Frame 95 | Timestamp: 0, metadata arrived: false, buffers left: 1
Frame 96 | Timestamp: 0, metadata arrived: false, buffers left: 1
Frame 97 | Timestamp: 0, metadata arrived: false, buffers left: 1
Frame 98 | Timestamp: 0, metadata arrived: false, buffers left: 1
Frame 99 | Timestamp: 0, metadata arrived: false, buffers left: 1
Frame 100 | Timestamp: 0, metadata arrived: false, buffers left: 1
Frame 101 | Timestamp: 0, metadata arrived: false, buffers left: 1
In the CameraX code in the original hang scenario I was only setting the aspect ratio for the Preview but was setting aspect ratio and resolution for the ImageCapture. So I suppose that is why Stream[1] in the CameraX output is 1920 x 1080, although it is still 9:16.
I then tried using the exact same ResolutionSelector for both the preview and the image, but the hang still happens:
CameraX: Using this same ResolutionSelector for Preview and ImageCapture
Shared setting:
val resolutionSelector = ResolutionSelector.Builder()
.setResolutionStrategy(
ResolutionStrategy(
Size(1_280, 720),
ResolutionStrategy.FALLBACK_RULE_CLOSEST_HIGHER_THEN_LOWER
)
)
.setAspectRatioStrategy(AspectRatioStrategy.RATIO_16_9_FALLBACK_AUTO_STRATEGY)
.build()
Output:
Stream configuration:
Operation mode: NORMAL (0)
No input stream.
Stream[0]: Output
Consumer name: SurfaceView - com.pinterest.dev/com.pinterest.feature.pin.creation.CreationActivity#0
State: 4
Dims: 1280 x 720, format 0x22, dataspace 0x0
Max size: 0
Combined usage: 1125899906844931, max HAL buffers: 8
Frames produced: 124, last timestamp: 1651355023909256 ns
Total buffers: 9, currently dequeued: 8
DequeueBuffer latency histogram: (132) samples
5 10 15 20 25 30 35 40 45 inf (max ms)
8.33 19.70 37.12 27.27 7.58 0.00 0.00 0.00 0.00 0.00 (%)
Stream[1]: Output
Consumer name: ImageReader-1280x720f100m4-4596-4
State: 4
Dims: 1280 x 720, format 0x21, dataspace 0x8c20000
Max size: 1625938
Combined usage: 3, max HAL buffers: 1
Frames produced: 0, last timestamp: 0 ns
Total buffers: 5, currently dequeued: 0
Stream[2]: Output
Consumer name: GraphicBufferSource
State: 4
Dims: 1280 x 720, format 0x22, dataspace 0x104
Max size: 0
Combined usage: 65539, max HAL buffers: 8
Frames produced: 0, last timestamp: 0 ns
Total buffers: 12, currently dequeued: 0
Camera3 Buffer Manager:
Total stream sets: 0
In-flight requests:
Frame 124 | Timestamp: 0, metadata arrived: false, buffers left: 1
Frame 125 | Timestamp: 0, metadata arrived: false, buffers left: 1
Frame 126 | Timestamp: 0, metadata arrived: false, buffers left: 1
Frame 127 | Timestamp: 0, metadata arrived: false, buffers left: 1
Frame 128 | Timestamp: 0, metadata arrived: false, buffers left: 1
Frame 129 | Timestamp: 0, metadata arrived: false, buffers left: 1
Frame 130 | Timestamp: 0, metadata arrived: false, buffers left: 1
Frame 131 | Timestamp: 0, metadata arrived: false, buffers left: 1
Hopefully that helps!
-Darren
Hi Charcoal,
Strange that it’s stopped happening for you, the hang remains 100% on my moto e20. Answers:
1. No problems with the test app. Although note that on my device it’s taking photos at 1080 × 1125, so not sure if that’s an invalid test since it’s not the problematic resolution.
2. I added a cameraPreview.implementationMode = COMPATIBLE … but the hang still occurs.
3. The hang still occurs after rebooting.
4. My device is on Android 11. From looking at user logs, it doesn’t appear to be OS specific: the hang occurred 100% of the time on every OS seen for impacted devices (specifically, OS 11, 12, 13 & 14)
Thanks for the continued investigations,
Darren