I've been working with the ImageAnalysis use case and CameraX version "1.0.0-rc03"
I'm becoming familiar with the YUV_420_888 format, I've used YuvToRgbConverter
(as well as my own versions) to convert the ImageProxy planes into Nv21 and then to a bitmap, however I've found that for some devices the conversion works fine and for others the first row of the image is in the wrong position.Samsung S8
This device works fine, when converting the ImageProxy planes to Nv21 I get the correct image and the planes byte buffers look good, here is a sample image captured.
Xiaomi Mi A2
This is one of the devices I'm having troubles with, when I convert this YUV planes into Nv21 the first "row" of the image appears to be misplaced and seems to fit rather to the end, see the attached image, if you pay attention to the first column from left to right, you'll notice that they do not fit and should be on the far right.
For the Mi A2 the result is the same no matter which method I try, and it's the same result if I use an image size where the planes include padding or not. This is not the only device with this problem, I've also found it on a Samsung Tab E and a Kyocera E6560, and it even happens if I just narrow it down to the Y plane and ignore the U and V planes.
The YUV planes buffer bytes
When using the deprecated Camera API the Nv21 byte arrays come just fine and populated with the correct data for all the devices, it's just CameraX where I'm having this problem.
Also what I was able to find, is that there is a strange pattern on the conflicting devices vs the S8 working device.
For the S8 the YUV planes come with the byte data just fine, while for the conflicting devices the Y plane first index is always 0 and the U/V planes first index is always -128, If I take the Y plane and ignore the first index on the conflicting devices, the image comes as it should, without the first row shifted, on the other hand if I do the opposite and take the working S8 device and make the first Y plane byte 0 on purpose, the output image comes shifted as the conflicting devices.
Of course, I have a few questions about this,
- What's the reason behind these values at the 0 index for the conflicting devices?
- Should that byte be skipped? or consumed differently? how can I workaround this conflict?
- Is there any way that I can query the YUV_420_888 underlying format to know if it's Nv21, Nv12, YV12, etc?
- On image sizes that add padding to the planes, the planes are not padded on all rows, the last row has no padding, why is that?
- Are these planes buffers individual pointers to a big single data structure in the C layer? or really 3 different ones?
I'd appreciate your help on this matter, since I've read almost every entry related to YUV format on this group and the CameraX issue tracker.