Achieving sharp images on close range objects

[Javier Gerardo Martinez Salomon]

Hi all,

I'm currently using CameraX 1.1.0, my usecase is to take photos of documents and the main goal of my application is to acquire a high quality image determined from the analysis of frames using an Analyzer.

It has come to my attention, that some devices do not produce sharp images when taking the documents, some devices produce images that are very sharp in the center of the image but start to become blurry to the edges of the image. Please see the attached example, you can notice how the edges start to get softer as well as the text around it, while the middle of the image is very sharp.

I want to achieve sharpness on the whole document and I'm looking for recommendations on how to do it and or to point me to the correct APIs that can help me.

The device I'm taking as an example is the Pixel 6, the pictures are taken from the back camera which I select based on custom criteria, for the Pixel 6, this is the data from the camera that is selected:
    ID: 0
    FACING: BACK
    LEVEL: FULL
    FOCAL LENGTHS: 6.81
    HYPERFOCAL DISTANCE: 0.1923077
    MINIMUM FOCUS DISTANCE: 0.19230779.523809
    CAPABILITIES: BACKWARD_COMPATIBLE, MANUAL_SENSOR, READ_SENSOR_SETTINGS, MANUAL_POST_PROCESSING, BURST_CAPTURE, UNKNOWN, RAW, CONSTRAINED_HIGH_SPEED_VIDEO, LOGICAL_MULTI_CAMERA
    FOCUS MODES: OFF, AUTO, MACRO, CONTINUOUS_VIDEO, CONTINUOUS_PICTURE, EDOF

So far, I've tried changing the focus mode, issuing auto-focus routines regularly at the center of the document, changing the LENS_FOCUS_DISTANCE and LENS_FOCUS_LENGTH using CaptureRequestOptions from Camera2CameraControl but to no avail.

The only thing that has worked so far, is to use CameraControl.setZoomRatio() to 1.5 or 2.x so that I'm forced to get the camera farther away from the document which has a better focus, but the image detail decreases and the exposure of the image changes as well.

1. Is there any value I can get from the camera that could indicate that the device would have problems focusing for my use case?
2. Is there any API I can use to force the camera to focus correctly?
3. Could this be a hardware limitation? I've seen the same issue on other devices like Samsung S21, S22, S23 and the list keeps going.
4. Is there a workaround for getting sharp images?

I'd be happy to share more details, thank you very much for your time.

[Eino-Ville Talvala]

Hi Javier -

This is generally speaking, a hardware issue. So no combination of control fields is likely to help.

Nearly all lenses are sharper in the center than they are at the edges, and some of the recent flagships have bigger sensors and wide apertures, which makes them more susceptible to this due to the shallow depth of field.

The best solution is to zoom in about 2x, since then you're not using the less-sharp sections of the lens.  Generally that provides a better perspective on the documents as well, being closer to an orthographic projection.  It does mean you have to be farther away from the documents to place them in the center part of the lens's field of view.  It does potentially reduce your captured resolution, unfortunately.

- Eddy

--
You received this message because you are subscribed to the Google Groups "Android CameraX Discussion Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camerax-develop...@android.com.
To view this discussion on the web visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/864e30dd-093e-4bd9-8d12-3512e1ee4f46n%40android.com.

[Javier Gerardo Martinez Salomon]

Hi Eddy,

Thank you very much for your quick response.

In general it does align with what I've seen so far from experimentation, getting the phone farther away does help, but for a more comfortable experience it would make sense to have the phone zoomed in, I've got a couple of questions related to that:

1. Is there a CameraCharacteristic or value I can query before opening the camera that would indicate that I'd have this issue and apply the zoom factor beforehand?
2. If it's not possible beforehand? is there anything at runtime?
   
3. Is there any other implication in a zoomed capture other than quality of the image? would the exposure change?

I've put my eye in the FOCAL_LENGTH property, I've noticed that some devices have a range of focal lengths while others have fixed lengths, does the focal length of the devices that have a range of them adjust automatically without specifying?
I've noticed that devices like the Samsung Galaxy S8, Xiaomi Mi A2 and Redmi Note 7 have a fixed focal length of ~4.0 which produces sharp images without soft edges, but then newer devices like the Pixel 6 and Samsung S.x series have a higher focal length of ~6.0, is this value an indication of the issue?

Thanks!

[Eino-Ville Talvala]

1. Unfortunately, I don't think so.  While the camera characteristics can include things like the lens's geometric distortion information (which lets you map each pixel to a ray out in the world), we don't have metadata for spherical aberration, non-planar depth-of-field or other lens problems which could cause loss of sharpness at the edges.  You could try using more zoom once the lens aperture gets to be a low enough value, under the assumption that wide-aperture devices are more likely to have issues at the edges of the field of view.  But I don't actually know if that holds up in practice.

3. The exposure is usually calculated based on the FOV, so if you're looking at the same content, there's no reason to expect the exposure or white balance to change.  If you zoom in enough (usually more than 4x), some devices will switch you to the telephoto lens, which would have different physical characteristics.  In that case the values could change. But generally, the telephoto lens can't focus close at all (minimum focus distances are more like 40 cm), so it's not really likely in your use case - the device should avoid switching to the tele lens when the scene is so close that the tele camera can't focus on it.

- Eddy

[Javier Gerardo Martinez Salomon]

I see, thanks for the insights Eddy,

What do you think about my report on `FOCAL_LENGTH` ?

are the values reported by this API accurate?

could this value(or any other) be used to determine the likelihood of encountering general issues while focusing on the documents?

Just to be clear, there's nothing exposed by CameraX that could help to calculate the likelihood of focus issues due to these conditions and it's rather a situation that must be
reviewed per device. Is that right

[Javier Gerardo Martinez Salomon]

Hi guys,

I'm still struggling a bit to get the best focus possible for this kind of devices, specially when it comes to performing manual focus, I've observed that in normal conditions the CameraX AF routine does a fairly good job at focusing in the ID(see picture in the first message), however AF struggles when there's a change in the distance of the target object and fails to re-focus automatically unless I issue a manual `FocusMeteringAction`, when I do that, I get worse results than the ones I would get with AF, that is, the soft edges are way more noticeable.

I am wondering, what is the best way of issuing a manual `FocusMeteringAction` to focus on an ID such as the one attached to this thread?, this is what I'm doing right now.

1. Get the x and y coordinates from the center of the SurfaceView, since I assume that the object is centered as well.

2. I use a `SurfaceOrientedMeteringPointFactory` to build the MeteringPoint, I use the default size but have tried changing it to different values with the same results.

3. Then I use the focus point like this:
val meteringAction =
FocusMeteringAction.Builder(meteringPoint, FocusMeteringAction.FLAG_AF)
.setAutoCancelDuration(AF_AUTO_CANCEL_TIME_MS, TimeUnit.MILLISECONDS)
.build()
val focusMeteringFuture = cameraControl.startFocusAndMetering(meteringAction)

I am using CameraX 1.3.0 and a Pixel 6 for testing and I'd at least want my manual focus to be as good as the auto focus results, I've tried adding multiple focus points to the builder but I'm still getting the same results.

Thanks in advance