How to get precise focal length and sensor infomation

[Masa]

Hi guys,

I am considering using photos taken with a smartphone for practical measurements, such as in Structure-from-Motion (SFM). To use them for measurements, information such as the focal length, image sensor size, and pixel count is essential. However, when using the setZoomRatio function to zoom in or out while taking photos, the configured zoomRatio is reflected in the picture, causing it to be enlarged or reduced. Unfortunately, this change is not reflected in the Exif data's focal length.

Moreover, in cases where multiple cameras are present, as in the Pixel 7a, changing the zoomRatio from 1.0 to 0.99 not only alters the focal length but also significantly affects the image size. Additionally, when examining CameraID and PhysicalCameraID on a Pixel 7a, it is evident that there are multiple sensors, making it necessary to know which sensor was used.

To accurately extract focal length and sensor information from photos taken, what steps should be taken?

Thanks,

The specifications for the Pixel 7a are as follows:

 CameraId = 0

 facing = 1(facingFront = 0, facingBack = 1)
 isLogicalCamera = true
 focal length = 5.43
 sensor size = 7.3984x5.5552
 pixel size = 4624x3472
 
 physical CameraID = 2
  focal length = 5.43
  sensor size = 7.3984x5.5552
  pixel size = 4624x3472
 physical CameraID = 3
  focal length = 1.835
  sensor size = 4.71296x3.4944
  pixel size = 4208x3120
 physical CameraID = 4
  focal length = 5.43
  sensor size = 3.6992x2.7776
  pixel size = 4624x3472

CameraId = 1
 facing = 0(facingFront = 0, facingBack = 1)
 isLogicalCamera = false
 focal length = 2.74
 sensor size = 4.71296x3.44064
 pixel size = 4208x3072

[Eino-Ville Talvala]

Hi Masa,

I don't think CameraX directly exposes this information currently, so you probably need to use the camera2 interop interface to receive CaptureResult objects for each completed frame.

Then you can look at the ACTIVE_PHYSICAL_ID entry to determine which specific image sensor that frame came from.  This information should be present on a Pixel 7a; unfortunately it's not yet consistently reported by all Android devices that have multi-camera clusters.

Note that each physical camera also has calibration information that may be useful to you (camera orientation, position, intrinsics, etc) which include more-accurate focal length estimates within them.

These values are best used from the capture results since they may change based on the camera's focus distance, physical orientation, and device age.  The top-level focal length number is the design value, not necessarily the calibrated value available in the intrinsics values.

Good luck!

- Eddy 

--
You received this message because you are subscribed to the Google Groups "Android CameraX Discussion Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camerax-develop...@android.com.
To view this discussion on the web visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/8b0bc16f-16e2-49be-902b-81c4a74fc3b9n%40android.com.

[Kailiang Chen]

Hi Masa,

We are working on exposing physical camera info from CameraX as official public APIs. 

Potentially this info could be used for different computer vision tasks (e.g. multiview, stereo, sfm).

Could you share a list of camera characteristics or other info (e.g. camera id, info in camera capture results), you want CameraX to expose?

That would be helpful for us to design multi-camera APIs in our next version.

As Eddy mentioned, Camera2Interop is the only way you can get these from the CameraX library now.

To view this discussion on the web visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/CAB8xOM9JUBV4A7FKGFLsvooSGmeDvX9JAUy%3DrKvcaiOyzCo_WQ%40mail.gmail.com.

[masakazu...@futu-re.co.jp]

Eddy-san, Kailiang-san,

Thank you for your response.

Thank you for providing information on obtaining camera orientation, position, and intrinsics. This information is very valuable to me as it is crucial for performing SFM and similar tasks.

As a beginner in Kotlin, Android Studio, CameraX, and Camera2, I was unsure how to use Camera2Interop from CameraX, so I tried various things using the Android Camera2Basic Sample. It took me quite some time to get Camera2Basic running, so I apologize for the delayed response.

By modifying the source code of Camera2Basic, I was able to obtain the following data from the Pixel 7a:

cameraId = 0

    orientation = Back

    SENSOR_INFO_PIXEL_ARRAY_SIZE = 4624x3472

    SENSOR_INFO_PHYSICAL_SIZE  = 7.3984x5.5552

    focalLengths = 5.43

    physicalCameraID = 2

       orientation = Back

       SENSOR_INFO_PIXEL_ARRAY_SIZE = 4624x3472

       SENSOR_INFO_PHYSICAL_SIZE  = 7.3984x5.5552

       focalLengths = 5.43

    physicalCameraID = 3

       orientation = Back

       SENSOR_INFO_PIXEL_ARRAY_SIZE = 4208x3120

       SENSOR_INFO_PHYSICAL_SIZE  = 4.71296x3.4944

       focalLengths = 1.835

    physicalCameraID = 4

       orientation = Back

       SENSOR_INFO_PIXEL_ARRAY_SIZE = 4624x3472

       SENSOR_INFO_PHYSICAL_SIZE  = 3.6992x2.7776

       focalLengths = 5.43

cameraId = 1

    orientation = Front

    SENSOR_INFO_PIXEL_ARRAY_SIZE = 4208x3072

    SENSOR_INFO_PHYSICAL_SIZE = 4.71296x3.44064

    focalLengths = 2.74

 

Min zoom: 0.5304938, Max zoom: 8.0

Next, by changing the value of CaptureRequest.CONTROL_ZOOM_RATIO, I conducted a parameter study and obtained the results shown in the attached Excel file.

I found that the switching between physicalCameraID=3 and 2 occurs at CONTROL_ZOOM_RATIO=1, and physicalCameraID=2 and 4 switch at CONTROL_ZOOM_RATIO=2. As the zoom ratio of the captured photos increases, the images are continuously enlarged, but the intrinsics do not follow suit.

Therefore, I have the following questions:

1. Since the image size changes with the set zoom ratio, I believe the intrinsics should also change accordingly. However, they do not change in this manner. In particular, at CONTROL_ZOOM_RATIO=1, where physicalCameraID=3 and 2 switch, the switch occurs discontinuously. I think the intrinsics need to be adjusted for use in SFM. How should I modify them?
2. Although the Pixel 7a has only two lenses on the back, why do I see three physicalCameraIDs?
3. The SENSOR_INFO_PIXEL_ARRAY_SIZE for physicalCameraID=3 is 4208x3120, but why does the captured image have a size of 4624x3472?
4. physicalCameraID=4 has the same focal length as physicalCameraID=2, and the SENSOR_INFO_PHYSICAL_SIZE is half that of physicalCameraID=2, yet the intrinsics are similar to those of physicalCameraID=2. It seems that physicalCameraID=2 and 4 are using the same sensor image. Why are they separated like this?

By the way, when I tried with the SHARP sense5g, which has three lenses, I could retrieve neither physicalCamera nor intrinsics.

To Kailiang-san, I would be delighted if CameraX had APIs that allow me to obtain camera orientation, position, intrinsics, and lens distiotion, with the zoom ratio reflected.

Also, I have another question. I've been attempting to set the white balance manually, using Pixel7a and setting CaptureResult.CONTROL_AWB_MODE to CONTROL_AWB_MODE_OFF, CONTROL_AWB_MODE_FLUORESCENT, and CONTROL_AWB_MODE_SHADE.

As a result, when CONTROL_ZOOM_RATIO is set to 1, CaptureResult.CONTROL_AWB_MODE remains CaptureResult.CONTROL_AWB_MODE_AUTO and does not change. However, when CONTROL_ZOOM_RATIO is set to values other than 1, such as 0.6, 0.9, 1.001, 2, 3, or 8, CaptureResult.CONTROL_AWB_MODE becomes the value specified in CaptureRequest.CONTROL_AWB_MODE.

Is there anything else I need to do when CONTROL_ZOOM_RATIO is set to 1?

 

I apologize for the lengthy text. Thank you.

Regards,

Masa,

--
You received this message because you are subscribed to a topic in the Google Groups "Android CameraX Discussion Group" group.
To unsubscribe from this topic, visit https://groups.google.com/a/android.com/d/topic/camerax-developers/HR6D7OR7jzU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to camerax-develop...@android.com.
To view this discussion on the web visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/CAJy3%2Bk_EGQWpzuuW6CJWLZYPP53ntG_ULdHHo2%3D3HC5YfyhR7w%40mail.gmail.com.