Is CameraX supported Time Of Flight (TOF) camera?

James Bedford

unread,

May 23, 2019, 4:42:26 AM5/23/19

to Android CameraX Discussion Group

Hi,

As you know new phones Samsung Galaxy S10 5G Ultimate Edition , Huawie P30Pro , Oppo RX17 Pro have Depth Or Time Of Flight Camera (TOF)

I worked on Camera2 API and read format ImageFormat.DEPTH16 base of capability CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES_DEPTH_OUTPUT

But all depth frame data is wrong. base on below link read data but seems give wrong copy of RGB data in Camera2!.

https://developer.android.com/reference/android/graphics/ImageFormat.html#DEPTH16

Any solution available on CameraX for get Depth data?

How i can read Depth information as millimeter from CameraX?

Regards,

James

Vinit Modi

unread,

May 23, 2019, 1:15:58 PM5/23/19

to James Bedford, Android CameraX Discussion Group

Hi James,

Thanks for the questions.

1. Re: devices

We currently do not mandate depth output across manufacturer devices that support this capability. Each manufacturer may choose to expose or not expose depth specific information to developers.

2. When you say you are getting incorrect data from camera2, can you please send over data, logs, etc. on a specific device? Which device are you using to collect DEPTH16 and where do you see the information not generated in millimeters?

3. Depth in CameraX

Currently CameraX does not have support for depth outputs. We've prioritized stabilizing the API and focused on older devices (Android L, M, N, etc.) to ensure LEGACY compatibility at this initial stage. I do not currently have a timeline for you but this is valuable feedback to add in depth support.

Thanks!

Vinit

--
You received this message because you are subscribed to the Google Groups "Android CameraX Discussion Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camerax-develop...@android.com.
To post to this group, send email to camerax-d...@android.com.
To view this discussion on the web visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/d46b4992-960a-40b8-8453-b7b93f535882%40android.com.

James Bedford

unread,

May 23, 2019, 2:58:35 PM5/23/19

to Android CameraX Discussion Group, james....@nepox.com

Hi Vinit,

Thank you for quick response.

1- Device is Huawei P30 Pro with ToF camera in back

2- I customize Camera2Raw sample and get two result ImageFormat.JPEG and second output ImageFormat.DEPTH16

In this test i take phone from 80 cm from floor but output attached as file values is different i also implement IOS version with IPhoneX but that output is completely correct.

Buffer decode is exactly same as Camera2 API ImageFormat.Depth16 also Logical camera support CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES_DEPTH_OUTPUT

https://developer.android.com/reference/android/graphics/ImageFormat.html#DEPTH16

I attached source code and output log file thank you for fast follow.

Regards,

James

case ImageFormat.DEPTH16: {

// Print Sample

// ############################################################################################

int bufW = mImage.getWidth();

int butH = mImage.getHeight();

Log.i("TestCam", "Depth Image Width:" + String.valueOf( bufW ) + " Depth Image Height: " + String.valueOf( butH ) );

ShortBuffer shortDepthBuffer = mImage.getPlanes()[0].getBuffer().asShortBuffer();

// Print sample depth values

for ( int i = 0 ; i < 10000 ; i++ ){

short tdepthSample = shortDepthBuffer.get();

short tdepthRange = (short) (tdepthSample & 0x1FFF);

short tdepthConfidence = (short) ((tdepthSample >> 13) & 0x7);

float tdepthPercentage = tdepthConfidence == 0 ? 1.f : (tdepthConfidence - 1) / 7.f;

if ( tdepthPercentage != 0 )

Log.i("TestCam", "Sample Depth[" + String.valueOf(i) + "] -> " + String.valueOf( tdepthRange ) );

}

OUTPUT >>>

2019-05-23 21:36:08.506 22338-22858/com.example.android.camera2raw I/TestCam: Depth Image Width:992 Depth Image Height: 744

2019-05-23 21:36:08.507 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[273] -> 63

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[274] -> 63

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[275] -> 64

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[276] -> 64

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[277] -> 64

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[278] -> 64

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[279] -> 64

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[280] -> 64

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[281] -> 64

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[282] -> 64

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[283] -> 64

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[284] -> 64

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[285] -> 64

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[286] -> 64

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[287] -> 321

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[288] -> 321

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[289] -> 321

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[290] -> 321

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[291] -> 321

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[292] -> 321

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[293] -> 321

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[294] -> 322

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[295] -> 834

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[296] -> 835

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[297] -> 835

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[298] -> 835

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[299] -> 835

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[300] -> 835

2019-05-23 21:36:08.508 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[301] -> 1091

2019-05-23 21:36:08.509 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[302] -> 1091

2019-05-23 21:36:08.509 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[303] -> 1092

2019-05-23 21:36:08.509 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[304] -> 1092

2019-05-23 21:36:08.509 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[305] -> 1092

2019-05-23 21:36:08.509 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[306] -> 1092

2019-05-23 21:36:08.509 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[307] -> 1092

2019-05-23 21:36:08.509 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[308] -> 1092

2019-05-23 21:36:08.509 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[309] -> 1092

2019-05-23 21:36:08.509 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[310] -> 1092

2019-05-23 21:36:08.509 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[311] -> 1092

2019-05-23 21:36:08.509 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[312] -> 1092

2019-05-23 21:36:08.509 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[313] -> 1092

2019-05-23 21:36:08.509 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[314] -> 1092

2019-05-23 21:36:08.509 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[315] -> 1092

2019-05-23 21:36:08.509 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[316] -> 1092

2019-05-23 21:36:08.509 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[317] -> 1092

2019-05-23 21:36:08.509 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[318] -> 1092

2019-05-23 21:36:08.509 22338-22858/com.example.android.camera2raw I/TestCam: Sample Depth[319] -> 1092

On Thursday, May 23, 2019 at 8:15:58 PM UTC+3, Vinit Modi wrote:

Hi James,

Thanks for the questions.

1. Re: devices
We currently do not mandate depth output across manufacturer devices that support this capability. Each manufacturer may choose to expose or not expose depth specific information to developers.

2. When you say you are getting incorrect data from camera2, can you please send over data, logs, etc. on a specific device? Which device are you using to collect DEPTH16 and where do you see the information not generated in millimeters?

3. Depth in CameraX
Currently CameraX does not have support for depth outputs. We've prioritized stabilizing the API and focused on older devices (Android L, M, N, etc.) to ensure LEGACY compatibility at this initial stage. I do not currently have a timeline for you but this is valuable feedback to add in depth support.

Thanks!
Vinit

On Thu, May 23, 2019 at 1:42 AM James Bedford <james....@nepox.com> wrote:

Hi,

As you know new phones Samsung Galaxy S10 5G Ultimate Edition , Huawie P30Pro , Oppo RX17 Pro have Depth Or Time Of Flight Camera (TOF)

I worked on Camera2 API and read format ImageFormat.DEPTH16 base of capability CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES_DEPTH_OUTPUT

But all depth frame data is wrong. base on below link read data but seems give wrong copy of RGB data in Camera2!.

https://developer.android.com/reference/android/graphics/ImageFormat.html#DEPTH16

Any solution available on CameraX for get Depth data?

How i can read Depth information as millimeter from CameraX?

Regards,

James

--
You received this message because you are subscribed to the Google Groups "Android CameraX Discussion Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to camerax-developers+unsub...@android.com.

camera2raw_output.rtf

IMG_5782.HEIC

Camera2RawFragment.java

Vinit Modi

unread,

May 23, 2019, 10:07:22 PM5/23/19

to James Bedford, Android CameraX Discussion Group

I've filed an internal tracking bug with the camera2 framework team, b/133455587, will update once we hear back.

Vinit

To unsubscribe from this group and stop receiving emails from it, send an email to camerax-develop...@android.com.

To post to this group, send email to camerax-d...@android.com.
To view this discussion on the web visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/d46b4992-960a-40b8-8453-b7b93f535882%40android.com.

--

You received this message because you are subscribed to the Google Groups "Android CameraX Discussion Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to camerax-develop...@android.com.

To post to this group, send email to camerax-d...@android.com.

To view this discussion on the web visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/c4c4f620-392b-40ef-8c78-6e43e1d3af16%40android.com.

James Bedford

unread,

May 24, 2019, 11:56:04 AM5/24/19

to Android CameraX Discussion Group, james....@nepox.com

Thank you so much

To unsubscribe from this group and stop receiving emails from it, send an email to camerax-developers+unsub...@android.com.

To post to this group, send email to camerax-d...@android.com.
To view this discussion on the web visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/d46b4992-960a-40b8-8453-b7b93f535882%40android.com.

--
You received this message because you are subscribed to the Google Groups "Android CameraX Discussion Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to camerax-developers+unsub...@android.com.

Matt Fischler

unread,

May 25, 2019, 1:15:48 PM5/25/19

to James Bedford, Android CameraX Discussion Group

Hi James,

So you can get the DEPTH16 frames with Huawei P30 but you're sure that the data is incorrect? I've never seen any commercial Android phone yet that can access a depth CameraID that has DEPTH_OUTPUT capability. 992x744 seems like a rather large size for a depth image.

Matt

To unsubscribe from this group and stop receiving emails from it, send an email to camerax-develop...@android.com.

To post to this group, send email to camerax-d...@android.com.
To view this discussion on the web visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/d46b4992-960a-40b8-8453-b7b93f535882%40android.com.

--
You received this message because you are subscribed to the Google Groups "Android CameraX Discussion Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to camerax-develop...@android.com.

To post to this group, send email to camerax-d...@android.com.
To view this discussion on the web visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/c4c4f620-392b-40ef-8c78-6e43e1d3af16%40android.com.

--

You received this message because you are subscribed to the Google Groups "Android CameraX Discussion Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to camerax-develop...@android.com.

To post to this group, send email to camerax-d...@android.com.

To view this discussion on the web visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/66d22866-b5a3-47ff-939a-ee30adcea23a%40android.com.

James Bedford

unread,

May 26, 2019, 5:21:22 AM5/26/19

to Android CameraX Discussion Group, james....@nepox.com

Hi Matt,

Yes I'm sure because I work about 6 month on IPhone X depth camera and in IOS all Depth samples is so accurate in millimetre.

So I do complete test and debug. first I measure distance from wall as attached photos distance is 60 cm in.

Second step take photo with Huawei P30 Pro and save Depth16 frame and JPG frame output as camera2 reference manual.( File Name:DEBUG_DEPTH_2019_05_26_11_48_17_606.txt.zip , JPG_2019_05_26_11_48_17_606.jpg )

In attachment you can find debug all Depth16 pixel include all variable.and main source code also attached.

Buffer decode is base of this link:

https://developer.android.com/reference/android/graphics/ImageFormat.html#DEPTH16

Regards,

James

To unsubscribe from this group and stop receiving emails from it, send an email to camerax-developers+unsub...@android.com.

To post to this group, send email to camerax-d...@android.com.
To view this discussion on the web visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/d46b4992-960a-40b8-8453-b7b93f535882%40android.com.

--
You received this message because you are subscribed to the Google Groups "Android CameraX Discussion Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to camerax-developers+unsub...@android.com.

To post to this group, send email to camerax-d...@android.com.
To view this discussion on the web visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/c4c4f620-392b-40ef-8c78-6e43e1d3af16%40android.com.

--
You received this message because you are subscribed to the Google Groups "Android CameraX Discussion Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to camerax-developers+unsub...@android.com.

DEBUG_DEPTH_2019_05_26_11_48_17_606.txt.zip

JPG_2019_05_26_11_48_17_606.jpg

Measurement Image 2019-05-26 at 11.56.18.jpeg

Measurement Image 2019-05-26 at 11.56.18(1).jpeg

Measurement Image 2019-05-26 at 11.56.18(2).jpeg

Measuremnet Image 2019-05-26 at 11.56.18(3).jpeg

Camera2RawFragment.java

Jaehyeon Lee

unread,

Mar 6, 2020, 2:48:42 AM3/6/20

to Android CameraX Discussion Group, james....@nepox.com

Hi,

Are there any updates about this subject?

I have a use case to get depth data from ToF censor while taking normal pictures,

so James's implementation with Camera2Raw sample is perfect with me.

But I got some error while testing on my device though, Galaxy Note 10+.

So, can I read depth data using CameraX API?(by Extension?)

or should I use Camera 2 API to access and open ToF camera?

Regards,

Lee

Vinit Modi

unread,

Mar 6, 2020, 12:17:29 PM3/6/20

to Jaehyeon Lee, Android CameraX Discussion Group, James Bedford

Hi Lee,

Accessing depth data is currently only supported via camera2 API. Thanks!

Vinit

--

You received this message because you are subscribed to the Google Groups "Android CameraX Discussion Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to camerax-develop...@android.com.
To view this discussion on the web visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/c3ec4608-59e9-4145-84bb-fe9410fe6f80%40android.com.

Reply all

Reply to author

Forward