Unsupported data type and operators in TFLite model while running with Android GPU

[Simon King]

Hi TFLite team and friends,

I am testing a TFLite model with the Google Pixel XL smartphone installed Android 8.1. I followed this example (https://www.tensorflow.org/lite/examples/image_classification/overview) to deploy my model in an Android app. While testing it with CPU, my model works good, but it got errors with GPU.

I attached the error message below, which basically reports two types of errors. The first error is about data type. My model uses tf.int32 and I think the GPU APIs only accept tf.uint8 and tf.float32. I can easily address this error with changing type to uint8. However, I used tf.gather operator in my model. There are two problems with this operator: (1) it's not supported and (2) it only accepts tf.int32 and tf.int64 types. To address the problem, I tried to find the alternative op but I cannot find one supported for the required data type. 

java.lang.RuntimeException: Unable to start activity ComponentInfo{com.example.wb/com.example.wb.MainActivity}: java.lang.IllegalArgumentException: Internal error: Failed to apply delegate: Following operations are not supported by GPU delegate: ADD: OP is supported, but tensor type isn't matched! GATHER: Operation is not supported. MUL: OP is supported, but tensor type isn't matched! RESHAPE: OP is supported, but tensor type isn't matched! No operations will run on the GPU, and all 372 operations will run on the CPU. ModifyGraphWithDelegate is disallowed when graph is immutable. 

How could I resolve the problem? I was also thinking about create customized op in both Tensorflow (for model conversion) and TFLite. But I'd like to know if there is easier solution first.

[Sachin Joglekar]

Hey Simon, a few questions:

1. What ops are present in your model?

2. What is the data type of the tensors flowing through these ops?

If you could share your model (even if untrained), that might help us figure it out faster.

Based on the LOG "No operations will run on the GPU, and all 372 operations will run on the CPU.", it seems like the GPU delegate isn't able to find any compatible op in your model.

[Simon King]

Hi Sachin,

The ops in the model are tf.math.add, tf.math.mul, tf.gather, and tf.reshape. The data type used in tf.math.add, tf.math.mul, and tf.reshape is tf.int32 which is reported as unsupported. I plan to change it to tf.uint8 to address the problem.

However, tf.gather is not supported in GPU delegate and it has to use tf.int32 and tf.int64 for the indice argument. I am not able to find the alternative op for it which is expected to be supported in GPU delegate and accepts tf.uint8.

[Simon King]

How can I share model to you? Adding it in attachment? Are there any more private ways?

[Sachin Joglekar]

Trying uint8 datatype makes sense. Its okay if an op or two isn't supported, as long as there are few instances of the op in the model.

--

Sachin Joglekar |

srjoglekar@google.com|

|

+1-650-309-9486