Are multiple GPU delegates supported?

Skip to first unread message

Iosif Hamlatzis

Oct 6, 2022, 7:25:46 AM10/6/22
to TensorFlow Lite
Does tflite support multiple GPU delegates?

Let's say I have a model and from all ops in the model only one is not supported on the GPU, but the rest are.

As I understand in the current version one partition of ops up to the unsupported op will run as one node in the GPU and the rest will run on the CPU

On the other hand if multiple GPU delegates are supported, the first partition will run on the GPU, then only one op will run on the CPU and the rest ops will be run again on the GPU

It will be something like: "GPU ops -> CPU op -> GPU ops"
Is this currently supported or  at least can easily modify tflite to support it?



Weiyi Wang

Oct 6, 2022, 12:31:38 PM10/6/22
to TensorFlow Lite,
By default, GPU delegate only delegates 1 partition. But you can configure the maximum number of partitions GPU delegate here with max_delegated_partitions in C/C++.
Reply all
Reply to author
0 new messages