Are multiple GPU delegates supported?

28 views
Skip to first unread message

Iosif Hamlatzis

unread,
Oct 6, 2022, 7:25:46 AMOct 6
to TensorFlow Lite
Does tflite support multiple GPU delegates?

Let's say I have a model and from all ops in the model only one is not supported on the GPU, but the rest are.

As I understand in the current version one partition of ops up to the unsupported op will run as one node in the GPU and the rest will run on the CPU

On the other hand if multiple GPU delegates are supported, the first partition will run on the GPU, then only one op will run on the CPU and the rest ops will be run again on the GPU

It will be something like: "GPU ops -> CPU op -> GPU ops"
Is this currently supported or  at least can easily modify tflite to support it?

thnx

iosif

Weiyi Wang

unread,
Oct 6, 2022, 12:31:38 PMOct 6
to TensorFlow Lite, i.ham...@gmail.com
By default, GPU delegate only delegates 1 partition. But you can configure the maximum number of partitions GPU delegate here with max_delegated_partitions in C/C++.
Reply all
Reply to author
Forward
0 new messages