Is Flex delegate compatible with CUDA?

Skip to first unread message

Iosif Hamlatzis

Jan 31, 2022, 8:49:16 AM1/31/22
to TensorFlow Lite
I have created a model with SELECT OPS but if I have CUDA enabled it crashes both under Python and C++, but works fine if CUDA isn't enabled

When CUDA is enabled (and only then) I get an error message 

"op 'ReadVariableOp' output%d tensor#%d shape mismatch for [144] != []" from file "./tensorflow/lite/delegates/flex/" at line 590 and then it goes to file "./tensorflow/lite/delegates/flex/" in method "CopyFromBufferHandle" in line 160
and crashes "memcpy(output->data.raw,, t_data.size());"

Any idea?

Terry (Woncheol) Heo

Mar 2, 2022, 1:49:43 PM3/2/22
to TensorFlow Lite,
Since the main target of TFLite is mobile and IoT devices, CUDA is not our major target environment.
So the using CUDA for SELECT OPS is not verified.

But if you're interested in GPU acceleration of mobile, you could use OpenCL for TFLite kernels.
Reply all
Reply to author
0 new messages