TFLite GPU Delegate is now available via TFLite with Google Play Services Beta

[Thomas Ezan]

Hi TensorFlow Lite users,

Earlier this year we announced a new Google Play Services API for TensorFlow Lite on Android in Beta. This API supported CPU, XNNPack and NNAPI, but GPU delegate wasn’t supported. 

We’re happy to announce that it now supports the GPU delegate. This means you can now execute your models on GPU without having to bundle GPU libraries to your app. We expect savings up to 5MB in APK size by migrating to TensorFlow Lite in Play Services, if you were using the bundled TFLite and GPU delegate on Android. Most migrations are straightforward. If you’re interested, go to the TFLite documentation to learn how to migrate. 

TFLite in Google Play Services is already being used by many first party Google apps as well as apps by external developers. It is serving a significant volume of users in production at Google scale.

The Android ML team is also accepting applications for early access to the Acceleration Service for Android, a set of API to enable you to select the optimal hardware accelerator available on-device. Read more about it here.

Thanks,

Thomas on behalf of the TensorFlow Lite and Android ML Platform teams.

Thomas Ezan Senior Developer Relations Engineer ‬

[Iosif Hamlatzis]

Does that mean select_ops will be supported in GPU?

[Thomas Ezan]

No, the supported operations remain the same. 

Why are you specifically asking about select_ops?

Thomas Ezan | Sr Developer Relations Engineer

[Nan YE]

I applied Acceleration Service for Android a few months ago, but haven't got any replies. Does that mean I am failed to be selected?  Should I apply for it again or just wait? I am really interested in that API.

[Thomas Ezan]

Hi,

We have had a lot of interest so we have been inviting teams based on availability. But we are aiming to get back to everyone and will get back to you as soon as possible. 

Thanks,

Thomas Ezan | Sr Developer Relations Engineer

[Iosif Hamlatzis]

because using select_ops on device means also training, if they can use GPU it means acceleration. Otherwise everyone will have to implement in GPU (OpenGL, OpenCL, Vulkun or Metal) the missing ops to have accelerated training