TFLite Boundless GAN for image extrapolation

[Sayak Paul]

Hola folks!

I hope you are doing well and a very Happy New Year!

Image extrapolation is an important topic of study specifically in areas like image editing, computer graphics, and computational photography. For realistic image extrapolation, the Boundless model was proposed. Today, I am delighted to share the TensorFlow Lite variants of the original Boundless model. Here are the sample outputs the model is capable of producing - 

Important links:

• Colab Notebook 
• TensorFlow Lite Models

Note the TensorFlow Lite models are metadata populated so if anyone wants to build a mobile application using them, proceed right away 😃 

Sayak Paul | sayak.dev

[Kshitiz Rimal]

Nice work, Sayak! It's always amazing to read your collab and work you do. 

Can you shed more light on the usage of Represenative dataset on this notebook? And if I someone need to run this on mobile app, besides the TF Lite model, what else do we need for better performance?

Regards,

Kshitiz Rimal

LinkedIn

On 9 Jan 2021, at 8:36 AM, Sayak Paul <spsay...@gmail.com> wrote:

Hola folks!

I hope you are doing well and a very Happy New Year!

Image extrapolation is an important topic of study specifically in areas like image editing, computer graphics, and computational photography. For realistic image extrapolation, the Boundless model was proposed. Today, I am delighted to share the TensorFlow Lite variants of the original Boundless model. Here are the sample outputs the model is capable of producing - 

<image.png>

Important links:

• Colab Notebook 
• TensorFlow Lite Models

Note the TensorFlow Lite models are metadata populated so if anyone wants to build a mobile application using them, proceed right away 😃 

Sayak Paul | sayak.dev

--
You received this message because you are subscribed to the Google Groups "ML on Mobile OS Working Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mlwg-mobile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mlwg-mobile/CAGa_XGF7pJ0hope4JJ1H5dRwpV4zi4He753Yix0%3D9%2B7pOr5Xbg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Sayak Paul]

Thank you, Kshitiz. 

The use of the representative dataset helps the TFLite converter to estimate the dynamic range of integer quantized activations. This further helps the converter to almost accurately map the continuous range of the activations to a discretized range. This is true for all the instances where you would see a representative dataset is being used. But if your model is originally trained using Quantization Aware Training (QAT) then passing a representative dataset during integer quantization is not required. This is because, during QAT, the ranges are already pretty well captured. Let me know if you have any more questions. 

> And if I someone need to run this on mobile app, besides the TF Lite model, what else do we need for better performance?

I think I am not experienced enough to comment on this yet but from my collaborations, here's what I have gathered - 

* Since the TFLite model is constructed with flex operators, it can actually increase the APK size quite a bit since the TFLite interpreter tends to include all the operators that might not be needed to build your final APK. The trick here is to select a specific set of ops and it can drastically reduce the APK size. More on this is described here. 

* Here's another guide provided by the TFLite team on reducing the TFLite binary size. 

@Khanh LeViet could you chime in here if you have any more suggestions on this? 

Sayak Paul | sayak.dev

[Kshitiz Rimal]

Thanks, Sayak! As always great work! :) 

Regards,

Kshitiz Rimal

LinkedIn

On 9 Jan 2021, at 9:30 AM, Sayak Paul <spsay...@gmail.com> wrote:

Thank you, Kshitiz. 

The use of the representative dataset helps the TFLite converter to estimate the dynamic range of integer quantized activations. This further helps the converter to almost accurately map the continuous range of the activations to a discretized range. This is true for all the instances where you would see a representative dataset is being used. But if your model is originally trained using Quantization Aware Training (QAT) then passing a representative dataset during integer quantization is not required. This is because, during QAT, the ranges are already pretty well captured. Let me know if you have any more questions. 

> And if I someone need to run this on mobile app, besides the TF Lite model, what else do we need for better performance?

I think I am not experienced enough to comment on this yet but from my collaborations, here's what I have gathered - 

* Since the TFLite model is constructed with flex operators, it can actually increase the APK size quite a bit since the TFLite interpreter tends to include all the operators that might not be needed to build your final APK. The trick here is to select a specific set of ops and it can drastically reduce the APK size. More on this is described here. 

* Here's another guide provided by the TFLite team on reducing the TFLite binary size. 

@Khanh LeViet could you chime in here if you have any more suggestions on this? 

Sayak Paul | sayak.dev

On Sat, Jan 9, 2021 at 9:04 AM Kshitiz Rimal <kshitizr...@gmail.com> wrote:

Nice work, Sayak! It's always amazing to read your collab and work you do. 

Can you shed more light on the usage of Represenative dataset on this notebook? And if I someone need to run this on mobile app, besides the TF Lite model, what else do we need for better performance?

Regards,

Kshitiz Rimal

LinkedIn
<email-sign-small.jpg>