Let there be light!

[Sayak Paul]

Hi folks,

I hope y'all are doing well. Today, I am delighted to share the TensorFlow Lite variants for the MIRNet model -- a model capable of enhancing low-light images to great extents. 

Find the models on TensorFlow Hub - https://tfhub.dev/sayakpaul/mirnet-fixed/1. Also, here's the project repository - https://github.com/sayakpaul/MIRNet-TFLite. 

Exciting, ain't it? 

Sayak Paul | sayak.dev

[Thierno Ibrahima DIOP]

Nice work sayak!

Sent from my iPhone

On 15 Dec 2020, at 15:33, Sayak Paul <spsay...@gmail.com> wrote:



Hi folks,

I hope y'all are doing well. Today, I am delighted to share the TensorFlow Lite variants for the MIRNet model -- a model capable of enhancing low-light images to great extents. 

<lol_results.gif>

Find the models on TensorFlow Hub - https://tfhub.dev/sayakpaul/mirnet-fixed/1. Also, here's the project repository - https://github.com/sayakpaul/MIRNet-TFLite. 

Exciting, ain't it? 

Sayak Paul | sayak.dev

--
You received this message because you are subscribed to the Google Groups "ML on Mobile OS Working Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mlwg-mobile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mlwg-mobile/CAGa_XGEGxTGU6dyoeKdxgUTVJQ3P%3DRtdbA3X20SVP0r--8iKug%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Margaret Maynard-Reid]

This is amazing, Sayak. This model will be very useful for camera apps.

Margaret

On Tue, Dec 15, 2020 at 7:51 AM Veysel Kocaman <vkoc...@gmail.com> wrote:

Really impressive !

You received this message because you are subscribed to the Google Groups "ML GDE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ml-gde+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ml-gde/C2B68E57-0E5D-4A86-8071-FC4AB310C03D%40gmail.com.

--
You received this message because you are subscribed to the Google Groups "ML GDE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ml-gde+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ml-gde/A0AFBB4C-DD01-4E41-B7FD-CBB7C0320F77%40gmail.com.

[Srivatsan Srinivasan]

Sayak,

This is really impressive work. Also Kudos to chasing every prominent models out there and getting it on tflite 👍

--
You received this message because you are subscribed to the Google Groups "ML GDE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ml-gde+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ml-gde/CAGa_XGEGxTGU6dyoeKdxgUTVJQ3P%3DRtdbA3X20SVP0r--8iKug%40mail.gmail.com.

[Sayak Paul]

Thanks much everyone for your kind words. I hope the models would be useful to the community. 

Sayak Paul | sayak.dev

[Lu Wang]

Very exciting! Nice work Sayak!

This model is also compatible with the Android ML Model binding feature, meaning users can integrate the model by simply dragging the model into AS.

+Xunkai Zhang +Lei Yu +Hoi Lam

You received this message because you are subscribed to the Google Groups "ML on Mobile OS Working Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mlwg-mobile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mlwg-mobile/CAGa_XGEjYwMYpm1128ub2-Nu4Rbbpufs5j5qNRRkyW5f_NF%3D5w%40mail.gmail.com.

[jerr...@gmail.com]

Nice job. 

 

Can I get this for my windshield?

 

From: mlwg-...@googlegroups.com <mlwg-...@googlegroups.com> On Behalf Of Sayak Paul
Sent: Tuesday, December 15, 2020 7:33 AM
To: ML GDE <ml-...@googlegroups.com>; ML on Mobile OS Working Group <mlwg-...@googlegroups.com>
Subject: Let there be light!

 

Hi folks,

 

I hope y'all are doing well. Today, I am delighted to share the TensorFlow Lite variants for the MIRNet model -- a model capable of enhancing low-light images to great extents. 

 

 

Find the models on TensorFlow Hub - https://tfhub.dev/sayakpaul/mirnet-fixed/1. Also, here's the project repository - https://github.com/sayakpaul/MIRNet-TFLite. 

 

Exciting, ain't it? 

Sayak Paul | sayak.dev

 

--

You received this message because you are subscribed to the Google Groups "ML on Mobile OS Working Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mlwg-mobile...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mlwg-mobile/CAGa_XGEGxTGU6dyoeKdxgUTVJQ3P%3DRtdbA3X20SVP0r--8iKug%40mail.gmail.com.

[Soonson Kwon]

Sayak: this is intuitive and will be very useful for camera apps as well as image editing apps.

If any new or existing apps incorporate this model, please keep us posted.

I am sure many folks in this thread will love to try out those apps! :-)

Lei: thanks for getting it added to the premium list as well!

This is a great collaboration.

On Wed, Dec 16, 2020 at 3:41 AM 'Christiaan Prins' via ML GDE <ml-...@googlegroups.com> wrote:

Very cool, congrats on publishing the model, looks very impressive!

Lei, thanks for getting it added to the premium list, so TF Hub will guide developers to Android Studio.

--Christiaan

On Tue, Dec 15, 2020 at 10:06 AM Lei Yu <jackq...@google.com> wrote:

Great work! +Shiyu Hu +Yi Zhou +Christiaan Prins 

Yeah currently ML Model BInding should support it however I will also have a sanity test before I add it into our premium model list

To view this discussion on the web visit https://groups.google.com/d/msgid/ml-gde/CABDesLOKA7opQa5haCNwrC9WcvnL8gbm3xCyq9r%3DTYjHcecQuA%40mail.gmail.com.

[Sayak Paul]

Sure thing!

Thanks for adding it to the list (I had no idea :D). 

Currently, the model has some amount of latency. I plan to work on compressing the model next year. 

Sayak Paul | sayak.dev

[Kshitiz Rimal]

Very cool, Sayak! Congratulations!

<lol_results.gif>

Find the models on TensorFlow Hub - https://tfhub.dev/sayakpaul/mirnet-fixed/1. Also, here's the project repository - https://github.com/sayakpaul/MIRNet-TFLite. 

Exciting, ain't it? 

Sayak Paul | sayak.dev

-- 
You received this message because you are subscribed to the Google Groups "ML GDE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ml-gde+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ml-gde/CAGa_XGEGxTGU6dyoeKdxgUTVJQ3P%3DRtdbA3X20SVP0r--8iKug%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups "ML on Mobile OS Working Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mlwg-mobile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mlwg-mobile/CAGa_XGEjYwMYpm1128ub2-Nu4Rbbpufs5j5qNRRkyW5f_NF%3D5w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "ML GDE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ml-gde+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ml-gde/CABDesLOKA7opQa5haCNwrC9WcvnL8gbm3xCyq9r%3DTYjHcecQuA%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups "ML on Mobile OS Working Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mlwg-mobile...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mlwg-mobile/CAGa_XGE_tEZq2FM1hYvJPg5LYeui_RqBd1TLDbCyzys6od0m_g%40mail.gmail.com.

[Kshitiz Rimal]

I evaluated the model with some new images taken from the scenes of The Batman teaser. It looks pretty good :)

Amazing work, Sayak! This will be very useful.

[Sayak Paul]

Yuhu!

Stoked to see the results, Kshitij. Thank you for sharing. 

Looking forward to the movie too :D 

Sayak Paul | sayak.dev

[Sayak Paul]

Hi folks,

As you might have noticed the TensorFlow Lite optimized models weren't fast enough to run on videos (low-light video enhancement is even more useful, ain't it?). I further optimized the model with TensorRT and now the inference latency has gone down to ~0.6 seconds from ~60 seconds (on a Tesla T4). I have tested it on a Jetson Nano (v1) as well and it runs pretty fast there too :D 

Here are the materials you might be interested to check out - 

• End to end Colab Notebook showing the TensorRT conversion process and inferencing on a video
• Demo of the model running on a video 

Enjoy and happy holidays! 

Sayak Paul | sayak.dev

[Kshitiz Rimal]

Great stuff, Sayak! It would be really great if we could integrate it in a flutter app using TF Lite. 

On 18 Dec 2020, at 2:31 PM, Sayak Paul <spsay...@gmail.com> wrote:

Hi folks,

As you might have noticed the TensorFlow Lite optimized models weren't fast enough to run on videos (low-light video enhancement is even more useful, ain't it?). I further optimized the model with TensorRT and now the inference latency has gone down to ~0.6 seconds from ~60 seconds (on a Tesla T4). I have tested it on a Jetson Nano (v1) as well and it runs pretty fast there too :D 

Here are the materials you might be interested to check out - 

• End to end Colab Notebook showing the TensorRT conversion process and inferencing on a video
• Demo of the model running on a video 

Enjoy and happy holidays! 

Sayak Paul | sayak.dev

On Tue, Dec 15, 2020 at 9:03 PM Sayak Paul <spsay...@gmail.com> wrote:

Hi folks,

I hope y'all are doing well. Today, I am delighted to share the TensorFlow Lite variants for the MIRNet model -- a model capable of enhancing low-light images to great extents. 

<lol_results.gif>

Find the models on TensorFlow Hub - https://tfhub.dev/sayakpaul/mirnet-fixed/1. Also, here's the project repository - https://github.com/sayakpaul/MIRNet-TFLite. 

Exciting, ain't it? 

Sayak Paul | sayak.dev

--
You received this message because you are subscribed to the Google Groups "ML on Mobile OS Working Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mlwg-mobile...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mlwg-mobile/CAGa_XGGjesVMMbJP4vf%2BqdhT6PhkhQ1JOpGWd6awXqvqi0B0FQ%40mail.gmail.com.

[Sayak Paul]

The problem is it's still very slow in TensorFlow Lite. Hence the use of TensorRT and plugging in a GPU.

Sayak Paul | sayak.dev

[M. Yusuf Sarıgöz]

Amazing work, Sayak. E2E TensorRT conversion  is a great example. Thank you for all the efforts you put into this.

Happy holidays :)

You received this message because you are subscribed to the Google Groups "ML GDE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ml-gde+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ml-gde/CAGa_XGEZUsSPn%3Dphhe1CKLWbuPD8%3DKMeikpSj0jMwGAOpW%2BH%2BA%40mail.gmail.com.

[Kshitiz Rimal]

Sayak, just curious if you perform any comparison between this model and traditional image processing techniques to adjust the brightness/contrast of images/videos? Comparison between the results and performance maybe?

[Sayak Paul]

Yes, I did it. I used a combination of exposure exposition, contour detection, etc. While in some cases, those work pretty well but they fail to generalize and that is when this model comes into the picture. 

Timing-wise, as you probably have guessed the pipeline consisting of the classical image processing ops would perform way faster than I presented in my last two emails :D 

Sayak Paul | sayak.dev

[Kshitiz Rimal]

Sounds interesting! Just thinking out loud here, if there could be a model that acts like a controller to the traditional image processing algorithms, that might speed up the output with comparable result may be. Like may be a weight matrix that can then be multiplied further to adjust the exposure, contour detection as required by the region of the image. Perhaps similar to a real-time image segmentation model. 

Thanks for your effort, Sayak! Inspiring work :) 

[Sayak Paul]

Even better would be to distill the current model into a smaller one. One of my university juniors tried this approach for the ESRGAN model and was able to belt out a 33 KB student model (Colab Notebook demo). 

This is on my radar for 2021. 

Sayak Paul | sayak.dev

[Kshitiz Rimal]

Wow! That’s super impressive!

[Xunkai Zhang]

Super cool! Thanks for the great work!

On Thu, Jan 7, 2021 at 12:37 AM Arun Venkatesan <avenk...@google.com> wrote:

Great work Sayak! Looks amazing.

To view this discussion on the web visit https://groups.google.com/d/msgid/ml-gde/CAGa_XGHcNTKYaWvFGAY%2BwYwg%3DYWdyjQTPxdMHUsU3iyHQW_FOw%40mail.gmail.com.