Python + Image Classifier: why must it re-train the model every time?

[Dev Lab]

Hi,

Just starting to dig in to MLK for my students now. Great project!

My question is about using Python to classify images. We've already labeled and trained a collection of images in the cloud using the MLK web interface. Then we set up a Python venv with all the necessary modules. The Python script itself runs well.

However, each time running the project, we have to call project.train_model(), which takes a fair bit of time. This step is not necessary with the text classifier. Is there a workaround for this?

[Dale Lane]

With text projects, the machine learning model is trained in, and hosted in, the cloud. Specifically, an IBM Cloud service called "Watson Assistant".

With image projects, the machine learning model used to be hosted in the cloud, using an IBM Cloud service called "Watson Visual Recognition". However, that was deprecated so I replaced that with an implementation where you train the model locally on your own computer. 

That's the reason for why it is the way it is now.

But to your actual question about a workaround - yes, we could save the model.

At the moment, it's just kept in memory, which is why you need to retrain when you restart the script. But we could save the model, and then at start-up load the trained model from disk.

I think that's a good idea, so I've added it to my todo list to update the sample code with functions for saving and loading the model https://github.com/IBM/taxinomitis/issues/407 

I'm not sure when I'll get a chance to do that, so if you'd rather not wait, you can find documentation for how to do it yourself at https://www.tensorflow.org/guide/keras/save_and_serialize

Kind regards

D

[Dev Lab]

Thanks for the explanation, Dale, I'll take a whack at saving the model myself. Your work on this project is greatly appreciated.

[Dev Lab]

Hey Dale -

Well, that was easy enough to get the basics going.

FWIW on a very small set (50 images), downloading and compiling the model from the cloud took 15 seconds, while loading the local model took about 9 seconds. I imagine at larger sizes it could be faster, but it would depend on your computer, internet connection, load on the IBM servers etc.

John