Inception_v3

[Gwenda Arguin]

Noteeach Keras Application expects a specific kind of input preprocessing.For InceptionV3, callkeras.applications.inception_v3.preprocess_input on your inputsbefore passing them to the model.inception_v3.preprocess_input will scale input pixels between -1 and 1.

Hi inference using the inception_v3 model from torchvision seems to be working in an unintended manner.

Judging by the error log, it seems that the error has to do with how batch norm is implemented

I have a problem execute quantized models on the NPU with ONNXRuntime. I downloaded the models mobilenet_v2_1.0_224.tflite, mobilenet_v2_1.0_224_quant.tflite, inception_v3.tflite and inception_v3_quant.tflite from the Machine Learning User's Guide and converted the models with the eIQ model converter.

For all the models I get correct results running it with the CPU_ACL EP. When I run the not quantized models with the Vsi_Npu EP, I get correct results too. But when I run the quantized models with the Vsi_Npu EP, I get wrong results.

I tried the following thing too: Convert the mobilenet_v2.tflite model to a mobilenet_v2.onnx model and quantize it then with float as input and output data type. Then I get wrong results even if I run the model on the CPU.

Maybe this is correct, but when I run not quantized models with float as input/output on the NPU (using VsiNpu or NNAPI from OnnxRuntime) I get correct results. The NPU don't support the layers but will fall back on the CPU to calculate the results. And the models I attached above are quantized with uint8_t input/output and should be supported by the NPU.Why do I get wrong results for them? I mean, when a layer in one of the models maybe is don't supported, it should fall back to the CPU and brings correct results too.

For ArmNN and TFLite I don't have problems like that. For both everything works fine with quantized models with uint8_t input and not quantized models with float input. So is there maybe a problem by OnnxRuntime? Or there is a problem in the convertion step from tflite to onnx format? Could you please have a look on the models? Maybe there is something wrong.

This blog post will go through the steps needed to perform transfer learningusing the Inception V3 architecture in python using Tensorflow. There areactually several types of transfer learning, as can be seen in the diagrambelow. This tutorial will cover the method A2. Thatis, we will use all the layers from the pre-trained model, except the final layer,creating a new output layer that is similar in structure to the original, butadapted to train on a new classification task. The layers fromthe pre-trained model will remain trainable, and continue to be fine-tuned forthe new task.

Inception V3 is a powerful deep neural network architecture developed byresearchers at Google and described in thispaper. A broad overview of the layers of thismodel can be viewed in the diagram below (using the exact same layer names asin the implementation we will use).

We will set up some important variables that will be used throughout the script.We want to specify the location of the pre-trained checkpoint file we downloaded,as well as where we want to save the snapshot for the fine-tuned model.

We will also specify the dimensions of the inputs and outputs. The weights forthe Inception V3 model that we downloaded was trained on RGB images of shape299 by 299, and 1001 output classes. If we keep the input dimensions exactly thesame, and modify the number of output classes, then we will be able to make useof all the pre-trained weights right before the output layer, and only theoutput layer will need to be initialized to random weights.

In python, the function that creates the Inception V3 architecture istf.contrib.slim.nets.inception.inception_v3(). But, in order for this functionto create the graph correctly, we have to first make use of something called anargument scope. An argument scope is a feature of Tensorflow that allows you tospecify your own default values for the arguments in Tensorflow functions. Ifyou wish to learn more about argument scopes, you can check out thisblog post.

tf.contrib.slim.nets.inception.inception_v3_arg_scope() creates an argumentscope that specifies a bunch of default values for the layer functions used bythe inception v3 model. We will create the argument scope as follows:

For the argument scope to take effect, we make use of tensroflow's argumentscope context manager tf.contrib.framework.arg_scope(). Any functions that arenested inside of this context manager will make use of the default valuesspecified by arg_scope. We can now call the tf.contrib.slim.nets.inception.inception_v3()function inside of this context manager:

tf.contrib.slim.nets.inception.inception_v3 takes as its first argument thetensor containing the batch of input images (scaled to values between 0-1). Italso takes the number of output classes, and a boolean (or boolean placeholder)specifying if the model is currently in training mode. By default, the dropoutkeep rate is set to 0.8, but you could set this to some other value, or evenhave a placeholder that feeds in the value.

tf.contrib.slim.nets.inception.inception_v3 returns two things. It returnsthe final output logits. It also returns end_points, which is a dictionarycontaining the output tensors for all the important layers from the model. Thetensors in end_points are particularly useful when you wish to modify andextend the inception v3 architecture in interesting ways. For instance, forimage segmentation, you need access to previous layers in order to createskip-connections.

As part of the graph, you will need to create two separate saver objects. Oneto load up theweights for the unmodified portion of the pre-trained inception v3 model. Thesecond saver object will be used to save and restore all the weights associatedwith the fine-tuned model.

In order to properly use the saver that will load from the pre-trained inceptionmodel, we need to specify which weights we need to extract from the checkpoint.Due to the way the name and variable scopes have been set up in the originalcode, it is quite easy to separate the weights that belong to the trunk of themodel, from the weights associated with the output layers.

All the weights associated with the Inception V3 model are inside the"InceptionV3" name scope. The weights associated with the final output layerare in the "InceptionV3/Logits" scope. There is an additional branch of weightsin the "InceptionV3/AuxLogits" scope that we do not need either.

We can get the full list of variables we want to extract by using thetf.contrib.framework.get_variables_to_restore() function. These variables arethen passed to the saver object that will be used to load up the weightsfrom the pre-trained inception V3 checkpoint.

When you create a session, you will need to initialize the variables. The veryfirst time you run the model, you will want to initialize the weights using thepre-trained snapshot (and random values for the final layer). However, aftersaving your model for the first time, you will want to load from your own savedsnapshot.

This, of course, is a very rudimentary example. It is the bare minimum needed toshow that training is occurring by only tracking the batch training losses. Youshould extend it further if you want to use it for training anythingseriously.

Tensorboard is a tool provided by tensorflow that allows you to visualize thearchitecture of your model visually. The code provided above automaticallycreated the necessary files to use tensorboard. The only thing you need to donow is run the following in a terminal window:

WARNING : Running and viewing Tensorboard can consume quite a lot of RAM.If your computer is already straining for memory space, then It is advisablethat you only run these steps after the model has finished training.

3a8082e126