Problem with executing InceptionV3 model

[Chris Bellew]

Hi everyone,

Could someone please help me with this problem?

I followed the InceptionV3 retraining tutorial here https://www.tensorflow.org/tutorials/image_recognition. I used my own images and classes and was able to get 82% validation accuracy. Great!

I'd love to be able to run this in JavaScript land so I used to the converter to create a web friendly model, like so:

tensorflowjs_converter --input_format=tf_saved_model --output_node_names='InceptionV3/Logits/GlobalPool' --saved_model_tags=train /mymodelfolder /myoutputfolder

(I guessed at the "output_node_names")

Then I made a JS app using the code here (https://github.com/tensorflow/tfjs-converter) and loaded and executed my model. It looked like it loaded my model fine because Chrome shows all the files loading in the network tab.

On this line:

model.execute({input: tfc.fromPixels(cat)});

I got an error:

Uncaught (in promise) TypeError: Cannot read property 'dtype' of undefined

    at Object.assertTypesMatch (util.js:46)

    at BinaryOps.mul (binary_ops.js:95)

    at operation.js:11

    at Object.Tracking.tidy (tracking.js:36)

    at Object.descriptor.value [as mul] (operation.js:11)

    at Object.exports.executeOp (arithmetic_executor.js:11)

    at Object.executeOp (operation_executor.js:17)

    at graph_executor.js:69

    at Array.reduce (<anonymous>)

    at graph_executor.js:68

The variable 'cat' for me is an HTMLImageElement.

Does anyone have an idea what I may have done wrong?

Thanks in advance!

Chris

[Daniel Smilkov]

Hi Chris,

Adding Ping in cc, who is the author of the SavedModel converter for his expertise. Can you share your code with us to help us reproduce it?

Thanks!

Daniel

--
You received this message because you are subscribed to the Google Groups "TensorFlow.js Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+uns...@tensorflow.org.
Visit this group at https://groups.google.com/a/tensorflow.org/group/tfjs/.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/59925de2-9d96-41f4-93e0-587579793ea4%40tensorflow.org.

[Chris Bellew]

Forgot to reply all...

Hi Daniel, Ping,

Thanks, here is a gist with my code, my converted model, my test image, and my original (before conversion) model.
https://gist.github.com/ChrisBellew/d0c15f007c196838f1f0459405f49c70

[ricky singh]

hi i have a object detection model, using tensorflow object detection inception v2.
cant able to figure out output node name which is required for tensorflow js... any help...

[Chris Bellew]

Anyone got any thoughts one original problem in this thread?

Thanks :)

On Sat., 7 Apr. 2018, 3:54 pm ricky singh, <rick...@gmail.com> wrote:

hi i have a object detection model, using tensorflow object detection inception v2.
cant able to figure out output node name which is required for tensorflow js... any help...

--
You received this message because you are subscribed to the Google Groups "TensorFlow.js Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+uns...@tensorflow.org.
Visit this group at https://groups.google.com/a/tensorflow.org/group/tfjs/.

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/80fdd55f-61c8-461c-9286-5352d33e8970%40tensorflow.org.

[Ping Yu]

Hi Chris

I just tried your model, the difference between your model and mobilenet demo is the 

input name, your model input node name is: "hub_input/images"

If replace the line with

const result = model.execute({'hub_input/images': tfc.fromPixels(image)});

It should work, let me know if you see other issues.

Thanks

Ping

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/CABi1%3DH4Rd7knopQTBaosKbwJNKdXEzmGaX0cAhfp3%3D6G5dHkNg%40mail.gmail.com.

[Nikhil Thorat]

Hi Chris,

I think the problem comes from this line:

const result = model.execute({input: tfc.fromPixels(image)});

You should replace "input" with the node name in TensorFlow that you're providing (similar to feed entry).

Ping, I filed two bugs against you to help out with this:

https://github.com/tensorflow/tfjs/issues/146

https://github.com/tensorflow/tfjs/issues/147

On Sat, Apr 7, 2018 at 10:20 AM, Chris Bellew <cjbe...@gmail.com> wrote:

Anyone got any thoughts one original problem in this thread?

Thanks :)

On Sat., 7 Apr. 2018, 3:54 pm ricky singh, <rick...@gmail.com> wrote:

hi i have a object detection model, using tensorflow object detection inception v2.
cant able to figure out output node name which is required for tensorflow js... any help...

--
You received this message because you are subscribed to the Google Groups "TensorFlow.js Discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+unsubscribe@tensorflow.org.

Visit this group at https://groups.google.com/a/tensorflow.org/group/tfjs/.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/80fdd55f-61c8-461c-9286-5352d33e8970%40tensorflow.org.

--

You received this message because you are subscribed to the Google Groups "TensorFlow.js Discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+unsubscribe@tensorflow.org.

Visit this group at https://groups.google.com/a/tensorflow.org/group/tfjs/.

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/CABi1%3DH4Rd7knopQTBaosKbwJNKdXEzmGaX0cAhfp3%3D6G5dHkNg%40mail.gmail.com.

[Nikhil Thorat]

Ha, just saw Ping responded with the same answer :)

On Sat, Apr 7, 2018 at 12:33 PM, Nikhil Thorat <nsth...@google.com> wrote:

Hi Chris,

I think the problem comes from this line:

const result = model.execute({input: tfc.fromPixels(image)});

You should replace "input" with the node name in TensorFlow that you're providing (similar to feed entry).

Ping, I filed two bugs against you to help out with this:

https://github.com/tensorflow/tfjs/issues/146

https://github.com/tensorflow/tfjs/issues/147

On Sat, Apr 7, 2018 at 10:20 AM, Chris Bellew <cjbe...@gmail.com> wrote:

Anyone got any thoughts one original problem in this thread?

Thanks :)

On Sat., 7 Apr. 2018, 3:54 pm ricky singh, <rick...@gmail.com> wrote:

hi i have a object detection model, using tensorflow object detection inception v2.
cant able to figure out output node name which is required for tensorflow js... any help...

--
You received this message because you are subscribed to the Google Groups "TensorFlow.js Discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+uns...@tensorflow.org.

Visit this group at https://groups.google.com/a/tensorflow.org/group/tfjs/.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/80fdd55f-61c8-461c-9286-5352d33e8970%40tensorflow.org.

--
You received this message because you are subscribed to the Google Groups "TensorFlow.js Discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+uns...@tensorflow.org.

Visit this group at https://groups.google.com/a/tensorflow.org/group/tfjs/.

[Ping Yu]

 Nikhil, thanks for tracking those issues.

[Chris Bellew]

Many thanks Ping and Nikhil! And thanks for raising the bug(s) for me.

That enabled me to get further. It makes sense now that I think about it.

Next I got the error "The dtypes of the first (int32) and second (float32) input must match". I figured that the input node in the graph is expecting float32 and the input I was giving it was int32. So I changed

const result = model.execute({'hub_input/images': tfc.fromPixels(image)});

to...

const intImageTensor = tfc.fromPixels(image);

const floatImageTensor = tfc.cast(intImageTensor, 'float32');

        

const result = model.execute({'hub_input/images': floatImageTensor});

Which got past that error. So its interesting that the converted InceptionV3 mode has an input node that takes float32, whereas tfc.fromPixels gives an int32. I guess that's just the way it is.

The next error I got was "All values in axis param must be in range [-3, 3) but got axis 3".

It appears to be trying to perform a 'slice_join' or 'concat' operation on some nodes in the middle of the graph, and it appears to be some kind of rank mismatch. I wonder if this is a bug?

I have updated my gist so you should be able to run it and get the same result

https://gist.github.com/ChrisBellew/d0c15f007c196838f1f0459405f49c70

Thanks for any more time you spare :)

[Nikhil Thorat]

Hi Chris,

Regarding int32 / float32, you are exactly right. fromPixels always returns int32 (values between 0 - 255), this is part of native browser APIs. You can call "toFloat()" or "cast" like you're doing to convert it.

Tangentially related: you will likely have to normalize your input before passing it in if you want correctness (inception has a particular mean for each channel you need to normalize to).

Regarding the axis problem, it's a little surprising that we're concating along axis=3 (which is likely the batch axis). Ping, could you look into this?

[Ping Yu]

Hi Chris

As Nikhil mentioned you might need to normalize the input to value betwen (-1, 1). 

Also, the 'hub_input/images' node has required shape as (-1, 299, 299, 3)

please scale your image to match that, I believe your example image is

much larger.

Ping

[Chris Bellew]

Thank you both for your great direction.

I will work with your suggestions.

Thanks

Chris

--
You received this message because you are subscribed to the Google Groups "TensorFlow.js Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+uns...@tensorflow.org.
Visit this group at https://groups.google.com/a/tensorflow.org/group/tfjs/.

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/CALwXrv%2Br5OAEoLf_-4SFK4%3Dn5txf9zfeORqwiYm-TvokL7v35g%40mail.gmail.com.

[Amit Sharma]

Did the suggestions work? Were you able to import and run inference in tfjs?