Retrain.py

[Brett Kuprel]

Super awesome that there is code for training inception:

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/image_retraining/retrain.py

It seems that only the final layers are trained though, right?  This isn't exactly fine tuning.  I have been running a for loop over the graph operations and replacing the weight constants with variables.  https://github.com/kuprel/skin/blob/master/python/inference.py#L276

It would be great if the code for constructing the inception network was released, then we wouldn't have to reverse engineer the graph.

[Pete Warden]

We are working on getting full training (and retraining) out, but we didn't want to block releasing this transfer learning example on that. We haven't updated the website with the documentation yet, but you can see more information on the scope of this sample code here:

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/how_tos/image_retraining/index.md

We are also working on a fix for the protobuf large files issue that affects classify_image.py and this code.

--
You received this message because you are subscribed to the Google Groups "Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@tensorflow.org.
To post to this group, send email to dis...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/387583a7-d160-40e4-85a9-c0cd9a4d4104%40tensorflow.org.

[carne...@gmail.com]

Hi,

this is awesome! Code for retraining would be great, but meanwhile I'm trying to adjust the published code. I'm not interested in caching the bottlenecks, so I'm trying to get rid of the extra call to sess.run to just get the bottlenecks.

My code looks a lot like the CIFAR-10 example one: https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/models/image/cifar10/cifar10.py

So I have a method inference() and I want the first nodes to be inception up to the bottleneck.

def inference(images, graph):

             # images is a 4D tensor of [batch_size, 299, 299, 3] size.

  bottleneck_tensor = graph.get_tensor_by_name(ensure_name_has_port(BOTTLENECK_TENSOR_NAME))

  logits = tf.matmul(bottleneck_tensor, my_new_layer_weights, name='final_matmul') + layer_biases

  ...

This code is obviously missing the part where the images are actually fed as input. Isn't there any way I can feed the images as input here directly without having to make an extra call to sess.run() to get the bottlenecks?

Thanks!

On Sunday, February 14, 2016 at 1:14:25 AM UTC+1, petewarden wrote:

We are working on getting full training (and retraining) out, but we didn't want to block releasing this transfer learning example on that. We haven't updated the website with the documentation yet, but you can see more information on the scope of this sample code here:

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/how_tos/image_retraining/index.md

We are also working on a fix for the protobuf large files issue that affects classify_image.py and this code.

On Saturday, February 13, 2016, Brett Kuprel <brku...@gmail.com> wrote:

Super awesome that there is code for training inception:

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/image_retraining/retrain.py

It seems that only the final layers are trained though, right?  This isn't exactly fine tuning.  I have been running a for loop over the graph operations and replacing the weight constants with variables.  https://github.com/kuprel/skin/blob/master/python/inference.py#L276

It would be great if the code for constructing the inception network was released, then we wouldn't have to reverse engineer the graph.

--
You received this message because you are subscribed to the Google Groups "Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss+unsubscribe@tensorflow.org.

[Pete Warden]

Thanks for the kind words!

You should definitely be able to run the graph in a single run call, you'll just need to supply the input image data and request the correct tensor output. From your description, it should look something like this:

  logit_values = sess.run(logits, {ensure_name_has_port('ResizeBilinear'): images})

'ResizeBilinear' just happens to be the name of the op that comes after the jpeg decoding, and rescales the image size to 299x299. By overriding that op with your prescaled images you should be able to fed in your own data and have the rest of the graph work as expected.

One thing to note is that the Inception v3 graph has a bug which prevents running batches larger than 1 through it. You just need to change one shape constant, I believe it's Node: pool_3/_reshape's first argument, to have a shape of -1 for the batch dimension. I don't have a code change for that yet though.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@tensorflow.org.

To post to this group, send email to dis...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/387583a7-d160-40e4-85a9-c0cd9a4d4104%40tensorflow.org.

--

You received this message because you are subscribed to the Google Groups "Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@tensorflow.org.

To post to this group, send email to dis...@tensorflow.org.

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/b14306c1-28fb-4311-84ba-5521b853af58%40tensorflow.org.

[Bernardo Pires]

Hi Pete,

thanks for the super fast reply! I tried what you said but it fails with 'The value of a feed cannot be a tf.Tensor object. Acceptable feed values include Python scalars, strings, lists, or numpy ndarrays.'

If I do 

images_val = sess.run(images)

and then feed images_val it works, but I kind goes against the purpose of only calling sess.run once. Any other ideas?

And thanks for the heads up about the batches larger than 1! I was actually surprised my code ran even without feeding images at all. Obviously I got the same result for every image I tried since it was never getting fed, but I expected tensorflow to throw an error if a tensor is missing?

[Pete Warden]

Thanks for the clarification, I'd misunderstood the type of your input.

It should be possible to change the op that takes ResizeBilinear as its input to use your 'images' tensor as its input instead. I haven't got code handy for doing this unfortunately, though I've been meaning to clean up the distortion path in the example by doing something similar and only running the graph once.

[Bernardo Pires]

That sounds cool Pete, I hope you guys can get the full training version out soon! Would love to be able to finetune the whole network.

[mar...@gmail.com]

It says throughout the docs things like:

The script will write out a version of the Inception v3 network with a final layer retrained to your categories to /tmp/output_graph.pb, and a text file containing the labels to /tmp/output_labels.txt. These are both in a format that the C++ and Python image classification examples can read in, so you can start using your new model immediately.

However it's not clear to me how to use this graph def in something like classify_images.py.

I can get it to load the file, I think, but it's not outputting the right category when I run classify_images.py on one of the sample files.

I would suggest that providing an example of how to then use the .pb file that it generates in the documentation would be prudent, to help noobs like me.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss+unsubscribe@tensorflow.org.

To post to this group, send email to dis...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/387583a7-d160-40e4-85a9-c0cd9a4d4104%40tensorflow.org.

--
You received this message because you are subscribed to the Google Groups "Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@tensorflow.org.
To post to this group, send email to dis...@tensorflow.org.

[Pete Warden]

Sorry to hear you're running into problems. I will update the documentation with examples of how to use the Python and C++ examples with the models you've created, that's a good idea.

To help me better understand, when you say "I can get it to load the file, I think, but it's not outputting the right category when I run classify_images.py on one of the sample files.", do you mean it's mis-classifying the sample image? Or that it's outputting a category that's not in your training set at all?

To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@tensorflow.org.
To post to this group, send email to dis...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/387583a7-d160-40e4-85a9-c0cd9a4d4104%40tensorflow.org.

--
You received this message because you are subscribed to the Google Groups "Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@tensorflow.org.
To post to this group, send email to dis...@tensorflow.org.

[Ben Slaney]

It still calls a sunflower a daisy, the same as it does when it’s run when it does not include the flower models that get created.

It’s not clear (to me) how to load a second .pb file. I can see this code here in classify_image.py:

  with tf.gfile.FastGFile(os.path.join(

      FLAGS.model_dir, 'classify_image_graph_def.pb'), 'rb') as f:

So I’ve tried reproducing that method to load the other .pb file that gets created on the flower categories after it has loaded the regular “classify_image_graph_def.pb” but it doesn’t seem to affect the outcome of classifying a sunflower.

Apart from that TensorFlow is amazing btw, it makes me really excited about the future.

[Pete Warden]

Thanks for the update Ben! The classify_image.py script is a bit harder to use because it doesn't have flags for the input graph. You can try the C++ example more easily, with something like this:

bazel build tensorflow/examples/label_image:label_image

bazel-bin/tensorflow/examples/label_image/label_image --image=$USER/flower_photos/daisy/21652746_cc379e0eea_m.jpg --graph=/tmp/output_graph.pb --labels=/tmp/output_labels.txt --logtostderr --output_layer=final_result

With this example the graph and label names are being passed on the command line, so you shouldn't need to alter any code.

[Pete Warden]

I've submitted an update to the documentation. It may take a little while to make it to the website, but here's the final tested command-line I used:

bazel build tensorflow/examples/label_image:label_image && \

bazel-bin/tensorflow/examples/label_image/label_image \

--graph=/tmp/output_graph.pb --labels=/tmp/output_labels.txt \

--output_layer=final_result \

--image=$HOME/flower_photos/daisy/21652746_cc379e0eea_m.jpg

I hope that helps, let me know if not.

[martin...@gmail.com]

Hello Pete,

First off, I'm very excited about using tensorflow. Hope it's not an issue that I am reviving this slightly aged thread!

I am running into some problems trying to use the `classify_image.py` with the retrained network. It works fine with the flags and the C++ example, but I would like to integrate this into a python project, so I would really like to get it to work.

The problem happens in `create_graph()`:

def create_graph():                                                                                                                            

        """Creates a graph from saved GraphDef file and returns a saver."""                                                          

        retrained_name = 'output_graph.pb'                                                                                           

        old_name = 'classify_image_graph_def.pb'                                                                                     

                                                                                                                                     

        # Creates graph from saved graph_def.pb.                                                                                     

        with tf.gfile.FastGFile(os.path.join(                                                                                        

                model_dir, retrained_name), 'rb') as f:                                                                              

            graph_def = tf.GraphDef()                                                                                                

            graph_def.ParseFromString(f.read())                                                                                      

            _ = tf.import_graph_def(graph_def, name='')

If I use the "old" network, it loads and classifies fine. The code above with the new "output_graph.pb" fails however, giving the following message:

E tensorflow/core/common_runtime/executor.cc:275] Executor failed to create kernel. Invalid argument: NodeDef mentions attr 'data_format' not in Op<name=Conv2D; signature=input:T, filter:T -> output:T; attr=T:type,allowed=[DT_FLOAT, DT_DOUBLE]; attr=strides:list(int); attr=use_cudnn_on_gpu:bool,default=true; attr=padding:string,allowed=["SAME", "VALID"]>; NodeDef: conv/Conv2D = Conv2D[T=DT_FLOAT, data_format="NHWC", padding="VALID", strides=[1, 2, 2, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/gpu:0"](Mul, conv/conv2d_params)

         [[Node: conv/Conv2D = Conv2D[T=DT_FLOAT, data_format="NHWC", padding="VALID", strides=[1, 2, 2, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/gpu:0"](Mul, conv/conv2d_params)]]

Traceback (most recent call last):

  File "classify_image.py", line 73, in <module>

    tf.app.run()

  File "/home/mingram/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/platform/default/_app.py", line 30, in run

    sys.exit(main(sys.argv))

  File "classify_image.py", line 69, in main

    run_inference_on_image(image)

  File "classify_image.py", line 55, in run_inference_on_image

    image_data})

  File "/home/mingram/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 315, in run

    return self._run(None, fetches, feed_dict)

  File "/home/mingram/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 511, in _run

    feed_dict_string)

  File "/home/mingram/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 564, in _do_run

    target_list)

  File "/home/mingram/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 586, in _do_call

    e.code)

tensorflow.python.framework.errors.InvalidArgumentError: NodeDef mentions attr 'data_format' not in Op<name=Conv2D; signature=input:T, filter:T -> output:T; attr=T:type,allowed=[DT_FLOAT, DT_DOUBLE]; attr=strides:list(int); attr=use_cudnn_on_gpu:bool,default=true; attr=padding:string,allowed=["SAME", "VALID"]>; NodeDef: conv/Conv2D = Conv2D[T=DT_FLOAT, data_format="NHWC", padding="VALID", strides=[1, 2, 2, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/gpu:0"](Mul, conv/conv2d_params)

         [[Node: conv/Conv2D = Conv2D[T=DT_FLOAT, data_format="NHWC", padding="VALID", strides=[1, 2, 2, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/gpu:0"](Mul, conv/conv2d_params)]]

Caused by op u'conv/Conv2D', defined at:

  File "classify_image.py", line 73, in <module>

    tf.app.run()

  File "/home/mingram/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/platform/default/_app.py", line 30, in run

    sys.exit(main(sys.argv))

  File "classify_image.py", line 69, in main

    run_inference_on_image(image)

  File "classify_image.py", line 40, in run_inference_on_image

    create_graph()

  File "classify_image.py", line 23, in create_graph

    _ = tf.import_graph_def(graph_def, name='')

  File "/home/mingram/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/importer.py", line 238, in import_graph_def

    compute_shapes=False, compute_device=False)

  File "/home/mingram/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2040, in create_op

    original_op=self._default_original_op, op_def=op_def)

  File "/home/mingram/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1087, in __init__

    self._traceback = _extract_stack()

Do you have any idea what might be going wrong?

Thank you and all the best,

Martin

To unsubscribe from this group and stop receiving emails from it, send an email to discuss+unsubscribe@tensorflow.org.

To post to this group, send email to dis...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/387583a7-d160-40e4-85a9-c0cd9a4d4104%40tensorflow.org.

--
You received this message because you are subscribed to the Google Groups "Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@tensorflow.org.
To post to this group, send email to dis...@tensorflow.org.

[Pete Warden]

Sorry you're having problems Martin! I believe that's the same issue as https://github.com/tensorflow/tensorflow/issues/1528.

The error happens when classify_image uses an older version of TensorFlow than the retrain script did when it was run. Unfortunately this is an easy state to get into.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@tensorflow.org.
To post to this group, send email to dis...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/387583a7-d160-40e4-85a9-c0cd9a4d4104%40tensorflow.org.

--
You received this message because you are subscribed to the Google Groups "Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@tensorflow.org.
To post to this group, send email to dis...@tensorflow.org.

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/b14306c1-28fb-4311-84ba-5521b853af58%40tensorflow.org.

--
You received this message because you are subscribed to the Google Groups "Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@tensorflow.org.
To post to this group, send email to dis...@tensorflow.org.

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/2e69e695-8a45-4427-9f0f-3e3b23ca116c%40tensorflow.org.

[martin...@gmail.com]

Hi Pete,

Thank you, this was indeed the problem! I had installed my python tensorflow version through pip but had created the model with the master branch from github. I solved it by building the tensorflow python library and using that instead.

Thanks again and all the best,

Martin

To unsubscribe from this group and stop receiving emails from it, send an email to discuss+unsubscribe@tensorflow.org.

To post to this group, send email to dis...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/387583a7-d160-40e4-85a9-c0cd9a4d4104%40tensorflow.org.

--
You received this message because you are subscribed to the Google Groups "Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@tensorflow.org.
To post to this group, send email to dis...@tensorflow.org.

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/b14306c1-28fb-4311-84ba-5521b853af58%40tensorflow.org.

[martin...@gmail.com]

Hi again Pete,

First off, thanks again for producing this example, it's been really useful!

I just wanted to ask: are you still working on a full retraining example? If so, do you have some idea of when it might be released? I'd be curious to try retraining the whole network to see whether it improves performance.

Thanks a lot and all the best,

Martin

On Sunday, 14 February 2016 00:14:25 UTC, petewarden wrote:

We are working on getting full training (and retraining) out, but we didn't want to block releasing this transfer learning example on that. We haven't updated the website with the documentation yet, but you can see more information on the scope of this sample code here:

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/how_tos/image_retraining/index.md

We are also working on a fix for the protobuf large files issue that affects classify_image.py and this code.

On Saturday, February 13, 2016, Brett Kuprel <brku...@gmail.com> wrote:

Super awesome that there is code for training inception:

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/image_retraining/retrain.py

It seems that only the final layers are trained though, right?  This isn't exactly fine tuning.  I have been running a for loop over the graph operations and replacing the weight constants with variables.  https://github.com/kuprel/skin/blob/master/python/inference.py#L276

It would be great if the code for constructing the inception network was released, then we wouldn't have to reverse engineer the graph.

--
You received this message because you are subscribed to the Google Groups "Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss+unsubscribe@tensorflow.org.

[Pete Warden]

My colleagues recently released an example of how to do both full retraining, and fine-tuning across all layers. I would recommend starting with the latter, since it often gives the best results:

https://github.com/tensorflow/models/blob/master/inception/README.md#how-to-fine-tune-a-pre-trained-model-on-a-new-task

Does that help?

On Thu, Mar 31, 2016 at 4:10 AM, <martin...@gmail.com> wrote:

Hi again Pete,

First off, thanks again for producing this example, it's been really useful!

I just wanted to ask: are you still working on a full retraining example? If so, do you have some idea of when it might be released? I'd be curious to try retraining the whole network to see whether it improves performance.

Thanks a lot and all the best,

Martin

On Sunday, 14 February 2016 00:14:25 UTC, petewarden wrote:

We are working on getting full training (and retraining) out, but we didn't want to block releasing this transfer learning example on that. We haven't updated the website with the documentation yet, but you can see more information on the scope of this sample code here:

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/how_tos/image_retraining/index.md

We are also working on a fix for the protobuf large files issue that affects classify_image.py and this code.

On Saturday, February 13, 2016, Brett Kuprel <brku...@gmail.com> wrote:

Super awesome that there is code for training inception:

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/image_retraining/retrain.py

It seems that only the final layers are trained though, right?  This isn't exactly fine tuning.  I have been running a for loop over the graph operations and replacing the weight constants with variables.  https://github.com/kuprel/skin/blob/master/python/inference.py#L276

It would be great if the code for constructing the inception network was released, then we wouldn't have to reverse engineer the graph.

--
You received this message because you are subscribed to the Google Groups "Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@tensorflow.org.

To post to this group, send email to dis...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/387583a7-d160-40e4-85a9-c0cd9a4d4104%40tensorflow.org.

--

You received this message because you are subscribed to the Google Groups "Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@tensorflow.org.

To post to this group, send email to dis...@tensorflow.org.

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/167421f4-0f36-4be7-8b34-076f0ec1acc9%40tensorflow.org.

[martin...@gmail.com]

Hi Pete,

That's brilliant, thanks! I look forward to giving it a try soon.

All the best,

Martin

On Thursday, 31 March 2016 15:14:27 UTC+1, petewarden wrote:

My colleagues recently released an example of how to do both full retraining, and fine-tuning across all layers. I would recommend starting with the latter, since it often gives the best results:

https://github.com/tensorflow/models/blob/master/inception/README.md#how-to-fine-tune-a-pre-trained-model-on-a-new-task

Does that help?

On Thu, Mar 31, 2016 at 4:10 AM, <martin...@gmail.com> wrote:

Hi again Pete,

First off, thanks again for producing this example, it's been really useful!

I just wanted to ask: are you still working on a full retraining example? If so, do you have some idea of when it might be released? I'd be curious to try retraining the whole network to see whether it improves performance.

Thanks a lot and all the best,

Martin

On Sunday, 14 February 2016 00:14:25 UTC, petewarden wrote:

We are working on getting full training (and retraining) out, but we didn't want to block releasing this transfer learning example on that. We haven't updated the website with the documentation yet, but you can see more information on the scope of this sample code here:

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/how_tos/image_retraining/index.md

We are also working on a fix for the protobuf large files issue that affects classify_image.py and this code.

On Saturday, February 13, 2016, Brett Kuprel <brku...@gmail.com> wrote:

Super awesome that there is code for training inception:

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/image_retraining/retrain.py

It seems that only the final layers are trained though, right?  This isn't exactly fine tuning.  I have been running a for loop over the graph operations and replacing the weight constants with variables.  https://github.com/kuprel/skin/blob/master/python/inference.py#L276

It would be great if the code for constructing the inception network was released, then we wouldn't have to reverse engineer the graph.

--
You received this message because you are subscribed to the Google Groups "Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss+unsubscribe@tensorflow.org.

To post to this group, send email to dis...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/387583a7-d160-40e4-85a9-c0cd9a4d4104%40tensorflow.org.

[Vlad Firoiu]

Has the batch size restricted to 1 been fixed, or is there a workaround?

[Vlad Firoiu]

So I found out that there's a new inception-v3 download at http://download.tensorflow.org/models/image/imagenet/inception-v3-2016-03-01.tar.gz (previously I was using inception-2015-12-05.tgz - are these listed somewhere?). However, this new tar file doesn't contain a classify_image_graph_def.pb, but instead has a checkpoint file. If I understand correctly, checkpoint files must be loaded with a Saver, but require the current session's graph to already be defined (since only the Variables are saved in the checkpoint, not the graph itself). I guess that means I need to get my hands on the original code that created the inception model, which I'm not sure where to find - this is less convenient than just loading the graph, but I want to be able to tune the existing inception variables for a new task.

[Bernardo Pires]

Not sure if you have seen this, but Pete Warden posted a linked teaching how to fully retrain inception: https://github.com/tensorflow/models/blob/master/inception/README.md#how-to-fine-tune-a-pre-trained-model-on-a-new-task

The architecture is described there fully, so you can use a saver to load your model. 

--
You received this message because you are subscribed to a topic in the Google Groups "Discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/a/tensorflow.org/d/topic/discuss/ee8UBKRjdQM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to discuss+u...@tensorflow.org.

To post to this group, send email to dis...@tensorflow.org.

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/9a62beac-bb8c-488d-a145-94a161ec9117%40tensorflow.org.

[Vlad Firoiu]

I have seen that, but in my case I don't actually want to classify images at all - instead I am taking existing layers in inception and adding my own additional layers on top, which the tutorial doesn't cover.

It looks like https://github.com/tensorflow/models/blob/master/inception/inception/slim/inception_model.py might be what I want in order to define the model myself?

[st553]

Im getting a lot of warnings since upgrading tensorflow to 0.8.0:

W tensorflow/core/kernels/batch_norm_op.cc:36] Op is deprecated. It will cease to work in GraphDef version 9. Use tf.nn.batch_normalization().

Is there a new version of classify_image_graph_def.pb that will fix this?

[Pete Warden]

The warnings are because BatchNormWithGlobalization is deprecated, but used in the example Inception v3 model. You don't see those warnings when you run it normally because it's saved with an older GraphDef version number (because it was created back in December). The retraining process writes out a new GraphDef with a current version number, and so when you run it you suddenly start to see warnings.

I'm aware this is annoying, so I'm discussing the right way to tackle it with the team. Ideally our released Inception will eventually use the non-deprecated approach to batch normalization.

--

You received this message because you are subscribed to the Google Groups "Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@tensorflow.org.

To post to this group, send email to dis...@tensorflow.org.

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/80526c4f-950f-4b60-8475-06ab50c33f61%40tensorflow.org.

[Bernardo Pires]

Hey Pete,

I've been  playing with full retraining and I was wondering how to reuse the variables in inception? I basically want to do inference with a different set of images (train and test images during training). So I basically import inference from inception/inception_model.py, call inference with train images, call tf.get_variable_scope().reuse_variables(), call inference with test images. Somehow the ops end up in different scopes: inception_v3/ and inception_v3_1/ respectively. Any idea why? It does seem like the variables are somehow shared as the code does not run without the reuse_variables call. Thanks!

--
You received this message because you are subscribed to a topic in the Google Groups "Discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/a/tensorflow.org/d/topic/discuss/ee8UBKRjdQM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to discuss+u...@tensorflow.org.

To post to this group, send email to dis...@tensorflow.org.

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/CAOLmV945YV3yYwSGiP6GSrksVeJfVdCvVnF6__7Nd4O8OvVL_g%40mail.gmail.com.

[Pete Warden]

I'm actually not sure about the variable scoping in this case, and I don't have much familiarity with the full retraining. I've cc-ed jon Shlens who might be able to help.

[Bernardo Pires]

Hey Pete and Jon. Sorry, this seems to be completely normal. I was mixing up my expectations for variables and ops I guess. The variables do all live inside the same scope, but it seems like the ops live in a different scope because they're indeed different results (my best guess). My code wasn't working properly but the mistake was somewhere else. All is good now. Thanks for the fast reply! :)

[sudha...@gmail.com]

Is it possible to list all the possible ways to feed input image in to the graph and possible useful outputs (example softmax:0 is the final output, pool_3:0 is the tensor before that that represents features etc ...)

for the inputs, useful info would be what operations are carried out on the image if a certain input method is used and what would be the best entry point for a completely pre-processed/resized image so that no further mpodifications are done within the graph.

As of now the documentation available for the pre-trained

http://download.tensorflow.org/models/image/imagenet/inception-v3-2016-03-01.tar.gz

is not very helpful

Regards,
Sudhanshu

On Thursday, March 31, 2016 at 7:44:27 PM UTC+5:30, petewarden wrote:

My colleagues recently released an example of how to do both full retraining, and fine-tuning across all layers. I would recommend starting with the latter, since it often gives the best results:

https://github.com/tensorflow/models/blob/master/inception/README.md#how-to-fine-tune-a-pre-trained-model-on-a-new-task

Does that help?

On Thu, Mar 31, 2016 at 4:10 AM, <martin...@gmail.com> wrote:

Hi again Pete,

First off, thanks again for producing this example, it's been really useful!

I just wanted to ask: are you still working on a full retraining example? If so, do you have some idea of when it might be released? I'd be curious to try retraining the whole network to see whether it improves performance.

Thanks a lot and all the best,

Martin

On Sunday, 14 February 2016 00:14:25 UTC, petewarden wrote:

We are working on getting full training (and retraining) out, but we didn't want to block releasing this transfer learning example on that. We haven't updated the website with the documentation yet, but you can see more information on the scope of this sample code here:

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/how_tos/image_retraining/index.md

We are also working on a fix for the protobuf large files issue that affects classify_image.py and this code.

On Saturday, February 13, 2016, Brett Kuprel <brku...@gmail.com> wrote:

Super awesome that there is code for training inception:

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/image_retraining/retrain.py

It seems that only the final layers are trained though, right?  This isn't exactly fine tuning.  I have been running a for loop over the graph operations and replacing the weight constants with variables.  https://github.com/kuprel/skin/blob/master/python/inference.py#L276

It would be great if the code for constructing the inception network was released, then we wouldn't have to reverse engineer the graph.

--
You received this message because you are subscribed to the Google Groups "Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss+unsubscribe@tensorflow.org.

To post to this group, send email to dis...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/387583a7-d160-40e4-85a9-c0cd9a4d4104%40tensorflow.org.

[crou...@gmail.com]

I wonder if there is a way of 'cutting' the network. What I want to achieve is fine-tuning of the whole network and then implement transfer learning by replacing the last 2 inception modules with a new conv layer and a softmax. I did the whole fine-tuning using this code https://github.com/tensorflow/models/tree/master/inception on my own dataset taking the last checkpoint that I need but now I don't know how I can implement the transfer learning by taking the output of 8th inception module and train the new layers. I know that in the retrain.py script there is the implementation I need but I can't do it because I need to export my whole fine-tuning in .pb file somehow. Any help? Thanks in advance.

[akshatba...@gmail.com]

When I follow the tutorials of "How to Retrain Inception's Final Layer for New Categories", I am running python retrain.py on Windows. I have not made any changes to the file retrain.py. I get the following error

C:\Users\student\Desktop\flowerrecog>python retrain.py

Traceback (most recent call last):

  File "retrain.py", line 1105, in <module>
    tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
  File "C:\Users\student\Anaconda3\lib\site-packages\tensorflow\python\platform\app.py", line 48, in run
    _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  File "retrain.py", line 810, in main
    maybe_download_and_extract()
  File "retrain.py", line 313, in maybe_download_and_extract
    _progress)
  File "C:\Users\student\Anaconda3\lib\urllib\request.py", line 188, in urlretrieve
    with contextlib.closing(urlopen(url, data)) as fp:
  File "C:\Users\student\Anaconda3\lib\urllib\request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Users\student\Anaconda3\lib\urllib\request.py", line 466, in open
    response = self._open(req, data)
  File "C:\Users\student\Anaconda3\lib\urllib\request.py", line 489, in _open
    'unknown_open', req)
  File "C:\Users\student\Anaconda3\lib\urllib\request.py", line 444, in _call_chain
    result = func(*args)

[Benjamin Ellenberger]

It seems you need to provide a url, which you either don´t provide or it can not be found. Check if you are given a url in the tutorial.

--

You received this message because you are subscribed to the Google Groups "Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@tensorflow.org.
To post to this group, send email to dis...@tensorflow.org.

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/discuss/ed3e28f5-b5db-4ec1-ac09-4eb2e50547af%40tensorflow.org.