What kind of optimization is applied during the model conversion with tensorflowjs_converter?

[Kenji Imamula]

Hi,

I looked into the codes of https://github.com/tensorflow/tfjs-converter and found that the conversion process of TensorFlow SavedModel includes some optimization of the model as follows (https://github.com/tensorflow/tfjs-converter/blob/50c0b13455fe103a3db66889f02993b5704c6e54/python/tensorflowjs/converters/tf_saved_model_conversion.py#L89-L106).

def optimize_graph(graph, output_graph):
  """Takes a Python Graph object and optimizes the graph.
  Args:
    graph: tf.Graph tensorflow dataflow graph
  """
  rewriter_config = rewriter_config_pb2.RewriterConfig()
  rewriter_config.optimizers[:] = [
      'pruning', 'constfold', 'arithmetic', 'dependency', 'pruning',
      'constfold', 'arithmetic', 'dependency'
  ]
  meta_graph = tf.train.export_meta_graph(
      graph_def=graph.as_graph_def(), graph=graph)
  optimized_graph = tf_optimizer.OptimizeGraph(
      rewriter_config, meta_graph, cluster=get_cluster())

  extract_weights(optimized_graph, output_graph)
  return optimize_graph

But I have no idea what exactly is happening here.

Is this something similar to what is mentioned in TensorFlow For Poet 2 codelab (https://codelabs.developers.google.com/codelabs/tensorflow-for-poets-2-tflite/#3)?

It would be nice if some official documentation is provided about this optimization.

Thanks!

Kenji

[Nikhil Thorat]

tf_optimizer is part of grappler. Essentially this is pruning nodes from the graph that aren't needed for inference to the node you provide (it gets rid of training params, etc).

I added Ping, who wrote that code and is the expert on this, to clarify if I missed something.

--
You received this message because you are subscribed to the Google Groups "TensorFlow.js Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+unsubscribe@tensorflow.org.
Visit this group at https://groups.google.com/a/tensorflow.org/group/tfjs/.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/7f1a48f6-44d7-42f9-adb4-f34351970b86%40tensorflow.org.

[Ping Yu]

Nikhil is right, the converter uses Grappler to prune the training nodes, also to provide many ways of graph optimizations, which is similar to what

toco provides but with more features in more generic fashion. (toco targets TFLite)

If you want to debug which optimization actually removed the Reshape node from your model, 

you can remove some of the optimizers in following line to disable them.

rewriter_config.optimizers[:] = [
      'pruning', 'constfold', 'arithmetic', 'dependency', 'pruning',
      'constfold', 'arithmetic', 'dependency'
  ]

Thanks

Ping

To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+uns...@tensorflow.org.

[Kenji Imamula]

Thanks for the explanation and giving a hint for my issue!

I will give it a try.

2018年4月5日木曜日 12時25分29秒 UTC+9 Ping Yu: