How to deal with Custom Layer

[Mike Chen]

I have converted a saved keras model which contains custom defined layer.
There's nothing wrong with the convert process.
However

model = await tf.loadModel('model.json')

showed the error.
error unknown layer: InputNormalize.

Does anyone know how to deal with such issue?

Thanks ahead.

[Shanqing Cai]

If you want to load a Keras model with custom layers, you need to rewrite and translate Python code that defines the custom layer into JavaScript or TypeScript.

This is mostly a manual process currently. But it shouldn't be too difficult, because TensorFlow.js has good parity with Keras. Let me know if you have questions or run into problems.

[Shanqing Cai]

Sorry I forgot to mention: Custom layers currently works only if you are composing models purely in JavaScript/TypeScript. They don't work yet, when you are loading Python-generated models with such layers, because we haven't exposed the ClassNameMap.register() method (see example here: https://github.com/tensorflow/tfjs-layers/blob/master/src/layers/convolutional_depthwise.ts#L161). We'll expose it in the next release.

[Mike Chen]

Thanks for your reply.
So what you mean is that it is not possible to implement a customer layer in python model by purely just rewrite in Javascript/Typescript yet.
Did I reach you point?

So, if it is not possible yet. Is there a way to write the model in tensorflowjs and just load the weights saved by keras?


2018年4月15日日曜日 11時04分49秒 UTC+9 Shanqing Cai:

[Michael Shi]

I was wondering how might we be able to access SerializationMap.register or any tfjs-core methods from the top level tfjs package? It seems like io/serialization are both not being exported even though it seems like it should judging from the source code.

[Nikhil Thorat]

We just haven't released yet. In the next release, 0.11, we'll support IO / serialization (from browser -- node will be next).

--
You received this message because you are subscribed to the Google Groups "TensorFlow.js Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+unsubscribe@tensorflow.org.
Visit this group at https://groups.google.com/a/tensorflow.org/group/tfjs/.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/ace2ae78-51a8-4b70-a3ec-b0e6098c757b%40tensorflow.org.

[Ada Dogrucu]

Hi,

I'd like to reiterate Mike's question for which I could not find an answer anywhere else.

Given that one converts a model that one has written in Keras into tensorflow-js, including rewriting the custom layers, is it possible to load the Keras model's saved weights into it's tensowflow-js counterpart??

Thank you

[Evgeny Demidov]

You could print weights as plain text in TF e.g. 

  w = dense1.get_weights()
  tf.print(tf.reshape(w[0],[1280*5]), summarize=-1)

and insert them in TFjs

  const dense1 = tf.layers.dense({
    inputShape: [1280],  units: 5,  activation: 'softmax',  useBias: true,
    weights: [tf.tensor(w0, [1280,5]), tf.tensor(w1)]
  })
  model = tf.sequential({ layers: [ dense1 ] });
see "Classifying Flowers with CNNs and Transfer Learning" at
https://www.ibiblio.org/e-notes/ml/flowers.htm.

But may be Nikhil Thorat will add weights IO / serialization examples?

And didn't you see SSD/YOLO Colab Transfer Learning demos (with flowers detection better :)

Evgeny

[Shanqing Cai]

Ada, is your goal to load the keras model with custom layer in JavaScript for inference only, or for both inference and further training?

--
You received this message because you are subscribed to the Google Groups "TensorFlow.js Discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+uns...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/e2a98dc9-2f84-4b75-9602-d506379d72bd%40tensorflow.org.

--

---

Shanqing Cai

Software Engineer

Google

ca...@google.com

[Shanqing Cai]

In either case, I have written a Gist to show how to write a custom layer with serializable weights in TensorFlow.js before. You can view it at:

https://gist.github.com/caisq/33ed021e0c7b9d0e728cb1dce399527d

If you use this approach, the loaded TF.js model will support both inference and further training in JavaScript.

If you just care about inference, you can save the keras model (presumably of the tf.keras flavor) as a TF SavedModel and convert it as a tf_graph_model with tensorflowjs_converter. Then the converted model can be loaded as a tf.GraphModel in TF.js for inference.

Let me know if you have any questions.

[Evgeny Demidov]

to Shanqing Cai
for tf.keras model from Colab
...
URL = 'https://tfhub.dev/google/tf2-preview/mobilenet_v2/feature_vector/4'
feature_extractor = hub.KerasLayer(URL,
     input_shape = (IMAGE_RES, IMAGE_RES, 3))

model = tf.keras.Sequential([
     feature_extractor,
     layers.Dense(num_classes, activation='softmax')
])

...
saved as model.h5 I get from tfjs_converter
error unknown layer: feature_extractor.

It is not very difficult to extract weights "by hand" but

How shall I use similar models in TFjs? or

How can I load binary weights from TF/Colab into TFjs?

Evgeny

On Monday, January 13, 2020 at 5:56:03 PM UTC+3, Shanqing Cai wrote:

[Shanqing Cai]

On Mon, Jan 13, 2020 at 10:30 AM Evgeny Demidov <demidov...@gmail.com> wrote:

to Shanqing Cai
for tf.keras model from Colab
...
URL = 'https://tfhub.dev/google/tf2-preview/mobilenet_v2/feature_vector/4'
feature_extractor = hub.KerasLayer(URL,
     input_shape = (IMAGE_RES, IMAGE_RES, 3))

model = tf.keras.Sequential([
     feature_extractor,
     layers.Dense(num_classes, activation='softmax')
])

...
saved as model.h5 I get from tfjs_converter
error unknown layer: feature_extractor.

The converter does two things (in that order):

1. It loads the model from the .h5 file.

2. It converts it to TensorFlow.js' JSON-based format and writes it to disk.

I'm guessing that this error happens at step #1, because the converter binary doesn't have the definition of the custom Python Keras layer class (i.e., hub.KerasLayer in this case).

To correct this, you can try using tensorflowjs_converter's Python API instead of the command line, that is the `tfjs.converters.save_keras_model` Python function.

The official documentation is at: https://www.tensorflow.org/js/tutorials/conversion/import_keras#alternative_use_the_python_api_to_export_directly_to_tfjs_layers_format

When this function runs in the same Python program as the one that defines the original tf.keras model with the custom layer class, it'll have access to the definition of that class.

 

It is not very difficult to extract weights "by hand" but

How shall I use similar models in TFjs? or

How can I load binary weights from TF/Colab into TFjs?

Evgeny

On Monday, January 13, 2020 at 5:56:03 PM UTC+3, Shanqing Cai wrote:

If you just care about inference, you can save the keras model (presumably of the tf.keras flavor) as a TF SavedModel and convert it as a tf_graph_model with tensorflowjs_converter. Then the converted model can be loaded as a tf.GraphModel in TF.js for inference.

Let me know if you have any questions.

--

You received this message because you are subscribed to the Google Groups "TensorFlow.js Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+uns...@tensorflow.org.

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/c4a82818-5bd9-4871-bbaf-42e119cee8ca%40tensorflow.org.