Attach arbritrary Java Object to an Operand/Op object?

[Jim Clarke]

Is there a way to attach an arbitrary Java Object to an Operand or Op object?

That way, the arbitrary Java Object could be queried later on in processing through the Operand or Op itself.

In looking at the Keras Model architecture, this appears to be needed. An object is attached to the Operand when the Model is created, and then later
queried during training, evaluating and prediction.

jim

[Ryan Nett]

It should be possible, but it's a bit fraught since `asOutput()` or `z` (or w/e) would give you a different Operand object.

Can you share a link to the Keras code?  I'm wondering if we can work around it.

[Jim Clarke]

Specifically, 

/Users/jbclarke/Code/tensorflow/tensorflow/python/keras/engine/node.py  class KerasHistory

which is stored in a “KerasTensor"

/Users/jbclarke/Code/tensorflow/tensorflow/python/keras/engine/keras_tensor.py class class KerasTensor

I am not sure we will copy this pattern, but for now I am just investigating.

In keras.Model, they store the model Tensors for input and output and use the “KerasHistory” class 

to get to the originating Layer from the “KerasTensor” when needed. The other approach I am thinking

about is just mapping the Layers to the Operands produced by the input and output layers.

SequentialModel is a subclass of Model, and I haven’t gotten that far.

Here is what sample TF Python code looks like for using a Model directly.

In this example, “inputs", “x", and “outputs" are tensors.

inputs = tf.keras.Input(shape=(3,))
x = tf.keras.layers.Dense(4, activation=tf.nn.relu)(inputs)
outputs = tf.keras.layers.Dense(5, activation=tf.nn.softmax)(x)
model = tf.keras.Model(inputs=inputs, outputs=outputs)

jim

[Karl Lessard]

But how will that work when you are loading your model from disk instead of building it from scratch? You’ll need to find a way to store the information of your object in the graph itself so you can recreate it and reattach it after loading, in which case your model will end up being very Java specific. Is the Python code also doing something like that?

On Mar 25, 2021, at 19:02, Jim Clarke <jimcla...@gmail.com> wrote:

Specifically, 

--
You received this message because you are subscribed to the Google Groups "SIG JVM" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jvm+uns...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/jvm/3C175DCE-35BE-4F13-9E58-08FA065AF0B5%40gmail.com.

[Jim Clarke]

Good point.

I am not sure how TF Python is handling it. They currently use an object, “KerasTensor”,  that holds the tensor and the “KerasHistory” object which holds the layer info.

Maybe all that gets serialized when the model is saved, I will need to investigate more.

jim

[Adam Pocock]

Graph serialization is causing me some difficulty in Tribuo, as when a graph is deserialized the graph's initialisers aren't populated so you have to create the init op before serialisation. In general the Graph that comes out from the protobuf is missing things compared to the richer Java object that we write out. Introducing more things which increase this divergence is only going to cause more trouble.

Adam

[Karl Lessard]

This proto seems an interesting starting point to look at: tensorflow-core/tensorflow-core-api/src/gen/java/org/tensorflow/proto/framework/SavedObjectOrBuilder.java

(warning: might be completely wrong too, I’ve just searched very quickly...)

On Mar 27, 2021, at 21:27, Adam Pocock <crai...@gmail.com> wrote:

Graph serialization is causing me some difficulty in Tribuo, as when a graph is deserialized the graph's initialisers aren't populated so you have to create the init op before serialisation. In general the Graph that comes out from the protobuf is missing things compared to the richer Java object that we write out. Introducing more things which increase this divergence is only going to cause more trouble.

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/jvm/48952917-d72b-4baf-b5f0-3d76816a0209n%40tensorflow.org.

[Ryan Nett]

I don't think SavedModel actually saves the Python object.  From their example notebook, all the loaded models are tensorflow.python.saved_model.load.Loader._recreate_base_user_object.<locals>._UserObject.  Though the tf.Module SavedModel I created isn't a valid SavedModel proto, so I couldn't inspect it manually.

I know for the h5 format the model objects are JSON serialized into the h5 metadata.

I'm hoping to do something about initialization saving/loading when I do init scopes, although I need to look at what Python does when saving.