Error in graph mode when subclassing _BaseAttentionMechanism

162 views
Skip to first unread message

George Sterpu

unread,
Feb 13, 2020, 9:47:49 AM2/13/20
to SIG Addons
Hi,

I am trying to write my own attention mechanism by subclassing _BaseAttentionMechanism, but I am running into errors when in graph mode.

More precisely, the build method of my layer/mechanism needs to create a variable with a shape equal to the batch size.
Since the mechanism.setup_memory method is called in advance, there is already a member in the superclass named self.batch_size, but it seems that I cannot use it.

The error message looks like this:
    TypeError: An op outside of the function building code is being passed
    a
"Graph" tensor. It is possible to have Graph tensors
    leak
out of the function building context by including a
    tf
.init_scope in your function building code.
   
For example, the following function will fail:
     
@tf.function
     
def has_init_scope():
        my_constant
= tf.constant(1.)
       
with tf.init_scope():
          added
= my_constant * 2
   
The graph tensor has name: model/decoder/MyAttention/strided_slice:0

In the build method I also have access to self.keys, but trying to calculate the batch size with tf.shape(self.keys)[0] results in a similar error, the graph tensor name being ../memory_layer/Tensordor:0 this time.

Could you please help me find a solution ?
I find it (very) challenging to debug tf.function-related errors, particularly the one above.

Thank you

Sean Morgan

unread,
Feb 13, 2020, 7:53:27 PM2/13/20
to George Sterpu, SIG Addons
Hi George,

Would you mind making a GitHub issue? It's easier to assign the question to relevant maintainers.

Thanks,
Sean

--
You received this message because you are subscribed to the Google Groups "SIG Addons" group.
To unsubscribe from this group and stop receiving emails from it, send an email to addons+un...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/addons/bdcee4b4-7b35-4c51-8e96-d345468ff637%40tensorflow.org.

George Sterpu

unread,
Feb 14, 2020, 8:00:24 AM2/14/20
to Sean Morgan, SIG Addons
Hello Sean, thanks for the reply.
I have just created this GitHub issue: https://github.com/tensorflow/addons/issues/1093
Sorry in case I posted in the wrong place.

Thank you

On Fri, 14 Feb 2020 at 00:45, Sean Morgan <seanm...@outlook.com> wrote:
Hi George,

Would you mind making a GitHub issue? It's easier to assign the question to relevant maintainers.

Thanks,
Sean

On Thu, Feb 13, 2020 at 9:47 AM George Sterpu <george...@gmail.com> wrote:
Reply all
Reply to author
Forward
0 new messages