The proper way to get the *height* of each trainable variable

9 views
Skip to first unread message

Hongbo Fan

unread,
Jul 13, 2022, 11:34:49 AMJul 13
to TensorFlow Developers
Hi All,

I have a use case where I want to get the height of all the trainable weights inside a model.

The height of a variable is define as: The number of trainable transforms it has gone through since the input layer.
I have tried using the layer concept in TensorFlow. However, layer is only a logic concept and the granularity of layer is usually not enough to accommodate what I need. A logic layer can contain multiple trainable transforms, sometimes can even be an entire (sub-)model.

A little bit of background of my use case: I am trying to implement a generic discriminative training method, where the learning rate exponentially increases as the height increases.
Reply all
Reply to author
Forward
0 new messages