The proper way to get the *height* of each trainable variable

[Hongbo Fan]

Hi All,

I have a use case where I want to get the height of all the trainable weights inside a model.

The height of a variable is define as: The number of trainable transforms it has gone through since the input layer.

I have tried using the layer concept in TensorFlow. However, layer is only a logic concept and the granularity of layer is usually not enough to accommodate what I need. A logic layer can contain multiple trainable transforms, sometimes can even be an entire (sub-)model.

A little bit of background of my use case: I am trying to implement a generic discriminative training method, where the learning rate exponentially increases as the height increases.