I have a use case where I want to get the height of all the trainable weights inside a model.
The height of a variable is define as: The number of trainable transforms it has gone through since the input layer.
I have tried using the layer concept in TensorFlow. However, layer is only a logic concept and the granularity of layer is usually not enough to accommodate what I need. A logic layer can contain multiple trainable transforms, sometimes can even be an entire (sub-)model.
A little bit of background of my use case: I am trying to implement a generic discriminative training method, where the learning rate exponentially increases as the height increases.