registering hparam set in user dir

[David Liebman]

hi. I'm trying to use my own user dir. I need to register the hparam set so that I can use visualization later. How do I verify that the name is set properly? I get the following error.

KeyError: 'transformer_chat never registered with registry hparams. 

'transformer_chat' is the name of my hparam set. After that error is a long list of what I imagine must be built-in sets. How do I check mine is registered? I use the following code:

@registry.register_hparams('transformer_chat')
def transformer_chat():
    hparams = transformer.transformer_base_v2()
    hparams.num_hidden_layers = 6
    hparams.hidden_size = 512
    hparams.filter_size = 2048
    hparams.num_heads = 8
    hparams.attention_dropout = 0.6
    hparams.layer_prepostprocess_dropout = 0.1
    hparams.learning_rate = 0.05
   
    hparams.learning_rate_schedule = 'legacy'


    return hparams

    

I don't know where I saw the name declared like that. If it's wrong please point out the correct method.

Thanks much

[Martin Popel]

Hi,
see https://github.com/tensorflow/tensor2tensor/#adding-your-own-components
and don't forget the __init__.py file
https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/test_data/example_usr_dir/__init__.py

Martin

> --
> You received this message because you are subscribed to the Google Groups
> "tensor2tensor" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to tensor2tenso...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/tensor2tensor/add44d9e-d18f-48ed-99d3-b2643135e551%40googlegroups.com.

[David Liebman]

Thanks for your quick response. I'm still having the same problem.

This is the notebook I'm looking at.

https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/visualization/TransformerVisualization.ipynb

This is my user_dir and the file where I try to register my hparams.

https://github.com/radiodee1/awesome-chatbot/blob/master/transformer/chat/trainer/problem.py

Though very messy, this is the file where I train my transformer. I do believe that the hparams are set because tf tells me that the base learning rate matches the one I set in the `problem.py` file above. It tells me this in the printout to the terminal from tensor2tensor when I run the train option.

https://github.com/radiodee1/awesome-chatbot/blob/master/transformer/tf_t2t_train_chat.py

I looked at the link to the 'user_dir' example and also the 'adding-your-own-components' example. It has changed my code slightly but I do not get the notebook to run anyway.

Do you have any other examples? Thanks. This is my new listing (in part)

@registry.register_hparams #('transformer_chat')

def transformer_chat():
 hparams = transformer.transformer_base_v2()

 hparams.num_hidden_layers = 6 # 2
 hparams.hidden_size = 512 # 128
 hparams.filter_size = 2048 # 512
 hparams.num_heads = 8 #4
 hparams.attention_dropout = 0.6
 hparams.layer_prepostprocess_dropout = 0.1 #0.6
 hparams.learning_rate = 0.05
 #hparams.learning_rate_constant = 0.05

 hparams.learning_rate_schedule = 'legacy'

 return hparams

D Liebman

[David Liebman]

from tensor2tensor.utils import usr_dir 
usr_dir.import_usr_dir('my-usr-dir')

I tried this as you suggested and I am seeing some output. Thanks.

On Saturday, February 15, 2020 at 1:47:53 PM UTC-5, David Liebman wrote: