Constraint Decoding using python API

[eng.abdul...@gmail.com]

Greetings,

The following details are copy-pasted from the issue I have created here. Any help is appreciated. 

## Since fairseq-interactive is not working, how can I use the python API to enforce constrained decoding?

I am trying to follow the instruction here: [Lexically constrained decoding with dynamic beam allocation](https://github.com/pytorch/fairseq/tree/master/examples/constrained_decoding) but I am getting this error:

```

AttributeError: 'Namespace' object has no attribute 'distributed_training'

```

I understand that this error is reported, and currently needs triage. However, I tried to use constrained decoding using the python API. 

I am loading the model using something similar to the following:

```

from fairseq.models.transformer import TransformerModel

zh2en = TransformerModel.from_pretrained(

  '/path/to/checkpoints',

  checkpoint_file='checkpoint_best.pt',

  data_name_or_path='data-bin/wmt17_zh_en_full',

  bpe='subword_nmt',

  bpe_codes='data-bin/wmt17_zh_en_full/zh.code'

)

zh2en.translate('你好 世界')

# 'Hello World'

```

How can I enforce constrained decoding through this API? 

I have tried adding `constrained` parameter at the model loading method, and the model was loaded by either specifying the argument to `ordered` or `unordered`. but when invoking `translate` nothing changed until I try to add the `constraints` parameter to `translate`. If this parameter is an empty list, nothing goes wrong. If I add anything, it will generate some errors. Like when adding `constraints=['▁نور']`, it generates the following error:

```

~/nmt/fairseq/fairseq/hub_utils.py in translate(self, sentences, beam, verbose, **kwargs)

~/nmt/fairseq/fairseq/hub_utils.py in sample(self, sentences, beam, verbose, **kwargs)

~/nmt/fairseq/fairseq/hub_utils.py in sample(self, sentences, beam, verbose, **kwargs)

~/nmt/fairseq/fairseq/hub_utils.py in generate(self, tokenized_sentences, beam, verbose, skip_invalid_size_inputs, inference_step_args, **kwargs)

~/nmt/fairseq/fairseq/tasks/fairseq_task.py in inference_step(self, generator, models, sample, prefix_tokens, constraints)

/usr/local/lib/python3.6/dist-packages/torch/autograd/grad_mode.py in decorate_context(*args, **kwargs)

     13         def decorate_context(*args, **kwargs):

     14             with self:

---> 15                 return func(*args, **kwargs)

     16         return decorate_context

     17 

~/nmt/fairseq/fairseq/sequence_generator.py in generate(self, models, sample, **kwargs)

    175         cand_offsets = torch.arange(0, cand_size).type_as(tokens)

    176 

--> 177         # helper function for allocating buffers on the fly

    178         buffers = {}

    179 

~/nmt/fairseq/fairseq/sequence_generator.py in _generate(self, sample, prefix_tokens, constraints, bos_token)

    220                     unfinalized hypotheses

    221             """

--> 222             assert bbsz_idx.numel() == eos_scores.numel()

    223 

    224             # clone relevant token and attention tensors

~/nmt/fairseq/fairseq/search.py in init_constraints(self, batch_constraints, beam_size)

    237             # make scores cumulative

    238             self.scores_buf.add_(

--> 239                 torch.gather(

    240                     scores[:, :, step - 1],

    241                     dim=1,

TypeError: 'NoneType' object is not iterable

```

Please advise,

Thank you.

#### What's your environment?

 - fairseq Version (e.g., 1.0 or master): master

 - PyTorch Version (e.g., 1.0): 1.5

 - OS (e.g., Linux): 18 LTS

 - How you installed fairseq (`pip`, source): source

 - Build command you used (if compiling from source): pip install -e . for fairseq

 - Python version: 3.6

 - CUDA/cuDNN version: 10.1

 - GPU models and configuration: RTX 2080 Ti

 - Any other relevant information: