Infer vectors for each word of a new document

[George Manoliadis]

Dear members, 

After training a model with either Word2Vec or Doc2Vec, I have a new document that I would like to infer a vector for each of its words. How is that possible?

I have seen that the Word2Vec only has train, and Doc2Vec has infer_vector for the whole document, not for each word of the new document.

Regards, 

George 

[Gordon Mohr]

There's no 'infer-word' functionality in Gensim. It's plausible that it could be written, & work similarly to `Doc2Vec.infer_vector()`.

If inferring from just one or a few examples of a new word's usage, the resulting vector might be quite peculiar to those limited usages: of little use elsewhere. But, hard to tell for sure without running experiments.

The crude approach of simply modeling any new unknown word as some weighted average of the words found near it might come usefully close to the results of a full formal inference, because after all, neighboring words are all trained with very-similar overlapping contexts. But, across a large varied corpus, it may be the slight remaining contrasts with common neighbors that dominate the distinctiveness of a fully-trained word. So, hard to know the value of inference-over-averaging without running experiments. 

- Gordon

[George Manoliadis]

Gordon, 

Thank you for your answer. 

When you say "full formal inference" what do you mean? And which functions can someone use to implement it? 

- George

[Gordon Mohr]

I mean something that's highly analogous to the `Doc2Vec` tag-vector inference: repeating a training process, but where only the new words, driven by examples of only their usage, get adjusted. There's no existing support. 

The methods supporting `Doc2Vec` inference might be a useful precedent to study for ideas. The crude/underdocumented/not-officially-supported `lockf` functionality deep in the code was one experiment along these lines, but is pretty inefficient & error-prone. 

- Gordon