averaging word vectors in gensim

[Donna Hoffman]

Does gensim have a method that could give (weighted) averages of the word2vec vectors for each phrase/sentence? That would be great, especially if weights were tf-idf or idf.

[Gordon Mohr]

There's no such single method in Word2VecKeyedVectors. 

For its own purposes KeyedVectors does a simple (non-weighted) average of multiple vectors as a one-liner inside a few of its other methods. Some examples:

https://github.com/RaRe-Technologies/gensim/blob/1f357a7c4db27ea9c946dbc6942d82b00815a55e/gensim/models/keyedvectors.py#L510

https://github.com/RaRe-Technologies/gensim/blob/1f357a7c4db27ea9c946dbc6942d82b00815a55e/gensim/models/keyedvectors.py#L736

https://github.com/RaRe-Technologies/gensim/blob/1f357a7c4db27ea9c946dbc6942d82b00815a55e/gensim/models/keyedvectors.py#L853

- Gordon

[Donna Hoffman]

Wow, Gordon, this is great! And that code actually addresses another question I had but hadn't posted yet. Thanks!  :)

[Radim Řehůřek]

Let me add that we've had good experience with weighted averaging too (for example, where the weights correspond to IDF = inverse document frequencies, as a simple proxy to "term importance"; other more domain/thesaurus/NER-specific weightings are worth considering too, depending on your application's goals).

-rr

[Ivan Menshikh]

Hello Donna,

I also very recommended look at this paper  (page 4, algorithm 1), IMO this is one of the best ways how to combine word-vectors.

[Radim Řehůřek]

Nice paper Ivan! Do we have this algo in Gensim (or any plans)?

-rr

[Donna Hoffman]

Very nice paper! thanks for the pointer.

[Ivan Menshikh]

Probably if we extend this idea, this can be simple, nice and useful incubator project (I have been thinking about it for a long time).

[Ivan Menshikh]

UPD: I created a feature request (it seems that I already have a student for this task)