Type of neural network used in Doc2Vec

[Miles Hinson]

I'm using the Doc2Vec class for a project, and I was wondering if what type of neural network is used to train the model - I wasn't sure based on looking at the code

[Gordon Mohr]

Since the gensim Doc2Vec class implements the 'Paragraph Vector' algorithm, the most formal/authoritative explanations will be via the original 'Paragraph Vectors' paper (http://arxiv.org/abs/1405.4053) and the Word2Vec papers on which that is based (http://arxiv.org/abs/1301.3781 and http://arxiv.org/abs/1310.4546). 

Some key aspects to note:

* the NN involved is 'shallow' with one hidden layer

* for each training example, the NN 'inputs' are determined by the choice of training contexts – in Doc2Vec 'DM' or 'DBOW' (after that paper's definitions), in Word2Vec 'CBOW' or 'Skip-Gram' (after that paper's definitions)

* the target NN 'outputs' (and thus errors-to-be-backpropagated) are determined by the choice of sparse training strategy – hierarchical softmax or negative-sampling

- Gordon

[Suhas]

Thanks, Gordon. I was wondering if that NN had a name or something. Is it a one-layer MLP? Any particular implementation that gensim uses (like Tensorflow)? 

[Gordon Mohr]

Gensim isn't using any other NN library, just numpy/scipy functions for this super-shallow and super-simple NN. 

I suppose it'd count as a multi-layer perceptron (with just one hidden layer), but if concerned about taxonomic accuracy, I'd rely on any terms used in the original Word2Vec/'Paragraph Vector' papers.

- Gordon