Wikicorpus - wordids.txt

[Nikolaos Aletras]

Hello,

Please let me know what is the third number in each line of the file
OUTPUT_PREFIX_wordids.txt (mapping between words and their integer
ids) in the WikiCorpus.

e.g. the first number is the integer id of a word, then it is the word
and there is a third number. Is it the term frequency in the entire
wikicorpus?

Thanks in advance,

Nikos

[Radim Řehůřek]

Hello Nikos,

that is the *document* frequency of that term (=in how many documents
did it appear).

Best,
Radim