Statistics on WS 4.0

[Elaine]

Hi, I'm using WS 4.0, and I'm having some problems with the
statistics. I'm looking at individual speaker total tokens etc. for
each file but once this is computed for each speaker, the combined
total for all speakers does not add up the the total running tokens
for the corpus itself. This is a corpus of online discussions, would
the reason for this be possibly due to emoticons etc. that WS is not
recognising?
Thanks,
E.

[Mike]

Elaine, hi

There are lots of complexities in handling words, such as the handling
of apostrophes & hyphens, whether you want numbers included in your
word list etc. In WS5 I included a category of "tokens included in the
word list" to distinguish between the tokens WordSmith recognises as
tokens of some sort in the corpus and those it is programmed to
include in a word list. Emoticons would get seen as mere punctuation
symbols and as such would not get counted, just as the pictures
accomanying the text don't get included.

Cheers -- Mike