help comparing mean sentence length

[john mckenny]

I'd appreciate some advice on comparing the significance of the
difference in mean sentence length and mean word length in two
corpora. Wordsmith 5 statistics are attached.
Thank you
Seanthepenman

[Stefan Th. Gries]

Hi

Since word and sentence lengths are not normally distributed, you
can't usually do t-tests using just the means, the standard
deviations, and the sample sizes as parameters. In your case, of
course the differences will be significant: your n's are huge!

### As for word lengths: the corpus samples can be recreated from the
stats you provide (unless I made a mistake, the corpus sample sizes
don't add up to the token frequencies, but the differences are
negligible):

# INPUT
corpus.A.freq<-c(3640, 20662, 22260, 16399, 11562, 9213, 9537, 6819,
5443, 3624, 2135, 1102, 551, 227, 92, 20, 9, 9); corpus.A<-rep(1:18,
corpus.A.freq)
corpus.B.freq<-c(3871, 21685, 21590, 16293, 11344, 9258, 9621, 7197,
6020, 4406, 2711, 1528, 912, 395, 210, 63, 30, 17);
corpus.B<-rep(1:18, corpus.B.freq)
length(corpus.A) # 113304
length(corpus.B) # 117151

# TEST FOR NORMALITY
ks.test(jitter(corpus.A), "pnorm", mean=mean(corpus.A), sd=sd(corpus.A))
ks.test(jitter(corpus.B), "pnorm", mean=mean(corpus.B), sd=sd(corpus.B))
par(mfrow=c(1, 2))
hist(corpus.A, xlim=c(0, 18), ylim=c(0, 25000))
hist(corpus.B, xlim=c(0, 18), ylim=c(0, 25000))
par(mfrow=c(1, 1)) # of course not normal

# SIGNIFICANCE TEST and DISTRIBUTION PLOT
wilcox.test(corpus.A, corpus.B, paired=F)
plot(ecdf(corpus.A), verticals=T, col.v="blue", col.h="blue");
lines(ecdf(corpus.B), verticals=T, col.v="red", col.h="red")
# Bottom line: significant result, as expected, but I think it's
pretty clear that the significance only results from the large sample
sizes: the effect size and the distributional differences are *tiny*.

### As for sentence lengths: the output does not provide the
information needed to recreate the data for the same test but with
that n, of course it's significant.

HTH,
STG
--
Stefan Th. Gries
-----------------------------------------------
University of California, Santa Barbara
http://www.linguistics.ucsb.edu/faculty/stgries
-----------------------------------------------