corpus comparison
aris
As part of my dissertation I intend to compare 3 corpora of General
English textbooks, Engineering MSc entrance English texts and the
subject-related authentic English texts for their coverage of the
Academic Word List (words that occur frequently in academic texts).
I have used the software RANGE (Nation, 2002) to determine the
coverage of each corpus, and I have found difference in coverage.
My question is if I can use percentages with Chi square to show if the
difference is significant, and also if a goodness of fit is also
needed to show that the percentages of coverage found are not
accidental. The sizes of my corpora are also not equal; is this
important? Can I compensate for it in any way?
thanks
Ray Koopman
If your dependent variable for each text is the proportion of words
in the Academic Word List that occur at least once in that text then
I would do a heteroscedastic (unequal variances) one-way analysis of
variance on those proportions. I generally prefer the Brown-Forsythe
test (Technometrics, 1974) to the Welch test (Biometrika, 1947), but
either is preferable to the ordinary anova if the sample sizes are
very unequal. (Note: I'm *NOT* referring to the Brown-Forsythe anova
on the absolute deviations from the group medians, that compares
spreads.)
Rich Ulrich
<ahakhv...@gmail.com> wrote:
>Hi
> As part of my dissertation I intend to compare 3 corpora of General
>English textbooks, Engineering MSc entrance English texts and the
>subject-related authentic English texts for their coverage of the
>Academic Word List (words that occur frequently in academic texts).
>
>I have used the software RANGE (Nation, 2002) to determine the
>coverage of each corpus, and I have found difference in coverage.
Wikipedia shows me that the Academic Word List has 570
words, divided into 10 sublists. I don't have any idea
how those are used by your software.
I have read Ray's post, and I wonder if he may be jumping
the gun by failing to nail down what measures are being
made, and whether the terms of description are appropriate
for drawing conclusions at all - that would be necessary before
doing tests.
Okay, you have found "a difference in coverage."
How do you describe that difference? Is it a description
that removes the doubts that the reader may have,
about comparability?
As I think you suggest below -- If one "score" is based on a
text that has 10 times as many words in all, it is only natural
that it will have more words from the List.
For any given BOOK, you could conceivably draw a curve
that describes the relation between [thousands-of-words-
counted] and [unique words from the List that are included].
I would expect there to be differences between authors;
if the differences-by-author are strong enough, it would be
appropriate to analyze by author rather than text -- assuming
that one author may have written more than one text, or
one text may include more than one author.
>
>My question is if I can use percentages with Chi square to show if the
>difference is significant, and also if a goodness of fit is also
>needed to show that the percentages of coverage found are not
>accidental. The sizes of my corpora are also not equal; is this
>important? Can I compensate for it in any way?
--
Rich Ulrich