VOCD is too high

[Nick Riches]

Hi

I'm teaching CLAN to Speech and Language Therapy students. As an exercise they have to calculate VOCD

One student is consistently getting VOCD scores much higher than 70 (for the VOCD program) and around 100 for KIDEVAL. This is a young child. I've had a brief look and can't see anything odd about the coding that would lead to these kind of results. I've attached the CHAT script, and the MOR.PST.CEX scripts.

Does anyone know why it's so high?

Thanks in advance.

Nick

[Brian MacWhinney]

Dear Nick,

      Looking at the file you sent, the VOCD score seems about right, because this child was using a lot of different words.  Just by way of comparison,  I ran VOCD on the various files in the Brown/Eve file and Eve got higher and higher scores as Eve got older.  Everything seemed fine.  Leonid checked on the issue you mention about KIDEVAL giving different values from a separate run of VOCD and he could not see problems there either.  

     

—Brian MacWhinney

--
You received this message because you are subscribed to the Google Groups "chibolts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chibolts+u...@googlegroups.com.
To post to this group, send email to chib...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/chibolts/de8b6285-bd47-4ae8-ae36-76e2608cb027%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Nan Ratner]

I didn't notice how old the child was, but I'd say Brian is right. In fact,
we just wrote an article together in which we took a look at language
sample analysis (LSA) metrics for a large number of kids in the Archive
talking with their mom or another adult (typical toy play scenarios) and we
definitely saw kids hit an "average" in that range between 4 and 6 years,
and the range definitely went that high well before that. As you know, the
problem with defining a real "norm" for VocD is its resampling algorithm
that won't give you exactly the same score twice, down to the decimal
point, but we are going to see if we can tackle this in the near future
statistically.
I am attaching a pdf of the article if you'd like to see our results. BTW,
I am glad to see someone using this kind of assignment. In case it helps
any other teachers out there, I kept the issue of VocD replicability to
myself until after the assignment was handed in, just to see if I could
spot people handing in one another's work, since for undergrads I gave them
a choice of only about a dozen taped interactions they could use to try
their hands at LSA. Over the years, I did catch just one pair of students.

Nan

--

You received this message because you are subscribed to the Google Groups "chibolts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chibolts+u...@googlegroups.com.
To post to this group, send email to chib...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/chibolts/de8b6285-bd47-4ae8-ae36-76e2608cb027%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Nan Bernstein Ratner,

Professor and Chairman

Dept. of Hearing and Speech Sciences

0100 Lefrak Hall

University of Maryland

College Park, MD 20742

301--405-4217

301-314-2023 (fax)

[Nick Riches]

Dear Brian

Thanks for getting back so quickly. I've just checked, and KIDEVAL is consistent with VOCD. Apologies - the student reported a difference, and I should have triple-checked before posting.

Regarding VOCD, I've been using the norms provided in  Durán, P., Malvern, D., Richards, B., & Chipere, N. (2004). Developmental Trends in Lexical Diversity. Applied Linguistics, 25(2), 220–242. http://doi.org/10.1093/applin/25.2.220. This seems to be the most recent published source of norms on D, but I may be wrong. According to these, a D score of 70 would put a child aged 6 years old in the 90th centile. However, according to the norms which Nan sent through in the 2016 article "Your laptop to the rescue", a D score of 70 is about average for this age. I presume that these recent norms are more comprehensive. So the issue is not the accuracy of the software, but where to obtain a good set of norms to interpret VOCD.

It would be great if I could circulate this chapter to the students, but it may be subject to copyright. I could also develop my own norms, but my CLAN skills are not quite up to scratch (just yet!)

Best wishes

Nick Riches

[Nick Riches]

Hi

Many thanks for the help. I've just replied to Brian (above). It appears that this is not a software issue, but one related to norms. The norms I circulate are those in   Durán, P., Malvern, D., Richards, B., & Chipere, N. (2004). Developmental Trends in Lexical Diversity. Applied Linguistics, 25(2), 220–242. http://doi.org/10.1093/applin/25.2.220. But these give a much lower average D per age-band than your more recent norms, which are obviously much more comprehensive.

Thanks for the Chapter. I was wondering whether I could distribute it to students, or whether it is subject to copyright?

That's a great tip about using the variability in D to investigate plagiarism. Sadly, I can't use it with our students as they collect their own data...

Best wishes

Nick

[Nick Riches]

P.S. the Chapter is clearly copyrighted, so I'll order a copy of the book when it becomes available.

Nick

[Brian MacWhinney]

Nick,

    Yes, the norms Nan extracted for that article are going to be the gold standard of the future.  The plan is to incorporate these norms directly into the KIDEVAL program.   Soon ….

—Brian

To view this discussion on the web visit https://groups.google.com/d/msgid/chibolts/5fa34d49-1032-4847-8d0e-d6d6bb8998eb%40googlegroups.com.