MLU and FREQ - words and tokens
13 views
Skip to first unread message
Ailín Franco
Nov 9, 2022, 1:18:55 PM11/9/22
to chibolts
Dear all,
Sorry to write to you, but I've just started to use the programm and I have some doubts.
I am using MLU and FREQ and I saw a difference (in some cases) between the count of the words and tokens, for example:
MLU for Speaker: *CHI:
MLU (xxx, yyy and www are EXCLUDED from the utterance and morpheme counts):
Number of: utterances = 19, words = 60
Ratio of words over utterances = 3.158
Standard deviation = 2.134
MLU (xxx, yyy and www are EXCLUDED from the utterance and morpheme counts):
Number of: utterances = 19, words = 60
Ratio of words over utterances = 3.158
Standard deviation = 2.134
54 Total number of different item types used
95 Total number of items (tokens)
0.568 Type/Token ratio
This TTR number was not calculated on the basis of %mor line forms.
If you want a TTR based on lemmas, run FREQ on the %mor line
with option: +sm;*,o%
95 Total number of items (tokens)
0.568 Type/Token ratio
This TTR number was not calculated on the basis of %mor line forms.
If you want a TTR based on lemmas, run FREQ on the %mor line
with option: +sm;*,o%
I am not calculating the numbers on %mor line, in neither of them.
I am reading the manual but I don't undertand why is a difference in the numers of tokens and words
Thank you in advance,
Ailín
Brian Macwhinney
Nov 9, 2022, 2:43:00 PM11/9/22
to ChiBolts
Alin,
FREQ looks at all words. MLU has lots of exclusions, based on the rules formulated by Brown (1973), as described in the manual in section 7.8. If you think that the rules stated there are not in accord with Brown’s principles, I would need to see your input file.
—Brian MacWhinney
> --
> You received this message because you are subscribed to the Google Groups "chibolts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to chibolts+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/chibolts/88f728e8-db36-4cbe-8121-38285bb44598n%40googlegroups.com.
FREQ looks at all words. MLU has lots of exclusions, based on the rules formulated by Brown (1973), as described in the manual in section 7.8. If you think that the rules stated there are not in accord with Brown’s principles, I would need to see your input file.
—Brian MacWhinney
> You received this message because you are subscribed to the Google Groups "chibolts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to chibolts+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/chibolts/88f728e8-db36-4cbe-8121-38285bb44598n%40googlegroups.com.
Ailín Franco
Nov 9, 2022, 3:08:15 PM11/9/22
to chib...@googlegroups.com
Hi, Brian
Thank you for your quick response. I am going to revise this section in order to understand the MLU exclusions (eg repetitions and reformulations that are coded as such, as I read in one of your papers)
Thank you again.
Ailín
To view this discussion on the web visit https://groups.google.com/d/msgid/chibolts/7FE33B7B-3A82-419A-957A-38F4FD5BA29B%40andrew.cmu.edu.
Ailín P. Franco Accinelli
Becaria Doctoral ConicetCentro Interdisciplinario de Investigaciones en Psicología Matemática y Experimental "Dr. HORACIO J. A. RIMOLDI" (CIIPME - CONICET)
Reply all
Reply to author
Forward
0 new messages