--
You received this message because you are subscribed to the Google Groups "chibolts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chibolts+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/chibolts/CANkUE2xW3-jbmfqNXyCVLAwZOh8dpCccdTeEWqujDsJQqTgH%3DQ%40mail.gmail.com.
rely +d2 +dm1 sample.cha samplea.cha +t%spa
The output from this command provides two counts: precision and accuracy. These are the two dimensions used in fields such as computational linguistics to evaluate correctness. Precision is the percentage of fields in the master file that are matched by the student. Accuracy is the percentage of fields in the student file that are matched to the master. A failure in precision occurs when the student misses a code in the master file. A failure in accuracy occurs when the student inserts a code that is not found in the master. If a researcher is interested in going beyond these two scores, they can also compute the F score which is the harmonic mean of precision and accuracy. By adding the +d2 switch, the command also produces Cohen's kappa.
On Jun 11, 2020, at 5:17 AM, Karin Edlund <karin.ev...@gmail.com> wrote:
To view this discussion on the web visit https://groups.google.com/d/msgid/chibolts/C233654B-250D-4B81-B2DC-F2005E936177%40andrew.cmu.edu.
On Jun 13, 2020, at 6:34 AM, Karin Edlund <karin.ev...@gmail.com> wrote:Hi Brian,Thank you for your input on the RELY function.What I would like to do is the following:1) Check reliability on percentage agreement for word and utterance level on each of 15 transcriptions comprising 5 min each.I was thinking of using the third function of RELY - the overall match on the main line between two transcripts - rely +d sample.cha samplea.chaThis would be only to check the transcription, not the coding.I would say the two transcribers are on the same level regarding transcribing, thus perhaps it is not a case of checking correctness of a student against a master transcript.Or is the relation master-student more at matter of what data counts as the primary data?
2) Compute Cohen's kappa for the coding. Two coders. I have five different categories for coding. I am not using %spa, but %cod.Would this be the right code for that:rely +d5 sample.cha samplea.cha +t%codwhat does dm1 stand for in the RELY code you suggested?
To view this discussion on the web visit https://groups.google.com/d/msgid/chibolts/CANkUE2y-xD6aGFLzh26e%2Bvx185OJwvSKdAnB4j_%2BTGfmTVSJ_w%40mail.gmail.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/chibolts/116DA890-299A-4EBA-B7C1-DFFB5C22AD1D%40andrew.cmu.edu.
--
You received this message because you are subscribed to the Google Groups "chibolts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chibolts+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/chibolts/CANkUE2zECn_n3DWDUv20P38HYCfzufK6onPhbdPwf9PuJvhVHg%40mail.gmail.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/chibolts/9EE985BB-D778-4BDE-A0CE-C4581D19E5F4%40andrew.cmu.edu.
To view this discussion on the web visit https://groups.google.com/d/msgid/chibolts/CANkUE2xEGX-uAou19zrpX0K_bS-L_8BFP%2B9hxWFR9KG%2BZqrLFw%40mail.gmail.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/chibolts/F1DB467F-5630-4C69-AD40-F2924ABD93AB%40andrew.cmu.edu.