Let's continue where we left off on 7/2 on aggregating turker labels
-- taking a look at the Whitehill paper linked below.
Whose vote should count more: Optimal integration of labels from
labelers of unknown expertise.
http://mplab.ucsd.edu/~jake/OptimalLabeling.pdf
-----------
For completeness, URLs for for the Xu paper we talked about last time,
as well as the snow paper we touched on; we can remind ourselves what
was going on here and then discuss Whitehill.
Fang Xu, Dietrich Klakow: Paragraph Acquisition and Selection for List
Question Using Amazon's Mechanical Turk. LREC 2010.
http://www.lsv.uni-saarland.de/241_Paper.pdf
Snow, R. and O'Connor, B. and Jurafsky, D. and Ng, A.Y. Cheap
and fast---but is it good?: evaluating non-expert annotations for
natural language tasks. Proceedings of the Conference on Empirical
Methods in Natural Language Processing.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.142.8286&rep=rep1&type=pdf