Nugget Matching Interface

[Andreas Rueckle]

Dear all,

I am currently working on a master thesis in the area of real-time summarization of big data streams. Since the focus of this work is somewhat different from the trec-ts challenge, I am not an active participant of this years summarization task. However, I am using the 2014 corpus as well as the 2014 queries. As the work progresses, I am now looking into different approaches for evaluating the system. For comparison reasons, I'd like to use the official 2014 trec-ts evaluation script. For this evaluation, a matching between nuggets and updates is required. I saw that in 2014 these matchings were created manually by assessors through a matching interface (as shown in figure 5 in the overview paper on http://trec.nist.gov/pubs/trec23/papers/overview-tempsumm.pdf).

Is this interface available for download somewhere? If not, can you publish it?

Thanks in advance,
Best regards,
Andreas Rücklé