Re: [TREC-TS14] TREC 2014 Temporal Summarization Test Events and Guidelines

[Sayaka Kitaguchi]

Hi Fernando,

Thank you for answering my question.
I see. I will use 2013 events to train our classifier.

Regards,
Sayaka

(2014/07/03 16:44), Fernando Diaz wrote:
> I want to stress that this is the only "evidence from the future" participants should use.
>
>
>
>> On Jul 3, 2014, at 12:42 AM, "Fernando Diaz" <fd...@microsoft.com> wrote:
>>
>> Sayaka,
>>
>> Thanks for pointing this out. This should be fine.
>>
>> Fernando
>>
>>
>>
>>> On Jul 3, 2014, at 12:19 AM, "Sayaka Kitaguchi" <kita3...@gmail.com> wrote:
>>>
>>> Dear Fernando,
>>>
>>> I am one of participants in the TREC 2014 Temporal Summarization track.
>>> I have a question about test and training events data from 2013 Temporal Summarization track.
>>>
>>> I'm planning to use 2013 test and training events to train our classifier.
>>> Some 2014 test events exist before the start time of 2013 events.
>>> May I use 2013 events to train our classifier for these 2014 events?
>>>
>>> For Example, May I use a classifier that has been trained in pooled updates on topic 1 (start time is 1329910380) to classify the sentences on topic 18 (start time is 1322956800) ?
>>>
>>> Thank you,
>>> Sayaka
>>>
>>>
>>> (2014/06/19 22:59), Fernando Diaz wrote:
>>>> Several important updates for temporal summarization.
>>>>
>>>> *I. 2014 Test Events*
>>>>
>>>> We have posted the test events on the Temporal Summarization website. You can find the events here,
>>>>
>>>> https://sites.google.com/site/temporalsummarization/documents
>>>>
>>>>
>>>> Because these are test events, you should not train your system using these events. We stress that participants should not use information `from the future' in their systems, including corpus-level IDF. This is explained further in the guidelines.
>>>>
>>>> *II. 2014 Guidelines
>>>> *
>>>> We have posted the 2014 guidelines,
>>>>
>>>> https://sites.google.com/site/temporalsummarization/documents
>>>>
>>>>
>>>> Important changes from last year include,
>>>>
>>>> * there is no value tracking subtask
>>>> * we will be releasing a smaller, filtered corpus with documents returned by a baseline retrieval algorithm.
>>>>
>>>>
>>>> *III. 2014 Metrics*
>>>>
>>>> We have included a composite measure of expected gain and comprehensiveness (Equation 26).
>>>>
>>>> *IV. 2013 Data*
>>>>
>>>> You may use 2013 events as training events for 2014. Data and evaluation scripts can be found on the TREC website,
>>>>
>>>> http://trec.nist.gov/data/tempsumm2013.html
>>>>
>>>>
>>>> As always, please post any questions to this list.
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google Groups "temporalsummarization2014" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send an email to temporalsummarizat...@googlegroups.com <mailto:temporalsummarizat...@googlegroups.com>.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups "temporalsummarization2014" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an email to temporalsummarizat...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups "temporalsummarization2014" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to temporalsummarizat...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.