The training data for SSF

[Yang Peng]

I am wondering what would the training data for SSF look like?

Would there be some kind of labeled information that we can use to train or test the system?

Since our team has not set up the server machine ready, we have not downloaded the whole data and take a close look at it.

I hope that I could get some answers here.

[John R. Frank]

> I am wondering what would the training data for SSF look like?
> Would there be some kind of labeled information that we can use to train
> or test the system?

Two answers:

1) The SSF task is judged "post-hoc" by pooling all the run submissions
from participants and presenting them (or a ranked subset) to human
assessors at NIST, so there is no training data of that type available
yet. This is one of the reasons everyone should jump on this task this
year, because then your system's output will be part of the training data
for future years of this SSF tasks.

2) Since the slots are existing slot types or super sets of existing
attribute types from TAC KBP and ACE, you can use training data from those
evaluations. This means you can use all of the wonderful data from LDC
and similar collections for detecting meeting locations, meeting
participants, relations between entities, etc.

> Since our team has not set up the server machine ready, we have not
> downloaded the whole data and take a close look at it.

You can see updated links here:

http://s3.amazonaws.com/aws-publicdatasets/trec/kba/index.html

The documents have been tagged with Lingpipe, and all of the hyperlinks
have been converted into "labels" on the tokens. This combined with the
NER and coref chains from Lingpipe is rich metadata.

For example, hyperlinks to sites other than the abs_url can be used to
setup a cross-site link graph. Links to twitter and wikipedia do appear.
The domain name of these hyperlinks can be used as feature in a variety of
models.

You can see an example of iterating over the hyperlink labels here:

https://github.com/trec-kba/streamcorpus/blob/master/examples/py/iterating-over-tokens.py


jrf

[Yang Peng]

Thank you very much! I will check on these things. Have a nice day!

在 2013年4月19日星期五UTC-4下午5时53分39秒，John R. Frank写道：

[John R. Frank]

This relation training data could be useful in KBA SSF:

http://googleresearch.blogspot.com/2013/04/50000-lessons-on-how-to-read-relation.html


jrf