difference between 2012 and 2013 KBA corpora

[John R. Frank]

KBAers,

Here are differences between the kba-stream-corpus-2012 and the
kba-streamcorpus-2013:

2013 .body.raw is the same as 2012, except that in the "social" substream,
the .anchor.raw and .title.raw have been pre-pended to the .body.raw
surrounded by html <p> tags.

2012 .body.cleansed was generated by boilerpipe and lost byte offsets from
raw.

2013 .body.clean_html was generated from raw using libxml

2013 .body.clean_visible was generated either from .body.raw if
media_type==text/plain or from .body.clean_html preserving byte offsets by
replacing all tag-related characters with ' '

2012 .body.ner ---> .body.taggings['stanford'].raw_tagging


**** IMPORTANT ****
2013 .body.sentences['lingpipe'] contains fully sentence-chunked and
tokenized NER+coref-chains generated from clean_visible


StreamItem.other_content dictionary can contain other ContentItem objects.
ContentItem is the class for StreamItem.body. For the
kba-streamcorpus-2013, the only other_content is "anchor" and "title" in
the "social" substream imported from the kba-stream-corpus-2012.

StreamItem.source_metadata carries other information that can be very
rich. The schema depends on the upstream source and can be complex.


Let us know if you have any questions.

jrf

[John R. Frank]

follow up to post in trec-kba:
https://groups.google.com/d/msg/trec-kba/24ClmbLg9TU/PGR-g10gM1IJ

> **** IMPORTANT ****
> 2013 .body.sentences['lingpipe'] contains fully sentence-chunked and
> tokenized NER+coref-chains generated from clean_visible

tokens inside of an HTML Anchor tag have Token.labels['author'] populated
with Label objects and Label.target.target_id=<HREF URL>

This can be a powerful feature for modeling.


jrf

[Anshul Mittal]

What we have noticed is that the NER tagging given with the data is not that good and simply running the NER tagging again using stanford corenlp gives much better results (may be because an older version was used when generating this for the corpus?)

[John R. Frank]

> > > follow up to post in trec-kba:
> > > https://groups.google.com/d/msg/trec-kba/24ClmbLg9TU/PGR-g10gM1IJ
> > >
> > > **** IMPORTANT ****
> > > 2013 .body.sentences['lingpipe'] contains fully sentence-chunked and
> > > tokenized NER+coref-chains generated from clean_visible
> >
> >
> > tokens inside of an HTML Anchor tag have Token.labels['author']
> > populated with Label objects and Label.target.target_id=<HREF URL This
> > can be a powerful feature for modeling.

Note that the 'author' labels are from the author of the web page, not an
automatic tagger. For example, the corpus contains hyperlinks to twitter
and wikipedia.

> What we have noticed is that the NER tagging given with the data is not
> that good and simply running the NER tagging again using stanford
> corenlp gives much better results (may be because an older version was
> used when generating this for the corpus?)

It is true that the Stanford CoreNLP can produce good results.

It is also >100x slower than LingPipe. Running the full within-doc coref
model from Stanford CoreNLP was simply too computationally expensive to
run on a billion documents.

The within-doc coref chains from LingPipe out-of-the-box are pretty good,
especially considering that we ran it without any tuning for the specific
genres observed in each of the different "source" substreams. Since they
are available for the entire corpus, you can use them as part of a
first-pass filter.


jrf