[trec-kba] exciting --- and important survey

[John R. Frank]

KBAers,

We're very excited about the KBA submissions that NIST received: 118 runs
from 11 teams. More on that soon.


Track proposals for 2015 are due next week, and we want your input.

Could you please take this survey?

https://www.surveymonkey.com/s/BSBNBCX



It has this question:

For TREC 2015, we are considering several options for entity-oriented
retrieval tasks. If you had to choose between a task that is
stream-filtering focused versus a different task that is end-user search
focused, which would you choose? In both tasks, the queries are entities
that are mentioned in a corpus of text, and the goal of the human user is
to improve a profile about the entity. The difference between these two
tasks is whether the human is reacting to the stream of data or is
proactively pursuing subtopics of interest within the entity profile. In
the first task, the user needs help filtering. In the second task, the
user needs help searching.

1) Stream-driven (filtering), like TREC KBA, in which the time-stamped
corpus data drives the evolution and expansion of the entity. See the TREC
KBA "Vitality Rating" task as an example of this. The truth data
identifies texts from the corpus that introduce timely, new information
into each entity's profile; the same data appearing later in the corpus is
consider less useful to the human user.

2) User-driven (searching), in which the human user's interests in the
entity drive the evolution of the profile. For example, the truth data in
such a task would identify which passages a user found "interesting" or
"surprising" about an entity. The truth data would associate these
passages into equivalence classes below the entity-level, i.e. subtopic
granular assessor data, and would ignore any timestamps on the corpus
data.




John