meaning of task = "IE"

[Karni Gilon]

HI,

I work with quick start now. and use the configuratipn: MaxEntClassificationEDA_Base+VO+TP+TPPos+TS_EN.xml

I created an xml test file with some pairs of sentences, and noticed that the option task="null" (which is used when testing a single T/H pair)

creates very different results from the option="IE which appears in the example file.

Can you please explain the difference or point me to the relevant link

Thanks

Karni..

[Roberto Ferrari]

Hello,

We've just asked the question to Rui who his the developer of that EDA. Below the answer

Best

Roberto

==================================================================================

Originally, the RTE data set was created from different sources of data. One source is information extraction (IE) tasks. By constructing the hypothesis (H) using named entities and their relations, those T-H pairs are annotated as "task=IE". The same for the other tasks (IR, QA, etc.). You may check out the description of the data preparation section of the original RTE challenge paper for more details:

http://u.cs.biu.ac.il/~dagan/publications/RTEChallenge.pdf

As for the performance difference, when training the model, this task (if available) is considered as one feature. We found out that T-H pairs from different task sources have different characteristics (e.g., there are more named entities in the IE pairs). If you train the model with this feature, during testing, whether providing the "task" will have an impact on the result; if your testing data do not have this information, I would suggest to remove "task" from the training set to have a 'general-purpose' model.

--
You received this message because you are subscribed to the Google Groups "eop-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eop-users+unsubscribe@googlegroups.com.
To post to this group, send email to eop-...@googlegroups.com.
Visit this group at https://groups.google.com/group/eop-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/eop-users/8422b4ba-556c-4296-9cab-66fed4990220%40googlegroups.com.