Format of the gold dataset in the scripts/goldDir

[Negacy D. Hailu]

Hi,

I have some problems running the evaluation scripts. What exactly is the format of the gold dataset in the goldDir?

I am assuming this dataset can be prepared from submission file but in an xml file. 

For example: assuming my submission file has two lines:

DDI-DrugBank.d769.s2|DDI-DrugBank.d769.s2.e1|DDI-DrugBank.d769.s2.e3|1|effect

DDI-DrugBank.d769.s2|DDI-DrugBank.d769.s2.e2|DDI-DrugBank.d769.s2.e3|0|null

So, am I correct if my gold dataset is:

<pair id="DDI-DrugBank.d769.s2" e1="DDI-DrugBank.d769.s2.e1" e2="DDI-DrugBank.d769.s2.e3" ddi="true" type="effect"/>

<pair id="DDI-DrugBank.d769.s2" e1="DDI-DrugBank.d769.s2.e2" e2="DDI-DrugBank.d769.s2.e3" ddi="false" type="null"/>

But when I run the evaluation script, I am having the following warning:

[evaluateDetection] Warning! DDI-DrugBank.d769.s2|DDI-DrugBank.d769.s2.e1|DDI-DrugBank.d769.s2.e3|0|effect does not exist in the gold

[evaluateDetection] Warning! DDI-DrugBank.d769.s2|DDI-DrugBank.d769.s2.e2|DDI-DrugBank.d769.s2.e3|0|null does not exist in the gold

Thanks.

-----------------
Negacy D. Hailu

[Isabel Segura]

Hi Negacy,

The format of the gold dataset in the goldDir is the same of the corpus with pair tags (http://www.cs.york.ac.uk/semeval-2013/task9/index.php?id=data).

It seems that you are doing well.

For me it is very difficult to determine where the problem is. What files do you have in the directoy goldDir?

Thanks, Isabel



 

--
You received this message because you are subscribed to the Google Groups "ddiextraction_semeval" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ddiextraction_se...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Isabel Segura Bedmar
Despacho 2.2.A.10, Telf: 91 624 99 88
Departamento de Informática, Universidad Carlos III de Madrid,

Laboratory for Advanced Database (LABDA)

http://labda.inf.uc3m.es/doku.php?id=en:inicio

[Isabel Segura]

Sorry, I do not understand, which is your gold dataset?

On 13 March 2013 20:52, Negacy Hailu <neg...@gmail.com> wrote:

Hi Isabel,

I have only my gold dataset inside the goldDir. And my submission file is on the scripts directory which is the parent directory for goldDir. I am getting the warning for each DDI pairs.

Thanks.