Test data gold standad and evaluation script

[Javier Artiles]

Dear participants,

I have updated the WePS 2 public resources to include the test data
gold standard and a new version of the scoring package.
http://nlp.uned.es/weps/weps2/WePS2_test_data.zip

The gold standard annotation is the result of a double annotation of
the corpus. It contains notes both in clusters and documents that
might be useful if you want to carry a qualitative analysis of the
results.

You can also download a new version of the scoring package (includes
source code in Java). It is compatible with the current gold standard
format and can be extended if you want to apply different scoring
metrics.

Best regards,

Javier Artiles (on behalf of the WePS organizers)

[Javier Artiles]

Hi Lan,

It seems to be a problem with the way Windows manages file paths (I´ve always used the scoring script on Linux machines). Please try using absolute paths in your command line parameters. I´ll test it on a Windows to see if there is something I can do to fix it.

Regards,

Javier.

On Mon, Jan 12, 2009 at 2:43 AM, Lan Man (Gmail) <lanman.sg@gmail.com> wrote:

Hi Javier,

 

Thanks for your work on this test data resources. However, when I use this new scoring package, I found there is only one wepsScorer.jar file in this zip file. When I input the command as follows:

java -jar wepsScorer.jar metadata truth result output -ALLMEASURES -AllInOne

It prompts the following error message as I attached a image for your reference.

 

I also tried other methods, such as upgrade or downgrade the java runtime version (as it works in the previous scoring package, wepsEvaluation.jar). But they all failed.

 

My system is Windows XP. I wonder if other participants have this prolem. This is improtant since I will do a analysis in my paper based on the evaluation.

 

Thanks a lot for your help.

 

best regards,

Lan Man

--
------------------------------------------------------------------------------
Javier Artiles Picón
Departamento de Lenguajes y Sistemas Informáticos
ETSI Informática, UNED

Phone: +34 91 398 8106
Fax:     +34 91 398 65 35
Home page:    nlp.uned.es/~javier
LinkedIn page: www.linkedin.com/in/javierartiles
------------------------------------------------------------------------------