Test data gold standad and evaluation script

Skip to first unread message

Javier Artiles

Dec 18, 2008, 4:24:11 PM12/18/08
to Web People Search Task
Dear participants,

I have updated the WePS 2 public resources to include the test data
gold standard and a new version of the scoring package.

The gold standard annotation is the result of a double annotation of
the corpus. It contains notes both in clusters and documents that
might be useful if you want to carry a qualitative analysis of the

You can also download a new version of the scoring package (includes
source code in Java). It is compatible with the current gold standard
format and can be extended if you want to apply different scoring

Best regards,

Javier Artiles (on behalf of the WePS organizers)

Javier Artiles

Jan 13, 2009, 11:35:41 PM1/13/09
to Lan Man (Gmail), web-people-...@googlegroups.com
Hi Lan,

It seems to be a problem with the way Windows manages file paths (I´ve always used the scoring script on Linux machines). Please try using absolute paths in your command line parameters. I´ll test it on a Windows to see if there is something I can do to fix it.



On Mon, Jan 12, 2009 at 2:43 AM, Lan Man (Gmail) <lanman.sg@gmail.com> wrote:
Hi Javier,
Thanks for your work on this test data resources. However, when I use this new scoring package, I found there is only one wepsScorer.jar file in this zip file. When I input the command as follows:
java -jar wepsScorer.jar metadata truth result output -ALLMEASURES -AllInOne
It prompts the following error message as I attached a image for your reference.
I also tried other methods, such as upgrade or downgrade the java runtime version (as it works in the previous scoring package, wepsEvaluation.jar). But they all failed.
My system is Windows XP. I wonder if other participants have this prolem. This is improtant since I will do a analysis in my paper based on the evaluation.
Thanks a lot for your help.
best regards,
Lan Man
Javier Artiles Picón
Departamento de Lenguajes y Sistemas Informáticos
ETSI Informática, UNED

Phone: +34 91 398 8106
Fax:     +34 91 398 65 35
Home page:    nlp.uned.es/~javier
LinkedIn page: www.linkedin.com/in/javierartiles
Reply all
Reply to author
0 new messages