Fwd: Semeval clarification mail draft

[Rakesh Dhanireddy]

I have following questions about the details of the semeval task:

1. Are we allowed to use external knowledge sources like YAGO or DBPedia at all? I understand that we cannot use these as sense repositories. Can we however use them to deduce similarities between various words / phrases in the snippets?

2. More generally, is there a list of do's and do-not's for this task?

--

Rakesh Dhanireddy

[Roberto Navigli]

Dear Rakesh,

we are about to update the page and send a call for participation in which we:

1) explain what you are allowed to do
2) provide an evaluation system, so you can test your system against an existing query data (MORESQUE, see Di Marco & Navigli, 2013).

Best,
Roberto

2013/1/15 Rakesh Dhanireddy <rakeshdh...@gmail.com>

--
 
 

--
=====================================
Roberto Navigli
Dipartimento di Informatica
SAPIENZA Universita' di Roma
Via Salaria, 113 (now in: Viale Regina Elena 295)
00198 Roma Italy
Phone: +39 0649255161 - Fax: +39 06 8541842
Home Page: http://wwwusers.di.uniroma1.it/~navigli
=====================================

[Micha]

Dear Roberto,

I've to ask again on the same topic of Dos and Donts.

I saw the Java Evaluation Tool (thanks for that), but when reading the website I'm still unsure about the exact rules for this task.

1) Search Results Clustering is the final goal in this task, for which WSI or WSD systems could be used. And the task distinguished between WSD and WSI system. Is this correct?

3) Will WSD and WSI systems be evaluated separately?

2) What are exactly the differences in limitations between WSI and WSD?

4) In particular, what are the limitations for WSI vs. WSD regarding external knowledge (dbpedia, wordnet, bablenet, etc)? Are they allowed, forbidden for disambiguating the query word, or forbidden at all for any purpose?

Thanks in advance for some clarification - and for organizing task

Cheers Michael

[Roberto Navigli]

Dear Micha,

2013/2/5 Micha <mic...@informatik.uni-mannheim.de>

Dear Roberto,

I've to ask again on the same topic of Dos and Donts.

I saw the Java Evaluation Tool (thanks for that), but when reading the website I'm still unsure about the exact rules for this task.

1) Search Results Clustering is the final goal in this task, for which WSI or WSD systems could be used. And the task distinguished between WSD and WSI system. Is this correct?

The task is exactly the same and can be performed by both WSD and WSI systems, because systems assign a "label" or "cluster id" to each web search result snippet. Then it's just a matter for us organizers to distinguish between the two classes of systems (see below).
 

3) Will WSD and WSI systems be evaluated separately?

Yes.
 

2) What are exactly the differences in limitations between WSI and WSD?

WSI are fully unsupervised. They just use raw corpora for acquiring the senses of a query (we will ask which kinds of corpora have been used etc.). They might also use the snippets themselves, like done by classical Search Result Clustering systems (see our Computational Linguistics 2013 paper).
 

4) In particular, what are the limitations for WSI vs. WSD regarding external knowledge (dbpedia, wordnet, bablenet, etc)? Are they allowed, forbidden for disambiguating the query word, or forbidden at all for any purpose?

WSD systems can be supervised (i.e. use any kind of training data) or knowledge-based (use resources, like WordNet, BabelNet, DBPedia, etc.).

Hope this helps!

Best,
Roberto
 

--
You received this message because you are subscribed to the Google Groups "Semeval-2013 Task 11: WSI & Disambiguation within An Application" group.
To unsubscribe from this group and stop receiving emails from it, send an email to semeval-2013-wsi-in-a...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.