ROUND 4 datasets are available online + SUBMISSION INSTRUCTIONS

58 views
Skip to first unread message

Ernesto Jimenez-Ruiz

unread,
Oct 20, 2020, 9:34:39 AM10/20/20
to Sem-Tab Challenge
Dear all

The tables and targets for Round 4 are available online:  www.cs.ox.ac.uk/isg/challenges/sem-tab/2020/

Note that the evaluation of this round will be BLIND and OFFLINE. That is, we will not be using AICrowd for the submission and evaluation. Participants should submit their solutions for the CEA, CTA and CPA tasks to https://tinyurl.com/semtab2020-round4

Deadline: October 27, 23:59 (AoE, Anywhere on Earth)

Results and award winners will be announced during the ISWC conference on November 5th.

Good luck!
Ernesto

Phuc

unread,
Oct 26, 2020, 10:31:00 AM10/26/20
to Sem-Tab Challenge
Dear organizers, and Ernesto,

Thank you very much for the information.

I would like to ask about the submission format for the Round 4

1. Could I submit non-prefix annotations?
For example "Q476028". A ground truth example is "http://www.wikidata.org/entity/Q476028"

2. The submision format is in double quotes or not?
In the Round 4 ground truth examples, I see the csv files is not in a double quotes format. Can I submit with the double quotes csv format (similar to the challenge description)?

3. What is the rule of namming submission files?
I would like to compress all my submissions into a zip file as the follow. Is it OK?
MTab4Wikidata.zip ~ (Size: 9 -10MB)
- cea.csv
- cta.csv
- cpa.csv

Thank you very much and I am looking forward seeing your reply.

Best regards,

Phuc
NII, Japan

Ernesto Jimenez-Ruiz

unread,
Oct 26, 2020, 11:11:28 AM10/26/20
to Phuc, Sem-Tab Challenge
Dear Phuc

1. Could I submit non-prefix annotations?
For example "Q476028". A ground truth example is "http://www.wikidata.org/entity/Q476028"
Yes, this will be accepted as in previous rounds.
 

2. The submision format is in double quotes or not?
In the Round 4 ground truth examples, I see the csv files is not in a double quotes format. Can I submit with the double quotes csv format (similar to the challenge description)?
Both formats will be accepted.

3. What is the rule of namming submission files?
I would like to compress all my submissions into a zip file as the follow. Is it OK?
MTab4Wikidata.zip ~ (Size: 9 -10MB)
- cea.csv
- cta.csv
- cpa.csv

You can submit a zip file, although I recommend to include the name of your system in the individual csv files too.

 Ernesto





On Tuesday, October 20, 2020 at 10:34:39 PM UTC+9 ernesto.ji...@gmail.com wrote:
Dear all

The tables and targets for Round 4 are available online:  www.cs.ox.ac.uk/isg/challenges/sem-tab/2020/

Note that the evaluation of this round will be BLIND and OFFLINE. That is, we will not be using AICrowd for the submission and evaluation. Participants should submit their solutions for the CEA, CTA and CPA tasks to https://tinyurl.com/semtab2020-round4

Deadline: October 27, 23:59 (AoE, Anywhere on Earth)

Results and award winners will be announced during the ISWC conference on November 5th.

Good luck!
Ernesto

--
You received this message because you are subscribed to a topic in the Google Groups "Sem-Tab Challenge" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sem-tab-challenge/7xeU0B_tF-Y/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sem-tab-challe...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/sem-tab-challenge/85b5efec-fcf0-4bec-8bbc-f610beda49dan%40googlegroups.com.


--
Ernesto Jiménez-Ruiz
Lecturer in Artificial Intelligence

Department of Computer Science
School of Mathematics, Computer Science and Engineering
City, University of London
T: +44 (0)20 7040 0212


Phuc

unread,
Oct 26, 2020, 11:16:04 AM10/26/20
to Sem-Tab Challenge
Dear Ernesto,

Thank you very much for your answers.

Best regards,

Phuc
Reply all
Reply to author
Forward
0 new messages