SemTab 2021 Round 1: DBpedia snapshot, CPA task and leaderboard

[Raphaël Troncy]

 Dear Enernesto, all SemTab organizers,

We have a few questions:

1) You're indicating that the reference Wikidata Knowledge Graph is the snapshot of 28/06/2021, https://dumps.wikimedia.org/wikidatawiki/entities/20210628/. What is the reference DBPedia Knowledge Graph snapshot? Can you please provide a similar snapshot date / download link as this can be a moving target?

2) A CPA task is mentioned at http://www.cs.ox.ac.uk/isg/challenges/sem-tab/2021/index.html, but no AICrowd tasks for CPA have been created. Is this an oversight? Is there a CPA task for the round 1?

3) Last year, we had a discussion on whether we should have a public or a private leaderboard. Having a public one, combined with a unlimited number of submissions, enables to cheat in the sense that one can submit gradually table diff until reaching the perfect score. Will you penalize the teams that abuse in term of numbers of submissions? Will you introduce a private leaderboard in subsequent rounds?

For this first round, is it true that the tables are a subset of last year Tough Tables part of round 4 as Phuc mentioned?
Best regards.

  Raphaël   

[Jiaoyan Chen]

Dear Raphael,

Here are answers to some of your questions.

Regarding the DBpedia version, it is 2016-10 (http://downloads.dbpedia.org/wiki-archive/downloads-2016-10.html). It’s pointed out in the challenge pages.

In Round #1, we do not consider CPA, but we are considering it for the following rounds. Making new benchmarks for CPA costs additional time. 

The challenges in Round #1 is based on Tough Tables, but we have made a bit change (e.g., making the GT be consistent to the Wikidata version). 

Regards,

Jiaoyan

--
You received this message because you are subscribed to the Google Groups "Sem-Tab Challenge" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sem-tab-challe...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/sem-tab-challenge/cb80b463-010c-4221-93ef-9289dfe5ceccn%40googlegroups.com.