dataset release

[Johannes Kiesel]

Hi all,

We are happy to announce the release of the competition dataset. We
extended the previous dataset considerably, from about 5000 to nearly
9000 arguments.

As usual, you can find all the relevant links on the task's web page:

https://valueeval.webis.de

We uploaded the dataset files both to Zenodo (so you can download
everything except the test labels) and to TIRA (so you can directly
submit your approaches as Docker images or upload your runs)!

As announced, we have a training, validation, and test split (which we
changed slightly to 60%/20%/20%). We decided to move the arguments from
the Chinese Q&A site Zhihu (from the original dataset) into an own
validation set. Now the training, validation, and test dataset are
roughly composed of
- 80% from IBM argument quality dataset (95% in original dataset),
- 15% from the Conference for the Future of Europe (new), and
- 5% from group discussion ideas (2% in original dataset).
The additional test dataset from religious texts, promised in the last
mail, will be added in the next few days.


EARLY BIRD (Deadline December 16th)
===================================
Though the final submission deadline is January 24th, you can already
submit one run (either as Docker image or upload) for the test data by
December 16th and get it scored. This way both you and us can test the
submission system. See the submission instructions on our web page:

https://touche.webis.de/semeval23/touche23-web/index.html#submission

However, note that you can also submit an unlimited number of runs for
the training and validation dataset in TIRA. Submission for the test
data is similar, but you will not directly see the results for test.

As always, please ask in case of questions!
- https://www.tira.io/c/touche/9 (TIRA forum)
- mailto:valu...@googlegroups.com (this list)
- mailto:valueeval-...@googlegroups.com (private mail to us)


Looking forward to your submissions!
Johannes, Milad, Nailia, Maximilian, Henning, and Benno

--
Johannes Kiesel

Bauhaus-Universität Weimar
Bauhausstr. 9a, Room 106
99423 Weimar, Germany

Phone: +49 (0)3643 - 58 3720