CFP: 11th Competition on Legal Information Extraction/Entailment (COLIEE) 2024

[佐藤健]

==================== CALL FOR PARTICIPATION ======================
11th Competition on Legal Information Extraction/Entailment (COLIEE) 2024,
in association with the International Workshop on Juris-informatics (JURISIN) 2024

http://www.ualberta.ca/~rabelo/COLIEE2024/
http://research.nii.ac.jp/~ksatoh/jurisin2024/

May 28-31, 2024 - ACT CITY Hamamatsu, Hamamatsu, Shizuoka, Japan

As an associated event of JURISIN 2024, we are happy to announce the 11th Competition on Legal Information Extraction and Entailment (COLIEE-2024), which will include tasks on both statute law and case law.

Four tasks are included in the 2024 edition: Tasks 1 and 2 are about case law, and tasks 3 and 4 are on statute law.
* Task 1 is a legal case retrieval task, and it involves reading a new case Q, and extracting supporting cases S1, S2..., Sn from the provided case law corpus, to support the decision for Q.
* Task 2 is the legal case entailment task, which involves the identification of one or more paragraphs from an existing case that entail the decision of a new case.
* As in previous COLIEE competitions, Task 3 is to consider a yes/no legal question Q and retrieve relevant statutes from a database of Japanese civil code statutes.
* Task 4 is to confirm entailment of a yes/no answer from the retrieved civil code statutes.

Further details can be found in the following URL: http://www.ualberta.ca/~rabelo/COLIEE2024/

The datasets for Task 1 and Task 2 are in English, and the datasets for Task 3 and Task 4 are available both in Japanese and English (translation from Japanese bar exams and civil code). You can choose any (or both) language datasets and any task for participation.

The intention is to build a community of practice regarding legal information processing and textual entailment, so that the adoption and adaptation of general methods from a variety of fields is considered, and that participants share their approaches, problems, and results.

Participants should clearly mention what dataset was used (for example, pretrained by Wikipedia dump data as of 2022xxxx, fine-tuned by ...) for reproducibility purposes. Participants can use any external data, but it is assumed that they do not use the test dataset and/or something which could directly contain the correct answers of the test dataset, because this task is intended to be a pure textual entailment task.

We require participants to submit a paper on their method and experimental results to JURISIN 2024 in accordance with the instructions specified at http://research.nii.ac.jp/~ksatoh/jurisin2024/ (section "Submissions") and to present the paper in person at the COLIEE workshop if accepted by a peer review.

Upon paper submission, participants are also required to provide a brief explanation of their method. Notice this is not a paper abstract, it should be a description of the method(s) employed following the structure below:
TeamID:
Brief explanation of the method:
Resources used in the submission:
For example:
- models, such as BERT (please include model details),
- software, such as Elastic Search
RunID specific settings:
If you submit two or more results, please explain the difference among them.
The form above must be sent to coliee_par...@nii.ac.jp.

If you want to see actual examples of such descriptions, please check section 5.3 of the COLIEE 2023 summary paper at "https://sites.ualberta.ca/~rabelo/COLIEE2023/COLIEE2023_summary.pdf".
Insufficient sdata may mean a submission will not be included in the overall completion summary.
======================================================================
Important dates:
01 Nov, 2023 Training data release.
13 Dec, 2023 Test data release
31 Dec, 2023 Registration deadline for COLIEE
31 Dec, 2023 Submission deadline for Task 3 test runs.
07 Jan, 2024 Announcements of rankings/ assessments for task 3 and announcements of answers
(relevant article(s) for each question) for Task 4.
17 Jan, 2024 Submission deadline for Tasks 1, 2 and 4 test runs.
20 Jan, 2024 Announcements of rankings/assessments for Tasks 1, 2, 4.
31 Jan, 2024 Paper submission deadline for the COLIEE workshop.
29 Feb, 2024 Notification of acceptance for the COLIEE workshop paper.
25 March, 2024 Camera-ready copy deadline.
28-31 May, 2024 JURISIN 2024 (one day for the COLIEE workshop - TBD).
======================================================================

Application Details:
Potential participants to COLLIE-2024 should respond to this call for participation by submitting an application.
To apply, submit the application and memorandums of the following URL to coliee_par...@nii.ac.jp .

Application:
http://www.ualberta.ca/~rabelo/COLIEE2024/application.pdf
Memorandum for Tasks 1 and/or 2 (Case law competition)
http://www.ualberta.ca/~rabelo/COLIEE2024/CL_memorandum.pdf
Memorandum for Tasks 3 and/or 4 (Statute law competition, English Data)
http://www.ualberta.ca/~rabelo/COLIEE2024/SL_EN_memorandum.pdf
Memorandum for Tasks 3 and/or 4 (Statute law competition, Japanese Data)
http://www.ualberta.ca/~rabelo/COLIEE2024/SL_JA_memorandum.pdf
An acknowledgement note will be sent to the email address supplied in the form once the form has been processed.

Task coordinators:
Mi-Young Kim, University of Alberta, Canada
Randy Goebel, University of Alberta, Canada
Juliano Rabelo, University of Alberta, Canada
Ken Satoh, National Institute of Informatics, Japan
Yoshinobu Kano, Shizuoka University, Japan
Masaharu Yoshioka, Hokkaido University, Japan

Questions and further information:
rabelo(at)ualberta.ca