FIRE 2024: Call for Track Proposals

[Bhargav Dave]

Dear Colleagues, 

Call for Tracks for FIRE 2024 is out now.

We are reaching out to you because you have been supportive of FIRE
and have organized one or more tracks in the past. Please consider
offering a track again this year and also help us spread the word
among other groups that might be interested.

We look forward to your continued support :-)

------

We invite proposals for offering evaluation tracks at FIRE 2024.

FIRE 2024 is the 16th edition of the annual meeting of Forum for
Information Retrieval Evaluation (fire.irsi.res.in). Since its
inception in 2008, FIRE has had a strong focus on shared tasks,
similar to those offered at Evaluation forums like TREC, CLEF and
NTCIR. The shared tasks focus on solving specific problems in the area
of information access and more importantly help in generating evaluation
datasets for the research community.

It is not required for the tasks to focus on a specific language, and
they can broadly cover any problem in the fields related to IR, NLP
and ML. However, the organizers especially encourage proposals for
tracks related to South Asian, African and middle Middle eastern
Eastern languages. In the past FIRE has hosted tracks from Arabic,
Persian, German, Russian and Urdu languages besides several Indian
languages. We aim to continue these efforts and include more language
groups from these regions. To know more about tracks in past FIRE
meetings, you can visit fire.irsi.res.in

Informal inquiries can also be sent to the track chairs.

Please include the following details in your proposal:

1. Track name
2. Track description
3. Broad categories of the track (eg: IR, NPL, ML, etc). More than one
category can be mentioned.
3. Use case/s
4. Target Audience and number of expected submissions
5. Data(*) (fair details)
6. Evaluation plan
7. Timeline: Please try to align with the FIRE conference dates as given below
8. Organizer/s Details:
9. Prior experience in organizing shared task/workshop at relevant venues

*Tentative timeline*

- 8th May – track proposals due
- 12th May – track acceptance notification due
- 25th May - open track websites and training data release
- 10th July – test data release
- 1st August – run submission deadline
- 15th August – results declared
- 15th September – Working notes due
- 15th Oct – Camera-ready copies of working notes and overview paper due
- Tentatively in December 2024

Please send these details in a PDF format to cl...@isical.ac.in with a
copy to kripa...@gmail.com and ma...@uni-hildesheim.de

(*) We require that after FIRE the data be made
publicly available through Information Retrieval Society of India. In
case of data that can not be distributed publicly (e.g. Twitter data)
a unique identifier that can be used to recreate the original corpus
can be provided (e.g. tweet ids in case of twitter data). This
disbursal will be governed by a copyright form, which the users have
to sign before getting the data. A sample form is available at
(http://fire.irsi.res.in/fire/static/data).

In case it is not preferable/possible for the track organizers to
share the data, please mention this in the proposal with specific
concerns. Exceptions can be made for tracks where data from industry
is used or in case of other serious legal or ethical concerns.

The aim of organizing these tracks at FIRE is to have debates and
discussions on focused topics and give feedback to participants. As a
result, at least one of the track organizers from each track is
expected to attend FIRE and present the overview of track in person.
In case of non-attendance of any of the organizers, the team will not
be allowed to offer a track next year. 

We will try to provide student volunteers for support in the proposed
tracks. They will basically be undergraduate students interested in IR
and related fields and can help with corpus creation, evaluation,
correspondence with participants, etc. If you require any such
support, kindly mention that in the track proposals along with the
number of students required.

Hoping to have an enthusiastic response from your end.

Regards,
Overall Track Coordinators, FIRE 2024
Thomas Mandl & Kripabandhu Ghosh