FIRE 2021: Call for Tracks
46 views
Skip to first unread message
Parth Mehta
Mar 21, 2021, 6:31:42 AM3/21/21
to fire...@googlegroups.com, si...@acm.org, cl...@mail.dei.unipd.it, ml-...@googlegroups.com, nt...@nii.ac.jp, clia, cor...@uib.no
Dear Colleagues,
We invite proposals for offering evaluation tracks at FIRE 2021.
FIRE 2021 is the 13th edition of the annual meeting of Forum for
Information Retrieval Evaluation (fire.irsi.res.in). Since its
inception in 2008, FIRE has had a strong focus on shared tasks,
similar to those offered at Evaluation forums like TREC, CLEF and
NTCIR. The shared tasks focus on solving specific problems in the area
information access and more importantly help in generating evaluation
datasets for the research community.
It is not required for the tasks to focus on a specific language, and
they can broadly cover any problem in the fields related to IR, NLP
and ML. However, the organizers especially encourage proposals for
tracks related to South Asian, African and middle Middle eastern
Eastern languages. In the past FIRE has hosted tracks from Arabic,
Persian, German, Russian and Urdu languages besides several Indian
languages. We aim to continue these efforts and include more language
groups from these regions. For knowing more about tracks in past FIRE
meetings, you can visit fire.irsi.res.in
Informal inquiries can also be sent to the track chairs.
Please include the following details in your proposal:
1. Track name
2. Track description
3. Broad categories of the track (eg: IR, NPL, ML, etc). More than one
categories can be mentioned.
3. Use case/s
4. Target Audience and number of expected submissions
5. Data(*) (fair details)
6. Evaluation plan
7. Timeline: Please try to align with the FIRE conference dates as given below
8. Organizer/s Details:
9. Prior experience in organizing shared task/workshop at relevant venues
*Tentative timeline*
- 3rd April – track proposals due
- 10th April – track acceptance notification due
- 30th April - open track websites and training data release
- 15th June – test data release
- 25th June – run submission deadline
- 15th July – results declared
- 31st August – Working notes due
- 15th Oct – Camera ready copies of working notes and overview paper due
- Tentatively in 1st or 2nd week of December - FIRE 2021
Please send this details in a pdf format to cl...@isical.ac.in with a
copy to parth.m...@gmail.com and ma...@uni-hildesheim.de
(*) Starting this year we require that after FIRE the data be made
publicly available through Information Retrieval Society of India. In
case of data that can not be distributed publicly (e.g. Twitter data)
a unique identifier that can be used to recreate the original corpus
can be provided (e.g. tweet ids in case of twitter data). This
disbursal will be governed by a copyright form, which the users have
to sign before getting the data. A sample form is available at
(http://fire.irsi.res.in/fire/static/data).
In case it is not preferable/possible for the track organizers to
share the data, please mention this in the proposal with specific
concerns. Exceptions can be made for tracks where data from industry
is used used or in case of other serious legal or ethical concerns.
The aim of organizing these tracks at FIRE is to have debates and
discussions on focused topics and give feedback to participants. As a
result, at least one of the track organizers from each track is
expected to attend FIRE and present the overview of track in person.
In case of non-attendance of any of the organizers, the team will not
be allowed to offer a track next year.
We will try to provide student volunteers for support in the proposed
tracks. They will basically be undergraduate students interested in IR
and related fields and can help with corpus creation, evaluation,
correspondence with participants, etc. If you require any such
support, kindly mention that in the track proposals along with the
number of students required.
Hoping to have an enthusiastic response from your end.
Regards,
Overall Track Coordinators, FIRE 2021
Parth Mehta (Parmonic, USA)
Thomas Mandl (University of Hildesheim, Germany)
We invite proposals for offering evaluation tracks at FIRE 2021.
FIRE 2021 is the 13th edition of the annual meeting of Forum for
Information Retrieval Evaluation (fire.irsi.res.in). Since its
inception in 2008, FIRE has had a strong focus on shared tasks,
similar to those offered at Evaluation forums like TREC, CLEF and
NTCIR. The shared tasks focus on solving specific problems in the area
information access and more importantly help in generating evaluation
datasets for the research community.
It is not required for the tasks to focus on a specific language, and
they can broadly cover any problem in the fields related to IR, NLP
and ML. However, the organizers especially encourage proposals for
tracks related to South Asian, African and middle Middle eastern
Eastern languages. In the past FIRE has hosted tracks from Arabic,
Persian, German, Russian and Urdu languages besides several Indian
languages. We aim to continue these efforts and include more language
groups from these regions. For knowing more about tracks in past FIRE
meetings, you can visit fire.irsi.res.in
Informal inquiries can also be sent to the track chairs.
Please include the following details in your proposal:
1. Track name
2. Track description
3. Broad categories of the track (eg: IR, NPL, ML, etc). More than one
categories can be mentioned.
3. Use case/s
4. Target Audience and number of expected submissions
5. Data(*) (fair details)
6. Evaluation plan
7. Timeline: Please try to align with the FIRE conference dates as given below
8. Organizer/s Details:
9. Prior experience in organizing shared task/workshop at relevant venues
*Tentative timeline*
- 3rd April – track proposals due
- 10th April – track acceptance notification due
- 30th April - open track websites and training data release
- 15th June – test data release
- 25th June – run submission deadline
- 15th July – results declared
- 31st August – Working notes due
- 15th Oct – Camera ready copies of working notes and overview paper due
- Tentatively in 1st or 2nd week of December - FIRE 2021
Please send this details in a pdf format to cl...@isical.ac.in with a
copy to parth.m...@gmail.com and ma...@uni-hildesheim.de
(*) Starting this year we require that after FIRE the data be made
publicly available through Information Retrieval Society of India. In
case of data that can not be distributed publicly (e.g. Twitter data)
a unique identifier that can be used to recreate the original corpus
can be provided (e.g. tweet ids in case of twitter data). This
disbursal will be governed by a copyright form, which the users have
to sign before getting the data. A sample form is available at
(http://fire.irsi.res.in/fire/static/data).
In case it is not preferable/possible for the track organizers to
share the data, please mention this in the proposal with specific
concerns. Exceptions can be made for tracks where data from industry
is used used or in case of other serious legal or ethical concerns.
The aim of organizing these tracks at FIRE is to have debates and
discussions on focused topics and give feedback to participants. As a
result, at least one of the track organizers from each track is
expected to attend FIRE and present the overview of track in person.
In case of non-attendance of any of the organizers, the team will not
be allowed to offer a track next year.
We will try to provide student volunteers for support in the proposed
tracks. They will basically be undergraduate students interested in IR
and related fields and can help with corpus creation, evaluation,
correspondence with participants, etc. If you require any such
support, kindly mention that in the track proposals along with the
number of students required.
Hoping to have an enthusiastic response from your end.
Regards,
Overall Track Coordinators, FIRE 2021
Parth Mehta (Parmonic, USA)
Thomas Mandl (University of Hildesheim, Germany)
Paolo Rosso
Jun 30, 2021, 12:28:41 PM6/30/21
to fire...@googlegroups.com
FYI
Regards
Paolo
-----
Paolo Rosso
Dpto. Sistemas Informáticos y Computación
Universitat Politècnica de València, Spain
URL: http://personales.upv.es/prosso/
----- Apologies for multiple postings -----
===================================
AUTOMATED DECISION MAKING -
AI systems and individual rights
-----------------------------------
https://digforasp.uca.es/admevent/
Online, July 15-16, 2021
===================================
This event is hosted by Eindhoven University of Technology,
Netherlands, as part of the activities of the COST Action DigForASP
(https://digforasp.uca.es/).
Link to register (no fees):
https://www.eventbrite.nl/e/automated-decision-making-ai-systems-and-individual-rights-registration-158893720395?aff=affiliate1
Description
Automated decision-making (ADM) systems, i.e., systems used to take a
decision by automated means without any human intervention, are
increasingly widespread. The development of new Artificial
Intelligence (AI) techniques is contributing in making those systems
even more common and accessible, and they are now integrated in
everyday life. Although they can lead to quicker and more consistent
decisions, by helping humans in reaching a better performance, there
may be significant risks for citizens' rights and freedom. Indeed,
they are often used to affect citizens' rights without them being
fully aware of the consequences on their lives, as in the case of
insurance contracts, verification of credit worthiness, hiring
processes, granting of social benefits, and many other tools. In some
cases, negative effects may occur even when used responsibly and
lawfully. In addition, the increasing availability of personal data on
the Internet and from interconnected devices has made the linking
between multiple sources easier, allowing the automated and large
scale analysis and prediction of individuals' preferences, habits,
personality, and behavior.
To address these problems, the European Commission recently published
a proposal for a new Regulation on AI, providing specific compliance
procedures for those systems that are considered more dangerous,
requiring humans to be aware that they are interacting with an AI, and
banning some technologies, such as facial recognition in public
accessible places.
This event aims to explore ethical, legal and technical issues of ADM
systems such as the impact of ADM on vulnerable groups, the need for
transparent and explainable AI, the consequences of ADM within the
trial and during investigations, and some recent cases of ADM causing
negative effects on individuals, for instance in the medical field.
Also, it will be discussed how the ADM topic is regulated by GDPR.
Speakers:
Stefania Costantini, University of L’Aquila, IT
Francisco de Elizalde, IE Law School, ES
César Hidalgo, University of Toulouse, FR
Alexandros Kalousis, University of Applied Sciences, CH
Angela Müller, Algorithm Watch
Vincent Müller, Eindhoven University of Technology, NL
Andrea Pin, University of Padua, IT
Marta Poblet Balcell, RMIT University, AU
Susana Vieira, University of Lisbon, PT
Sicco Verwer, TU Delft, NL
Agenda
Day 1 - Thursday 15 July 9:30 AM - 4:30 PM
Day 2 - Friday 16 July 9:15 AM - 12:45 PM
More information & detailed programme: https://digforasp.uca.es/admevent/
Organizers
Chiara Gallese Nobile, Eindhoven University of Technology, NL
David Billard, HES-SO, University of Applied Sciences in Geneva, CH
Elena Falletti, Carlo Cattaneo University LIUC, IT
Francesca A. Lisi, University of Bari, IT (Science Communication
Manager of COST Action DigForASP)
Regards
Paolo
-----
Paolo Rosso
Dpto. Sistemas Informáticos y Computación
Universitat Politècnica de València, Spain
URL: http://personales.upv.es/prosso/
----- Apologies for multiple postings -----
===================================
AUTOMATED DECISION MAKING -
AI systems and individual rights
-----------------------------------
https://digforasp.uca.es/admevent/
Online, July 15-16, 2021
===================================
This event is hosted by Eindhoven University of Technology,
Netherlands, as part of the activities of the COST Action DigForASP
(https://digforasp.uca.es/).
Link to register (no fees):
https://www.eventbrite.nl/e/automated-decision-making-ai-systems-and-individual-rights-registration-158893720395?aff=affiliate1
Description
Automated decision-making (ADM) systems, i.e., systems used to take a
decision by automated means without any human intervention, are
increasingly widespread. The development of new Artificial
Intelligence (AI) techniques is contributing in making those systems
even more common and accessible, and they are now integrated in
everyday life. Although they can lead to quicker and more consistent
decisions, by helping humans in reaching a better performance, there
may be significant risks for citizens' rights and freedom. Indeed,
they are often used to affect citizens' rights without them being
fully aware of the consequences on their lives, as in the case of
insurance contracts, verification of credit worthiness, hiring
processes, granting of social benefits, and many other tools. In some
cases, negative effects may occur even when used responsibly and
lawfully. In addition, the increasing availability of personal data on
the Internet and from interconnected devices has made the linking
between multiple sources easier, allowing the automated and large
scale analysis and prediction of individuals' preferences, habits,
personality, and behavior.
To address these problems, the European Commission recently published
a proposal for a new Regulation on AI, providing specific compliance
procedures for those systems that are considered more dangerous,
requiring humans to be aware that they are interacting with an AI, and
banning some technologies, such as facial recognition in public
accessible places.
This event aims to explore ethical, legal and technical issues of ADM
systems such as the impact of ADM on vulnerable groups, the need for
transparent and explainable AI, the consequences of ADM within the
trial and during investigations, and some recent cases of ADM causing
negative effects on individuals, for instance in the medical field.
Also, it will be discussed how the ADM topic is regulated by GDPR.
Speakers:
Stefania Costantini, University of L’Aquila, IT
Francisco de Elizalde, IE Law School, ES
César Hidalgo, University of Toulouse, FR
Alexandros Kalousis, University of Applied Sciences, CH
Angela Müller, Algorithm Watch
Vincent Müller, Eindhoven University of Technology, NL
Andrea Pin, University of Padua, IT
Marta Poblet Balcell, RMIT University, AU
Susana Vieira, University of Lisbon, PT
Sicco Verwer, TU Delft, NL
Agenda
Day 1 - Thursday 15 July 9:30 AM - 4:30 PM
Day 2 - Friday 16 July 9:15 AM - 12:45 PM
More information & detailed programme: https://digforasp.uca.es/admevent/
Organizers
Chiara Gallese Nobile, Eindhoven University of Technology, NL
David Billard, HES-SO, University of Applied Sciences in Geneva, CH
Elena Falletti, Carlo Cattaneo University LIUC, IT
Francesca A. Lisi, University of Bari, IT (Science Communication
Manager of COST Action DigForASP)
Paolo Rosso
Nov 24, 2021, 2:40:53 AM11/24/21
to fire...@googlegroups.com
Apologies for multiple copies...
#good-start #irony
Paolo
-----
Apologies for multiple copies
1st CALL FOR PAPERS
27th International Conference on Natural Language & Information
Systems (NLDB 2022)
June 15-17, 2022, Valencia, Spain
http://nldb2022.prhlt.upv.es/
NLDB 2022 invites authors to submit papers for oral or poster
presentations on unpublished research that addresses theoretical
aspects, algorithms, applications, architectures for applied and
integrated NLP, resources for applied NLP, and other aspects of NLP,
as well as survey and discussion papers.
CONFERENCE TOPICS (not limited to)
Social Media and Web Analytics
Deep Learning and eXplainable Artificial Intelligence
Argumentation Mining and Applications
Question Answering
Corpus Analysis
Semantic Web, Open Linked Data, and Ontologies
Natural Language in Conceptual Modeling
Natural language and Ubiquitous Computing
Big Data and Business Intelligence
SUBMISSION AND DATES
Authors should follow the LNCS format: submissions can be full papers
(12 pages maximum including references), short papers (8 pages
including references) or papers for a poster presentation (6 pages
including references).
Paper submission: 28 February 2022
Notification of acceptance: 1 April 2022
Camera-ready papers: 12 April 2022
Conference: 15-17 June 2022
KEYNOTES
Eneko Agirre, University of the Basque Country / HiTZ Centre on
Language Technology
Lucie Flek, University of Marburg
INDUSTRY KEYNOTE
Ramit Sawhney, Lead AI Scientist at ShareChat
PROGRAM CHAIRS
Valerio Basile, University of Turin, Italy
Raquel Martínez, UNED, Spain
PROCEEDINGS
The Proceedings of NLDB 2022 will be published as part of the Lecture
Notes in Computer Science book serie.
As in the previous editions, we plan to publish the extended version
of the best papers in the Data & Knowledge Engineering Journal
(tentative).
NLDB 2022 will be held at the Universitat Politècnica de València,
Spain. Although we plan to organize the conference in an hybrid mode,
we sincerely hope to see you in Valencia!
Paolo Rosso, Elisabeth Métais, Farid Meziane
NLDB-2022 general chairs
Contact: nldb...@gmail.com
#good-start #irony
Paolo
-----
Apologies for multiple copies
1st CALL FOR PAPERS
27th International Conference on Natural Language & Information
Systems (NLDB 2022)
June 15-17, 2022, Valencia, Spain
http://nldb2022.prhlt.upv.es/
NLDB 2022 invites authors to submit papers for oral or poster
presentations on unpublished research that addresses theoretical
aspects, algorithms, applications, architectures for applied and
integrated NLP, resources for applied NLP, and other aspects of NLP,
as well as survey and discussion papers.
CONFERENCE TOPICS (not limited to)
Social Media and Web Analytics
Deep Learning and eXplainable Artificial Intelligence
Argumentation Mining and Applications
Question Answering
Corpus Analysis
Semantic Web, Open Linked Data, and Ontologies
Natural Language in Conceptual Modeling
Natural language and Ubiquitous Computing
Big Data and Business Intelligence
SUBMISSION AND DATES
Authors should follow the LNCS format: submissions can be full papers
(12 pages maximum including references), short papers (8 pages
including references) or papers for a poster presentation (6 pages
including references).
Paper submission: 28 February 2022
Notification of acceptance: 1 April 2022
Camera-ready papers: 12 April 2022
Conference: 15-17 June 2022
KEYNOTES
Eneko Agirre, University of the Basque Country / HiTZ Centre on
Language Technology
Lucie Flek, University of Marburg
INDUSTRY KEYNOTE
Ramit Sawhney, Lead AI Scientist at ShareChat
PROGRAM CHAIRS
Valerio Basile, University of Turin, Italy
Raquel Martínez, UNED, Spain
PROCEEDINGS
The Proceedings of NLDB 2022 will be published as part of the Lecture
Notes in Computer Science book serie.
As in the previous editions, we plan to publish the extended version
of the best papers in the Data & Knowledge Engineering Journal
(tentative).
NLDB 2022 will be held at the Universitat Politècnica de València,
Spain. Although we plan to organize the conference in an hybrid mode,
we sincerely hope to see you in Valencia!
Paolo Rosso, Elisabeth Métais, Farid Meziane
NLDB-2022 general chairs
Contact: nldb...@gmail.com
Paolo Rosso
Jan 14, 2022, 7:51:23 AM1/14/22
to fire...@googlegroups.com
Apologies for multiple copies
2nd CALL FOR PAPERS
27th International Conference on Natural Language & Information
Systems (NLDB 2022)
June 15-17, 2022, Valencia, Spain
http://nldb2022.prhlt.upv.es/
NLDB 2022 invites authors to submit papers for oral or poster
presentations on unpublished research that addresses theoretical
aspects, algorithms, applications, architectures for applied and
integrated NLP, resources for applied NLP, and other aspects of NLP,
as well as survey and discussion papers.
This year's edition of NLDB also introduces an Industry Track, to
foster fruitful interaction between the industry and the research
community. Papers can be submitted to either the main conference or
the industry track.
CONFERENCE TOPICS (not limited to)
Social Media and Web Analytics
Deep Learning and eXplainable Artificial Intelligence
Argumentation Mining and Applications
Question Answering
Corpus Analysis
Semantic Web, Open Linked Data, and Ontologies
Natural Language in Conceptual Modeling
Natural language and Ubiquitous Computing
Big Data and Business Intelligence
SUBMISSION AND DATES
Authors should follow the LNCS format: submissions can be full papers
(12 pages maximum including references), short papers (8 pages
including references) or papers for a poster presentation (6 pages
including references).
Paper submission: 28 February 2022
Notification of acceptance: 1 April 2022
Camera-ready papers: 12 April 2022
Conference: 15-17 June 2022
KEYNOTES
Eneko Agirre, University of the Basque Country / HiTZ Centre on
Language Technology
Lucie Flek, University of Marburg
INDUSTRY KEYNOTE
Ramit Sawhney, Tower Research Capital
2nd CALL FOR PAPERS
27th International Conference on Natural Language & Information
Systems (NLDB 2022)
June 15-17, 2022, Valencia, Spain
http://nldb2022.prhlt.upv.es/
NLDB 2022 invites authors to submit papers for oral or poster
presentations on unpublished research that addresses theoretical
aspects, algorithms, applications, architectures for applied and
integrated NLP, resources for applied NLP, and other aspects of NLP,
as well as survey and discussion papers.
foster fruitful interaction between the industry and the research
community. Papers can be submitted to either the main conference or
the industry track.
CONFERENCE TOPICS (not limited to)
Social Media and Web Analytics
Deep Learning and eXplainable Artificial Intelligence
Argumentation Mining and Applications
Question Answering
Corpus Analysis
Semantic Web, Open Linked Data, and Ontologies
Natural Language in Conceptual Modeling
Natural language and Ubiquitous Computing
Big Data and Business Intelligence
SUBMISSION AND DATES
Authors should follow the LNCS format: submissions can be full papers
(12 pages maximum including references), short papers (8 pages
including references) or papers for a poster presentation (6 pages
including references).
Paper submission: 28 February 2022
Notification of acceptance: 1 April 2022
Camera-ready papers: 12 April 2022
Conference: 15-17 June 2022
KEYNOTES
Eneko Agirre, University of the Basque Country / HiTZ Centre on
Language Technology
Lucie Flek, University of Marburg
INDUSTRY KEYNOTE
Paolo Rosso
Feb 18, 2022, 5:40:15 AM2/18/22
to fire...@googlegroups.com
Regards
Paolo
-----
Please consider participating and/or forwarding to appropriate
colleagues and groups.
*****We apologize for the multiple copies of this e-mail*****
CALL FOR PARTICIPATION
IberLEF 2022 Task DETEST: DETEction and classification of racial
Stereotypes in Spanish
This task will take part of IberLEF 2022, the 4th Workshop on Iberian
Languages Evaluation Forum at the SEPLN 2022 Conference, which will be
held in A Coruña, Spain, on September 20th.
The aim of the task is to detect and classify stereotypes in comments
posted in Spanish in response to different online news articles
related to immigration. The task is designed in a hierarchical fashion
by chaining two subtasks and allowing participants to either model the
simple binary scenario (a stereotype is present or not) or complete
the entire pipeline by modeling the complex multi-label classification
problem (different types of stereotypes). Next, a description of both
subtasks is provided:
Subtask 1: Participants tackling this problem will have to determine
whether the comment contains at least one stereotype (positive
example) or none (negative example) considering the full distribution
of labels provided by the annotators based on the proposal of learning
with disagreements. The actual gold label of this subtask is left as a
proxy to determine the subset of comments that will be evaluated in
the posterior subtask.
Subtask 2: This subtask consists of determining whether the comment
contains at least one stereotype or none and assigning those comments
previously marked as positive (with stereotypes) to ten categories
that present immigrants as: 1) ‘victims of xenophobia’, 2) ‘suffering
victims’, 3) ‘economic resources’, 4) a problem of ‘migration
control’, 5) people with ‘cultural and religious differences’, 6)
people which take ‘benefits’ of our social policy, 7) a problem for
‘public health’, 8) a threat to ‘security’, 9) ‘dehumanization’ and
10) ‘other’ types of stereotypes. Since a comment can contain multiple
stereotypes belonging to different categories, this subtask will be
presented as a multi-label hierarchical classification problem.
Although we recommend participating in both subtasks, participants are
allowed to participate just in one of them (e.g., subtask 1).
Teams will be allowed (and encouraged) to submit multiple runs (max. 5).
The present task is proposed to participants interested in racial,
national, or ethnic stereotype detection and classification tasks,
which is a relevant and relatively novel area of research due to its
impact on modern society. Furthermore, the annotated dataset is a
valuable resource for exploratory linguistic analysis, as well as for
comparing the application of deep learning and classical machine
learning models on Spanish stereotyped expressions under the recently
introduced learning with disagreements paradigm. Participants will be
provided with the annotated data by each of the annotators and the
gold standard.
Linguistic resources:
Our DETESTS corpus is made up of two parts – a subset of the
NewsCom-TOX corpus and the StereoCom corpus. Both corpora consist of
comments (at least 50) published in response to manually selected
articles extracted from Spanish online newspapers. The common topic of
all articles is immigration.
The DETESTS corpus consists of 5,628 sentences. We will provide
participants with 70% of the dataset to train their models, while the
remaining 30% will be used to test their models.
To avoid any conflict with the sources of the comments regarding their
intellectual property rights (IPR), the data will be sent privately to
each participant who is interested in the task. The corpus will only
be made available for research purposes.
Important dates (All deadlines are 11:59 PM UTC-12:00):
Training dataset release: March 21, 2022
Test dataset release: April 20, 2022
Systems results: May 16, 2022
Results notification: May 23, 2022
Working papers submission: June 9, 2022
Working papers (peer-)reviewed: June 20, 2022
Camera-ready versions: July 4, 2022
Workshop at IberLEF 2022: September 20, 2022
Task organizers:
Mariona Taulé (Universitat de Barcelona, UB)
Montserrat Nofre (Universitat de Barcelona, UB)
Alejandro Ariza (Universitat de Barcelona, UB)
Wolfgang Schmeisser (Universitat de Barcelona, UB)
Enrique Amigó (Universidad Nacional de Educación a Distancia, UNED)
Paolo Rosso (Universitat Politècnica de València, UPV)
Berta Chulvi (Universitat Politècnica de València, UPV)
Contact:
Contact the organizers by writing to detests...@gmail.com.
We invite participants to join our Google Groups to be kept up to date
with the latest news related to the task.
For more information, please visit our website
detestsiberlef.wixsite.com/detests.
Paolo
-----
Please consider participating and/or forwarding to appropriate
colleagues and groups.
*****We apologize for the multiple copies of this e-mail*****
CALL FOR PARTICIPATION
IberLEF 2022 Task DETEST: DETEction and classification of racial
Stereotypes in Spanish
This task will take part of IberLEF 2022, the 4th Workshop on Iberian
Languages Evaluation Forum at the SEPLN 2022 Conference, which will be
held in A Coruña, Spain, on September 20th.
The aim of the task is to detect and classify stereotypes in comments
posted in Spanish in response to different online news articles
related to immigration. The task is designed in a hierarchical fashion
by chaining two subtasks and allowing participants to either model the
simple binary scenario (a stereotype is present or not) or complete
the entire pipeline by modeling the complex multi-label classification
problem (different types of stereotypes). Next, a description of both
subtasks is provided:
Subtask 1: Participants tackling this problem will have to determine
whether the comment contains at least one stereotype (positive
example) or none (negative example) considering the full distribution
of labels provided by the annotators based on the proposal of learning
with disagreements. The actual gold label of this subtask is left as a
proxy to determine the subset of comments that will be evaluated in
the posterior subtask.
Subtask 2: This subtask consists of determining whether the comment
contains at least one stereotype or none and assigning those comments
previously marked as positive (with stereotypes) to ten categories
that present immigrants as: 1) ‘victims of xenophobia’, 2) ‘suffering
victims’, 3) ‘economic resources’, 4) a problem of ‘migration
control’, 5) people with ‘cultural and religious differences’, 6)
people which take ‘benefits’ of our social policy, 7) a problem for
‘public health’, 8) a threat to ‘security’, 9) ‘dehumanization’ and
10) ‘other’ types of stereotypes. Since a comment can contain multiple
stereotypes belonging to different categories, this subtask will be
presented as a multi-label hierarchical classification problem.
Although we recommend participating in both subtasks, participants are
allowed to participate just in one of them (e.g., subtask 1).
Teams will be allowed (and encouraged) to submit multiple runs (max. 5).
The present task is proposed to participants interested in racial,
national, or ethnic stereotype detection and classification tasks,
which is a relevant and relatively novel area of research due to its
impact on modern society. Furthermore, the annotated dataset is a
valuable resource for exploratory linguistic analysis, as well as for
comparing the application of deep learning and classical machine
learning models on Spanish stereotyped expressions under the recently
introduced learning with disagreements paradigm. Participants will be
provided with the annotated data by each of the annotators and the
gold standard.
Linguistic resources:
Our DETESTS corpus is made up of two parts – a subset of the
NewsCom-TOX corpus and the StereoCom corpus. Both corpora consist of
comments (at least 50) published in response to manually selected
articles extracted from Spanish online newspapers. The common topic of
all articles is immigration.
The DETESTS corpus consists of 5,628 sentences. We will provide
participants with 70% of the dataset to train their models, while the
remaining 30% will be used to test their models.
To avoid any conflict with the sources of the comments regarding their
intellectual property rights (IPR), the data will be sent privately to
each participant who is interested in the task. The corpus will only
be made available for research purposes.
Important dates (All deadlines are 11:59 PM UTC-12:00):
Training dataset release: March 21, 2022
Test dataset release: April 20, 2022
Systems results: May 16, 2022
Results notification: May 23, 2022
Working papers submission: June 9, 2022
Working papers (peer-)reviewed: June 20, 2022
Camera-ready versions: July 4, 2022
Workshop at IberLEF 2022: September 20, 2022
Task organizers:
Mariona Taulé (Universitat de Barcelona, UB)
Montserrat Nofre (Universitat de Barcelona, UB)
Alejandro Ariza (Universitat de Barcelona, UB)
Wolfgang Schmeisser (Universitat de Barcelona, UB)
Enrique Amigó (Universidad Nacional de Educación a Distancia, UNED)
Paolo Rosso (Universitat Politècnica de València, UPV)
Berta Chulvi (Universitat Politècnica de València, UPV)
Contact:
Contact the organizers by writing to detests...@gmail.com.
We invite participants to join our Google Groups to be kept up to date
with the latest news related to the task.
For more information, please visit our website
detestsiberlef.wixsite.com/detests.
Paolo Rosso
Feb 25, 2022, 5:02:57 AM2/25/22
to fire...@googlegroups.com
Apologies for multiple copies
Paper submission (EXTENDED): 14 March 2022
Notification of acceptance (extended): 10 April 2022
Camera-ready papers (extended): 19 April 2022
Notification of acceptance (extended): 10 April 2022
Camera-ready papers (extended): 19 April 2022
Conference: 15-17 June 2022
KEYNOTES
Eneko Agirre, University of the Basque Country / HiTZ Centre on
Language Technology
Lucie Flek, University of Marburg
INDUSTRY KEYNOTE
KEYNOTES
Eneko Agirre, University of the Basque Country / HiTZ Centre on
Language Technology
Lucie Flek, University of Marburg
INDUSTRY KEYNOTE
Ramit Sawhney, Lead AI Scientist at ShareChat
Reply all
Reply to author
Forward
0 new messages