2nd CFP: 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA2023)

127 views
Skip to first unread message

Ekaterina Kochmar

unread,
Mar 10, 2023, 4:06:43 AM3/10/23
to ml-...@googlegroups.com

Second Call for Papers

The 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023)

Toronto

Thursday, July 13, 2023

(co-located with ACL 2023)

https://sig-edu.org/bea/current

 Submission Deadline: Monday, April 24, 2023, 11:59pm UTC-12

WORKSHOP DESCRIPTION

The BEA Workshop is a leading venue for NLP innovation in the context of educational applications. It is one of the largest one-day workshops in the ACL community with over 100 registered attendees in the past several years. The growing interest in educational applications and a diverse community of researchers involved resulted in the creation of the Special Interest Group in Educational Applications (SIGEDU) in 2017, which currently has over 300 members.

 

The workshop’s continuing growth highlights the alignment between societal needs and technological advances: for instance, BEA16 in 2021 hosted a panel discussion on New Challenges for Educational Technology in the Time of the Pandemic addressing the pressing issues around COVID19. NLP capabilities can now support an array of learning domains, including writing, speaking, reading, science, and mathematics, as well as the related intra-personal (e.g., self-confidence) and inter-personal (e.g., peer collaboration) skills. Within these areas, the community continues to develop and deploy innovative NLP approaches for use in educational settings. Another breakthrough for educational applications within the CL community is the presence of a number of shared-task competitions organized by the BEA workshop over the past several years, including four shared tasks on grammatical error detection and correction alone. NLP/Education shared tasks have also seen new areas of research, such as the Automated Evaluation of Scientific Writing at BEA11, Native Language Identification at BEA12, Second Language Acquisition Modelling at BEA13, and Complex Word Identification at BEA13. These competitions increased the visibility of, and interest in, our field.

 

The 18th BEA workshop will have keynotes by Susan Lottridge (Cambium Assessment) and Jordana Heller (Textio), an invited paper presentation by a member of one of the educational societies from the International Alliance to Advance Learning in the Digital Era (IAALDE), oral presentation sessions, and a large poster session to maximize the amount of original work presented. We expect that the workshop will continue to highlight novel technologies and opportunities for educational NLP in English as well as other languages. The workshop will solicit both full papers and short papers for either oral or poster presentation.

 

We will solicit papers that incorporate NLP methods, including, but not limited to:

 

  • automated scoring of open-ended textual and spoken responses;

  • automated scoring/evaluation for written student responses (across multiple genres);

  • game-based instruction and assessment;

  • educational data mining;

  • intelligent tutoring;

  • collaborative learning environments;

  • peer review;

  • grammatical error detection and correction;

  • learner cognition;

  • spoken dialog;

  • multimodal applications;

  • annotation standards and schemas;

  • tools and applications for classroom teachers, learners and/or test developers; and

  • use of corpora in educational tools.

 

INVITED TALKS

  

The workshop will feature invited talks from Susan Lottridge (Cambium Assessment) and Jordana Heller (Textio), as well as a speaker from one of the IAALDE societies.


IMPORTANT DATES

All deadlines are 11:59 pm UTC-12 (anywhere on earth).

  • Anonymity Period Begins: Friday, March 24, 2023

  • Submission Deadline: Monday, April 24, 2023

  • Notification of Acceptance: Monday, May 22, 2023

  • Camera-ready Papers Due: Tuesday, May 30, 2023

  • Workshop: Thursday, July 13, 2023

 

SUBMISSION INFORMATION

We will be using the ACL Submission Guidelines for the BEA Workshop this year. Authors are invited to submit a long paper of up to eight (8) pages of content, plus unlimited references; final versions of long papers will be given one additional page of content (up to 9 pages) so that reviewers’ comments can be taken into account. We also invite short papers of up to four (4) pages of content, plus unlimited references. Upon acceptance, short papers will be given five (5) content pages in the proceedings. Authors are encouraged to use this additional page to address reviewers’ comments in their final versions. Papers which describe systems are also invited to give a demo of their system. If you would like to present a demo in addition to presenting the paper, please make sure to select either “long paper + demo” or “short paper + demo” under “Submission Category” in the START submission page.

Previously published papers cannot be accepted. The submissions will be reviewed by the program committee. As reviewing will be blind, please ensure that papers are anonymous. Self-references that reveal the author’s identity, e.g., “We previously showed (Smith, 1991) …”, should be avoided. Instead, use citations such as “Smith previously showed (Smith, 1991) …”.

We have also included conflict of interest in the submission form. You should mark all potential reviewers who have been authors on the paper, are from the same research group or institution, or who have seen versions of this paper or discussed it with you.

We will be using the START conference system to manage submissions: https://www.softconf.com/acl2023/bea2023/ 

DOUBLE SUBMISSION POLICY

We will follow the official ACL double-submission policy. Specifically:

Papers being submitted both to BEA and another conference or workshop must:

●       Note on the title page the other conference or workshop to which they are being submitted.

●       State on the title page that if the authors choose to present their paper at BEA (assuming it was accepted), then the paper will be withdrawn from other conferences and workshops.

ORGANIZING COMMITTEE

 

Workshop contact email address: bea.nlp....@gmail.com

 

PROGRAM COMMITTEE

  • Rania Abdelghani, INRIA Bordeaux - EvidenceB Paris
  • Tazin Afrin, Educational Testing Services
  • David Alfter, Université catholique de Louvain
  • Erfan Al-Hossami, University of North Carolina at Charlotte
  • Michael Gringo Angelo Bayona, Trinity College Dublin
  • Lee Becker, Pearson
  • Marie Bexte, FernUniversität in Hagen
  • Serge Bibauw, Universidad Central del Ecuador; KU Leuven
  • Daniel Brenner, ETS
  • Dominique Brunato, Institute of Computational Linguistics / CNR (Pisa, Italy)
  • Christopher Bryant, University of Cambridge
  • Dumitru-Clementin Cercel, University Politehnica of Bucharest
  • Li-Hsin Chang, University of Turku
  • Guanliang Chen, Monash University
  • Hyundong Cho, USC ISI
  • Martin Chodorow, The City University of New York
  • Mark Core, University of Southern California
  • Kordula De Kuthy, University of Tübingen
  • Jasper Degraeuwe, Ghent University
  • Carrie Demmans Epp, University of Alberta
  • Yuning Ding, FernUniversität in Hagen
  • Rahul Divekar, ETS
  • George Duenas, Universidad Pedagogica Nacional
  • Mariano Felice, British Council
  • Michael Flor, Educational Testing Service
  • Yo Ehara, Tokyo Gakugei University
  • Ananya Ganesh, University of Colorado Boulder
  • Lingyu Gao, TTIC
  • Samuel González-López, Technological University of Nogales Sonora
  • Tanja Heck, University of Tübingen
  • Nicolas Hernandez, Nantes University - LS2N
  • Chung-Chi Huang, Frostburg State University
  • Yi-Ting Huang, NTUST
  • Joseph Marvin Imperial, University of Bath
  • Radu Tudor Ionescu, University of Bucharest
  • Qinjin Jia, North Carolina State University
  • Richard Johansson, University of Gothenburg
  • Elma Kerz, RWTH Aachen University
  • Mamoru Komachi, Tokyo Metropolitan University
  • Kristopher Kyle, University of Oregon
  • Ji-Ung Lee, UKP Lab, TU Darmstadt, Germany
  • Arun Balajiee Lekshmi Narayanan, University of Pittsburgh
  • Xu Li, Zhejiang University
  • Diane Litman, University of Pittsburgh
  • Yudong Liu, Western Washington University
  • Zhexiong Liu, University of Pittsburgh
  • Zoey Liu, University of Florida
  • Peter Ljunglöf, University of Gothenburg
  • Anastassia Loukina, Grammarly Inc.
  • Gunnar Lund, Grammarly
  • Jakub Macina, ETH Zurich
  • Lieve Macken, Ghent University, Belgium
  • James Martin, University of Colorado Boulder
  • Sandeep Mathias, Presidency University, Bangalore
  • Janet Mee, NBME
  • Detmar Meurers, University of Tübingen
  • Farah Nadeem, UNICEF
  • Ben Naismith, Duolingo
  • Sungjin Nam, ACT Inc.
  • Diane Napolitano, Associated Press
  • Kamel Nebhi, Education First
  • Hwee Tou Ng, National University of Singapore
  • Huy Nguyen, Amazon
  • Kostiantyn Omelianchuk, Grammarly
  • Simon Ostermann, Deutsches Forschungszentrum für Künstliche Intelligenz
  • Robert Östling, Department of Linguistics, Stockholm University
  • Ulrike Pado, Hochschule fuer Technik Stuttgart
  • Lis Pereira, Ochanomizu University
  • Reinald Adrian Pugoy, University of the Philippines
  • Long Qin, Alibaba Cloud
  • Mengyang Qiu, University at Buffalo
  • Martí Quixal, University of Tübingen
  • Vipul Raheja, Grammarly
  • Vivi Rantung, Universitas Negeri Manado
  • Arjun Ranesh Rao, Microsoft
  • Manav Rathod, Glean
  • Brian Riordan, ETS
  • Frankie Robertson, University of Jyväskylä
  • Aiala Rosá, Instituto de Computación, Facultad de Ingeniería, Udelar
  • Alla Rozovskaya, Queens College, CUNY
  • Katherine Stasaski, Salesforce Research
  • Helmer Strik, Radboud University
  • Abhijit Suresh, Reddit Inc.
  • Jan Švec, University of West Bohemia
  • Anaïs Tack, KU Leuven, UCLouvain
  • Elaine Uí Dhonnchadha, Trinity College Dublin
  • Alexandra Uitdenbogerd, RMIT
  • Sowmya Vajjala, National Research Council, Canada
  • Giulia Venturi, Institute for Computational Linguistics “A. Zampolli” (CNR-ILC)
  • Carl Vogel, Trinity College Dublin
  • Elena Volodina, University of Gothenburg, Sweden
  • Michael White, The Ohio State University
  • Alistair Willis, The Open University
  • Yiqiao Xu, North Carolina State University
  • Klaus Zechner, ETS
  • Torsten Zesch, FernUniversität in Hagen

Ekaterina Kochmar

unread,
Mar 30, 2023, 2:31:50 AM3/30/23
to ml-...@googlegroups.com

Call for Participation

BEA 2023 Shared Task: Generating AI Teacher Responses in Educational Dialogues

https://sig-edu.org/sharedtask/2023


SHARED TASK DESCRIPTION

Conversational agents offer promising opportunities for education. They can fulfill various roles (e.g., intelligent tutors and service-oriented assistants) and pursue different objectives (e.g., improving student skills and increasing instructional efficiency) (Wollny et al. 2021). Among all of these different vocations of an educational chatbot, the most prevalent one is the AI teacher helping a student with skill improvement and providing more opportunities to practice. Some recent meta-analyses have even reported a significant effect of chatbots on skill improvement, for example in language learning (Bibauw et al. 2022). What is more, current advances in AI and natural language processing have led to the development of conversational agents that are founded on more powerful generative language models.

Despite these promising opportunities, the use of powerful generative models as a foundation for downstream tasks also presents several crucial challenges. In the educational domain in particular, it is important to ascertain whether that foundation is solid or flimsy. Bommasani et al. (2021: pp. 67-72) stressed that, if we want to put these models into practice as AI teachers, it is imperative to determine whether they can (a) speak to students like a teacher, (b) understand students, and (c) help students improve their understanding. Therefore, Tack and Piech (2022) formulated the AI teacher test challenge: How can we test whether state-of-the-art generative models are good AI teachers, capable of replying to a student in an educational dialogue?

Following the AI teacher test challenge, we organize a first shared task on the generation of teacher language in educational dialogues. The goal of the task is to use NLP and AI methods to generate teacher responses in real-world samples of teacher-student interactions. These samples are taken from the Teacher Student Chatroom Corpus (Caines et al. 2020; Caines et al. 2022). Each training sample is composed of a dialogue context (i.e., several teacher-student utterances) as well as the teacher’s response. For each test sample, participants are asked to submit their best generated teacher response.

The purpose of the task is to benchmark the ability of generative models to act as AI teachers, replying to a student in a teacher-student dialogue. Submissions will be ranked according to several automated dialogue evaluation metrics, with the top submissions selected for further human evaluation. During this manual evaluation, human raters will compare a pair of teacher responses in terms of three abilities: can speak like a teacher, can understand a student, can help a student (Tack & Piech 2022). As such, we adopt an evaluation method that is akin to ACUTE-Eval for evaluating dialogue systems (Li et al. 2019).

PARTICIPATION

The shared task is hosted on CodaLab (Pavao et al. 2022). Anyone participating in the shared task will be asked to:

1. Register on the CodaLab platform.
2. Fill in the registration form with their CodaLab ID. Participants must comply with the terms and conditions of the task and the TSCC data outlined in the form.
3. Register for the CodaLab competition using the CodaLab ID. We will only accept people who submitted the registration form. Note that you can participate as a member of one team only.

IMPORTANT DATES

Fri Mar 24, 2023          Training data release
Mon May 1, 2023         Test data release
Fri May 5, 2023            Final submissions due
Mon May 8, 2023         Results announced
Fri May 12, 2023          Human evaluation results announced
Mon May 22, 2023       System papers due
Fri May 26, 2023          Paper reviews returned
Tue May 30, 2023        Camera-ready papers due
Mon June 12, 2023     Pre-recorded video due
July 13, 2023               BEA Workshop at ACL

ORGANIZERS

Anaïs Tack, KU Leuven; Ekaterina Kochmar, MBZUAI; Zheng Yuan, King’s College London; Serge Bibauw, Universidad Central del Ecuador; Chris Piech, Stanford University

Webpage: https://sig-edu.org/sharedtask/2023 

Ekaterina Kochmar

unread,
Apr 9, 2023, 4:52:38 AM4/9/23
to ml-...@googlegroups.com

Final Call for Papers

The 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023)


Location: Toronto

Date: Thursday, July 13, 2023 (co-located with ACL 2023)

Website: https://sig-edu.org/bea/current

Submission Deadline: Monday, April 24, 2023, 11:59pm UTC-12

WORKSHOP DESCRIPTION

The BEA Workshop is a leading venue for NLP innovation in the context of educational applications. It is one of the largest one-day workshops in the ACL community with over 100 registered attendees in the past several years. The growing interest in educational applications and a diverse community of researchers involved resulted in the creation of the Special Interest Group in Educational Applications (SIGEDU) in 2017, which currently has over 300 members.


The 18th BEA workshop will have keynotes by Susan Lottridge (Cambium Assessment) and Jordana Heller (Textio), an invited paper presentation by a member of one of the educational societies from the International Alliance to Advance Learning in the Digital Era (IAALDE), oral presentation sessions, and a large poster session. We expect that the workshop will continue to highlight novel technologies and opportunities for educational NLP in a range of languages.

 

We will solicit papers that incorporate NLP methods, including, but not limited to:

  • automated scoring of open-ended textual and spoken responses;
  • automated scoring/evaluation for written student responses (across multiple genres);
  • AI and ML methods in educational applications;
  • integration of large language models (LLMs) in educational applications;
  • game-based instruction and assessment;
  • educational data mining;
  • intelligent tutoring;
  • collaborative learning environments;
  • peer review;
  • grammatical error detection and correction;
  • learner cognition;
  • spoken dialog;
  • multimodal applications;
  • annotation standards and schemas;
  • tools and applications for classroom teachers, learners and/or test developers; and
  • use of corpora in educational tools.

     

    SHARED TASK

    Webpage: https://sig-edu.org/sharedtask/2023 

    BEA 2023 is hosting a shared task on generation of teacher responses in educational dialogues. Participants will be provided with teacher–student dialogue samples from the Teacher Student Chatroom Corpus (Caines et al., 2020) of real-world teacher–student interactions and will be asked to generate teacher responses using NLP and AI methods. Submissions will be ranked according to automated evaluation metrics, with the top submissions selected for further human evaluation. Given active participation in the previous BEA-hosted shared tasks, we expect to attract around 20 participating teams.

    Organizers: Anaïs Tack, KU Leuven; Ekaterina Kochmar, MBZUAI; Zheng Yuan, King’s College London; Serge Bibauw, Universidad Central del Ecuador; Chris Piech, Stanford University

    INVITED TALKS

      

    BEA 2023 will feature invited talks from Susan Lottridge (Cambium Assessment) and Jordana Heller (Textio), as well as a speaker from one of the IAALDE societies.


    IMPORTANT DATES

    All deadlines are 11:59pm UTC-12 (anywhere on earth).

    • Anonymity Period Begins: Friday, March 24, 2023
    • Submission Deadline: Monday, April 24, 2023
    • Notification of Acceptance: Monday, May 22, 2023
    • Camera-ready Papers Due: Tuesday, May 30, 2023
    • Workshop: Thursday, July 13, 2023

     

    SUBMISSION INFORMATION

    We will be using the ACL Submission Guidelines for the BEA Workshop this year. Authors are invited to submit a long paper of up to eight (8) pages of content, plus unlimited references; final versions of long papers will be given one additional page of content (up to 9 pages) so that reviewers’ comments can be taken into account. We also invite short papers of up to four (4) pages of content, plus unlimited references. Upon acceptance, short papers will be given five (5) content pages in the proceedings. Authors are encouraged to use this additional page to address reviewers’ comments in their final versions. Papers which describe systems are also invited to give a demo of their system. If you would like to present a demo in addition to presenting the paper, please make sure to select either “long paper + demo” or “short paper + demo” under “Submission Category” in the START submission page.

    Previously published papers cannot be accepted. The submissions will be reviewed by the program committee. As reviewing will be blind, please ensure that papers are anonymous. Self-references that reveal the author’s identity, e.g., “We previously showed (Smith, 1991) …”, should be avoided. Instead, use citations such as “Smith previously showed (Smith, 1991) …”.

    We have also included conflict of interest in the submission form. You should mark all potential reviewers who have been authors on the paper, are from the same research group or institution, or who have seen versions of this paper or discussed it with you.

    We will be using the START conference system to manage submissions: https://www.softconf.com/acl2023/bea2023/ 

    DOUBLE SUBMISSION POLICY

    We will follow the official ACL double-submission policy. Specifically:

    Papers being submitted both to BEA and another conference or workshop must:

    • Note on the title page the other conference or workshop to which they are being submitted.
    • State on the title page that if the authors choose to present their paper at BEA (assuming it was accepted), then the paper will be withdrawn from other conferences and workshops.

    ORGANIZING COMMITTEE

    Ekaterina Kochmar

    unread,
    Apr 21, 2023, 12:51:55 PM4/21/23
    to ml-...@googlegroups.com

    Deadline EXTENDED: Tuesday, May 2, 2023, 11:59pm UTC-12

    The 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023)


    Location: Toronto

    Date: Thursday, July 13, 2023 (co-located with ACL 2023)

    Website: https://sig-edu.org/bea/current

    Submission Deadline (extended): Tuesday, May 2, 2023, 11:59pm UTC-12

    • Submission Deadline (extended): Tuesday, May 2, 2023
    • Notification of Acceptance: Monday, May 22, 2023
    • Camera-ready Papers Due: Tuesday, May 30, 2023
    • Workshop: Thursday, July 13, 2023
    Reply all
    Reply to author
    Forward
    0 new messages