Use of external tools in Restricted Track

Yoav Katz

unread,

Mar 28, 2019, 7:22:54 AM3/28/19

to BEA 2019 Shared Task: Grammatical Error Correction

Hi,

Is the use of language tool (https://languagetool.org/) allowed in the Restricted Track or only in the Unrestricted Track?
Also, are there any limitations on the use of spell checkers in the Restricted Track?

Yoav Katz
IBM Research

BEA 2019 Shared Task Organisers

unread,

Mar 28, 2019, 8:09:12 AM3/28/19

to BEA 2019 Shared Task: Grammatical Error Correction

Hi Yoav,

LanguageTool is allowed in the Restricted Track because it's rule-based (afaik) and hasn't been trained on a learner corpus.

I think most spellcheckers fall into the same category, so both should be fine in the Restricted Track.

Chris

BEA 2019 Shared Task Organisers

unread,

Mar 28, 2019, 2:08:53 PM3/28/19

to BEA 2019 Shared Task: Grammatical Error Correction

Hi Yoav,

On closer inspection, actually LanguageTool is not allowed in the Restricted Track because it has restraints on pricing and character limits.

This is a bit of a grey area, but even though some of it is free, it's not entirely free so we decided it's only allowed in Unrestricted.

Sorry if this causes you any issues, and we're happy to help out if you or anyone submitted to the wrong track.

Chris

Yoav Katz

unread,

Mar 28, 2019, 11:27:17 PM3/28/19

to BEA 2019 Shared Task: Grammatical Error Correction

Hi Chris.

The LanguageTool core is an open source project released under lgpl. The code can be downloaded and run locally (https://github.com/languagetool-org/languagetool) without any cost or restrictions. Therefore I believe it meets that criteria of the Restricted track.

Yoav

BEA 2019 Shared Task

unread,

Mar 29, 2019, 7:34:12 AM3/29/19

to Yoav Katz, BEA 2019 Shared Task: Grammatical Error Correction

Hi Yoav,

Even though it's open source, we also have a question about novelty. Since the main goal of the Restricted Track is to encourage research into new systems and approaches, it doesn't seem particularly innovative to add an existing system to your pipeline (unless it's your own system or a significantly modified version of it).

Spellchecking is a bit different because it's just a component in a system, but it rather defeats the point of the track if you can use someone else's complete system to increase scores. Afterall, everyone could do the same so we don't really learn much.

Hope that makes sense,
Chris

> --
> You received this message because you are subscribed to the Google Groups "BEA 2019 Shared Task: Grammatical Error Correction" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to bea2019st+...@googlegroups.com.
> To post to this group, send an email to bea2...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/bea2019st/631a3fbe-6358-4cb2-a138-4b40cae7132b%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

Reply all

Reply to author

Forward