Max-Match scorer Vs Span-based correction in ERRANT

83 views

Skip to first unread message

awasthia...@gmail.com

unread,

Mar 28, 2019, 10:05:57 AM3/28/19

to BEA 2019 Shared Task: Grammatical Error Correction

Hi,

Is there any difference between Max-Match scorer Vs Span-based correction in ERRANT ?

I observe a significantly higher score when I evaluate my system with Max-Match scorer as compared to Span-based correction in ERRANT

BEA 2019 Shared Task Organisers

unread,

Mar 28, 2019, 8:23:26 PM3/28/19

to BEA 2019 Shared Task: Grammatical Error Correction

Yes. MaxMatch attempts to extract the edits from the hypothesis such that they maximally overlap with the reference, while ERRANT separately annotates the hypothesis and then compares the edits with the reference later.

One disadvantage of MaxMatch in this regard is that it merges false positives within 2 tokens of each other (controlled by the max_unchanged_word parameter) which artificially increases the scores.