Newstest 2020 en-ja BLEU scores

[Ross Zhu]

Hello,

I'm looking for the top English-Japanese WMT20 for BLEU. It seems that BLEU score of  http://wmt.ufal.cz/newstest2020-en-ja/ is unnormal.  

And,  what is the evaluation method of Japanese BLEU ?

 I found 

sacrebleu kyoto-test.ref.ja -i kyoto-test.hyp.ja --tokenize ja-mecab -b

which is described in https://github.com/mjpost/sacrebleu.  Is it?

Thanks a lot in advance for your help.

Ross

[Barry Haddow]

Hi Ross

We used character-based bleu to evaluate en-ja. i.e. sacrebleu with "--tokenize char"

best

Barry

On 02/04/2022 04:12, Ross Zhu wrote:

This email was sent to you by someone outside the University.

You should only click on links or attachments if you are certain that the email is genuine and the content is safe.

--
You received this message because you are subscribed to the Google Groups "Workshop on Statistical Machine Translation" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wmt-tasks+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wmt-tasks/f0a391ac-3fc4-4bff-9236-822f8d68a408n%40googlegroups.com.

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.