Re: Formula used to calculate BLEU

[Martin Popel]

Hi Arben,
t2t-bleu implementation (as well as SacreBLEU) uses the correct BLEU formula
(as in the original paper https://www.aclweb.org/anthology/P02-1040).

The formula in https://i.stack.imgur.com/MZygZ.png is wrong
because of the brevity penalty, which should be
BP = exp(min(0, 1 - r/c)) = min(1, exp(1 - r/c))
but the image uses
BP = min(1, c/r)

Except for the brevity penalty, the formula on the image is correct
because the geometric mean can also be expressed as the exponential of the arithmetic mean of logarithms,
see https://en.wikipedia.org/wiki/Geometric_mean#Relationship_with_logarithms


Martin

----- Original Message -----
> From: "Arben Sabani" <arben....@gmail.com>
> To: "tensor2tensor" <tensor...@googlegroups.com>
> Sent: Saturday, July 13, 2019 6:01:05 AM
> Subject: Formula used to calculate BLEU

> Hi Martin
> Hi everyone
>
> In the paper https://arxiv.org/pdf/1804.00247.pdf is mentioned that BLEU is
> calculated using "t2t-bleu" command.
> Does anyone know how the mathematical formula looks like.
>
>
> In https://www.aclweb.org/anthology/P02-1040 is this formula mentioned:
>
> [image: 2019-07-13 14_50_49-bleu.dvi.png] <about:invalid#zClosurez>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> But there are other sources where the formula looks a bit different:
>
> <https://i.stack.imgur.com/MZygZ.png>
>
> --
> You received this message because you are subscribed to the Google Groups
> "tensor2tensor" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to tensor2tenso...@googlegroups.com.
> To post to this group, send email to tensor...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/tensor2tensor/d20aaa29-8709-4f29-b112-900193c31881%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.