On Wed, Jan 28, 2009 at 5:47 PM, Yuval Marton <yuvalmar
...@gmail.com> wrote:
> Hi Chris,
> Following the previous concern about the references, I have to say that
> reading some of the references makes me wonder if they were produced by
> humans (I hope I am not offending anyone here! :-) Some references, I
> think, were not very fluent, and/or not very faithful to the source. For
> example :
> In the production of metal processing a price increase of 13.2 percent, in
> the field of food, beverage, tobacco production 11 percent growth was seen.
> (source: En el sector de la elaboración del metal se observó un aumento de
> un 13,2 por ciento, mientras que en la industria de alimentos, bebidas y
> tabacos, de peso determinante, hubo un aumento de 11 por ciento. )
> or:
> "I want to be a young mother, doesn't matter the candidate"
> (source: "Quiero ser mamá joven; el candidato no me preocupa" )
> So I decided to use my non-native (but hopefully good enough) English, and
> very poor Spanish, in order to judge translations as "Yes" if they were
> around the same quality as the reference (or better), and not vote them down
> if I thought the reference was not better. Please feel free to ignore my
> judgments if you disagree with this policy.
> -Yuval
> On Wed, Jan 28, 2009 at 3:19 AM, Chris Callison-Burch <c...@cs.jhu.edu>
> wrote:
>> Hi Gregor,
>> Good point: if you know the source language, feel free to use that
>> instead of, or in addition to, the reference.
>> --Chris
>> On Jan 27, 2009, at 6:40 PM, in...@filmstudio.rwth-aachen.de wrote:
>> > Hi,
>> > in the "judge the edited" task -- should we go by the source or by
>> > the reference in the cases where the two say something completely
>> > different?
>> > For example, in the following sentence:
>> > "In the sky he is accompanied by the pride of his daughters who are
>> > also his fans on the earth, something not so common in America."
>> > the German translation
>> > "Im Himmel wird er stolz darauf sein, dass auf der Erde seine
>> > Töchter die gleiche Meinung wie seine Fans haben, was in Amerika
>> > nicht unbedingt üblich ist."
>> > would roughly translate as
>> > "In heaven, he will be proud that his daughters and his fans have
>> > the same opinion, which is not really usual in America."
>> > So, what should we do here?
>> > Gregor
>> > On Tue, 27 Jan 2009, Chris Callison-Burch wrote:
>> >> We have updated the WMT09 manual evaluation tasks. Now, instead of
>> >> editing the output of machine translation systems, we are asking you
>> >> to judge whether the edited translations are correct or not.
>> >> The new task shows the source and the reference along with some
>> >> context. You should supply simple Yes/No judgments about whether the
>> >> edited translations represent fully fluent and meaning-equivalent
>> >> alternatives to the reference.
>> >> The deadline for the manual evaluation is the end of Friday. If you
>> >> participated in the shared task, and your group has not yet completed
>> >> your 8 hour time commitment, then please log in and judge some
>> >> translations. The acceptability judgments are highest priority, but
>> >> the ranking statistics are still useful.
>> >> Here is the URL for the manual evaluation:
>> >> http://statmt.org/wmt09/judge/
>> >> Thanks again for your participation!
>> >> Chris Callison-Burch
>> >> Philipp Koehn
>> >> Christof Monz
>> >> Josh Schroeder
>> > --
>> > Gregor Leusch
>> > leu...@i6.informatik.rwth-aachen.de
>> > Aachen University of Technology, Lab for Computer Science VI
