Dilluns 20 de juliol: Lluís Màrquez - Discourse Structure in Machine Transla tion Evaluation

[Xavier Lluís]

Next monday July 20th, Lluís Màrquez from Qatar Computing Research Institute will give a talk about his recent research on Machine Translation.

Títol	Discourse Structure in Machine Translation Evaluation
Ponent	Lluís Màrquez
Lloc	Omega-S208 Campus Nord - UPC
Dia	Dilluns 20 de Juliol 2015
Horari	11:00h - Presentació
Abstract

 

In this talk I will describe our research at the Arabic Language Technologies group from the Qatar Computing Research Institute on applying discourse-level information to automatic machine translation (MT) evaluation. 

I will start by describing some variants of a discourse-aware similarity measure, which uses the `all-subtree’ convolution kernel to compare discourse parse trees in accordance with the Rhetorical Structure Theory. Then, I will show that these measures help improve a number of already existing MT evaluation metrics both at the segment and at the system level by increasing the correlation with human judgements. This indicates that discourse information is complementary to the state-of-the-art metrics, and thus could be taken into account in the development of richer evaluation measures.

In a second part I will present a strong and robust evaluation measure combining the discourse-based similarity with other metrics from the Asiya MT evaluation toolkit, and tuning the weights of the combination on actual human judgments. Experiments on the WMT12, WMT13, and WMT14 metrics shared task datasets show correlation with human judgments that outperforms those of the state-of-the-art, both at the segment and at the system level with very consistent results across language pairs.

In the final part of the talk, I will introduce two preliminary attempts of learning metrics from finer-grained features for pairwise quality comparison. In the first one, we use preference reranking with kernels to learn from tree structured representation. In the second one, we use a Neural Network architecture to learn from a distributed representation of syntax and semantics. Both frameworks are developed with the spirit of being general and extensible from MT evaluation to quality estimation and machine translation.