AI Seminar: A Distribution-Aware Decision Rule for Neural Machine Translation

Join us for a talk by Bryan Eikema from the University of Amsterdam on May 11, 2022 at 1300.

Title

A Distribution-Aware Decision Rule for Neural Machine Translation

Abstract

In neural machine translation (NMT) we search for the mode of the model distribution to form predictions. The mode as well as other high probability translations found by beam search have been shown to often be inadequate in a number of ways. While this is problematic for mode-seeking search, our findings show that the model is not inherently flawed: most probability mass is put on reasonable translations. This motivates a change in decision rule that takes into account more than just the highest probability sequence. Minimum Bayes Risk is an appealing alternative that seeks for the translation with highest expected utility, akin to finding a consensus translation that is most representative of the entire model distribution under a given utility. In particular, we propose using an unbiased approximation of expected utility using ancestral samples from the model. We explore this alternative decision rule in depth and propose efficient approximations. We find that MBR scales well with more computation, appearing to not suffer from an equivalent of the beam search curse, and eventually is able to outperform beam search.