Lexical ambiguity is one of the many challenging linguistic phenomena involved in translation, i.e., translating an ambiguous word with its correct sense. In this respect, previous work has shown that the translation quality of neural machine translation systems can be improved by explicitly modeling the senses of ambiguous words. Recently, several evaluation test sets have been proposed to measure the word sense disambiguation (WSD) capability of machine translation systems. However, to date, these evaluation test sets do not include any training data that would provide a fair setup measuring the sense distributions present within the training data itself. In this paper, we present an evaluation benchmark on WSD for machine translation for 10 language pairs, comprising training data with known sense distributions. Our approach for the construction of the benchmark builds upon the wide-coverage multilingual sense inventory of BabelNet, the multilingual neural parsing pipeline TurkuNLP, and the OPUS collection of translated texts from the web. The test suite is available at http://github.com/Helsinki-NLP/MuCoW.

Raganato, A., Scherrer, Y., Tiedemann, J. (2020). An Evaluation Benchmark for Testing the Word Sense Disambiguation Capabilities of Machine Translation Systems. In Proceedings of the 12th Language Resources and Evaluation Conference (pp.3668-3675). European Language Resources Association.

An Evaluation Benchmark for Testing the Word Sense Disambiguation Capabilities of Machine Translation Systems

Raganato, A
;
2020

Abstract

Lexical ambiguity is one of the many challenging linguistic phenomena involved in translation, i.e., translating an ambiguous word with its correct sense. In this respect, previous work has shown that the translation quality of neural machine translation systems can be improved by explicitly modeling the senses of ambiguous words. Recently, several evaluation test sets have been proposed to measure the word sense disambiguation (WSD) capability of machine translation systems. However, to date, these evaluation test sets do not include any training data that would provide a fair setup measuring the sense distributions present within the training data itself. In this paper, we present an evaluation benchmark on WSD for machine translation for 10 language pairs, comprising training data with known sense distributions. Our approach for the construction of the benchmark builds upon the wide-coverage multilingual sense inventory of BabelNet, the multilingual neural parsing pipeline TurkuNLP, and the OPUS collection of translated texts from the web. The test suite is available at http://github.com/Helsinki-NLP/MuCoW.
Si
paper
lexical ambiguity; machine translation; word sense disambiguation
English
Twelfth International Conference on Language Resources and Evaluation
979-10-95546-34-4
Raganato, A., Scherrer, Y., Tiedemann, J. (2020). An Evaluation Benchmark for Testing the Word Sense Disambiguation Capabilities of Machine Translation Systems. In Proceedings of the 12th Language Resources and Evaluation Conference (pp.3668-3675). European Language Resources Association.
Raganato, A; Scherrer, Y; Tiedemann, J
File in questo prodotto:
File Dimensione Formato  
2020.lrec-1.452.pdf

Solo gestori archivio

Dimensione 340.67 kB
Formato Adobe PDF
340.67 kB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/361579
Citazioni
  • Scopus 5
  • ???jsp.display-item.citation.isi??? 2
Social impact