Transformer-based architectures brought a breeze of change to Word Sense Disambiguation (WSD), improving models' performances by a large margin. The fast development of new approaches has been further encouraged by a well-framed evaluation suite for English, which has allowed their performances to be kept track of and compared fairly. However, other languages have remained largely unexplored, as testing data are available for a few languages only and the evaluation setting is rather matted. In this paper, we untangle this situation by proposing XL-WSD, a cross-lingual evaluation benchmark for the WSD task featuring sense-annotated development and test sets in 18 languages from six different linguistic families, together with language-specific silver training data. We leverage XL-WSD datasets to conduct an extensive evaluation of neural and knowledge-based approaches, including the most recent multilingual language models. Results show that the zero-shot knowledge transfer across languages is a promising research direction within the WSD field, especially when considering low-resourced languages where large pretrained multilingual models still perform poorly.

Pasini, T., Raganato, A., Navigli, R. (2021). XL-WSD: An Extra-Large and Cross-Lingual Evaluation Framework for Word Sense Disambiguation. In 35th AAAI Conference on Artificial Intelligence, AAAI 2021 (pp.13648-13656). 2275 E BAYSHORE RD, STE 160, PALO ALTO, CA 94303 USA : Association for the Advancement of Artificial Intelligence.

XL-WSD: An Extra-Large and Cross-Lingual Evaluation Framework for Word Sense Disambiguation

Raganato, A;
2021

Abstract

Transformer-based architectures brought a breeze of change to Word Sense Disambiguation (WSD), improving models' performances by a large margin. The fast development of new approaches has been further encouraged by a well-framed evaluation suite for English, which has allowed their performances to be kept track of and compared fairly. However, other languages have remained largely unexplored, as testing data are available for a few languages only and the evaluation setting is rather matted. In this paper, we untangle this situation by proposing XL-WSD, a cross-lingual evaluation benchmark for the WSD task featuring sense-annotated development and test sets in 18 languages from six different linguistic families, together with language-specific silver training data. We leverage XL-WSD datasets to conduct an extensive evaluation of neural and knowledge-based approaches, including the most recent multilingual language models. Results show that the zero-shot knowledge transfer across languages is a promising research direction within the WSD field, especially when considering low-resourced languages where large pretrained multilingual models still perform poorly.
No
paper
word sense disambiguation; multilingual; benchmark;
English
35th AAAI Conference on Artificial Intelligence, AAAI 2021 - 2 February 2021 through 9 February 2021
978-171383597-4
Pasini, T., Raganato, A., Navigli, R. (2021). XL-WSD: An Extra-Large and Cross-Lingual Evaluation Framework for Word Sense Disambiguation. In 35th AAAI Conference on Artificial Intelligence, AAAI 2021 (pp.13648-13656). 2275 E BAYSHORE RD, STE 160, PALO ALTO, CA 94303 USA : Association for the Advancement of Artificial Intelligence.
Pasini, T; Raganato, A; Navigli, R
File in questo prodotto:
File Dimensione Formato  
Pasini_XL-WSD_2021.pdf

Solo gestori archivio

Tipologia di allegato: Publisher’s Version (Version of Record, VoR)
Dimensione 287.42 kB
Formato Adobe PDF
287.42 kB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/361597
Citazioni
  • Scopus 8
  • ???jsp.display-item.citation.isi??? 2
Social impact