Super donors and super recipients: Studying cross-lingual transfer between high-resource and low-resource languages

Despite the increasing popularity of multilingualism within the NLP community, numerous languages continue to be underrepresented due to the lack of available resources.

Our findings surprisingly reveal that the optimal language pairs with improved performance do not necessarily align with direct linguistic motivations, with subtoken overlap playing a more crucial role.

Specific languages tend to be almost universally beneficial for pretraining (super donors), while others benefit from pretraining with almost any language (super recipients).

The Seventh Workshop on Technologies for Machine Translation of Low-Resource Languages (LoResMT 2024) is being held in Bangkok , Thailand .

The conference addresses the gap between high-resource ( 158 -high-resource) languages and 31 low-resource languages (LRLR) languages) Across 158

31 HRLR pairs of language pairs, we investigate how continued pretraining on different languages affects the pretraining model.

Our findings surprisingly reveal that the optimal language pairs with improved performance do not necessarily align with direct linguistic motivations, with subtoken overlap playing a more crucial role.

Specific languages tend to be almost universally beneficial for pretraining (super donors), while others benefit from pretraining with almost any language (super recipients).