LangBridge: Multilingual Reasoning Without Multilingual Supervision
We introduce LangBridge, a zero-shot approach to adapt language models for multilingual reasoning tasks without multilingual supervision. LangBridge operates by bridging two models, each specialized in different aspects: (1) one specialized in understanding multiple languages (e.g., mT5 encoder) and...
Main Authors: | , , , , , |
---|---|
Format: | Text |
Language: | unknown |
Published: |
2024
|
Subjects: | |
Online Access: | http://arxiv.org/abs/2401.10695 |
id |
ftarxivpreprints:oai:arXiv.org:2401.10695 |
---|---|
record_format |
openpolar |
spelling |
ftarxivpreprints:oai:arXiv.org:2401.10695 2024-02-27T08:44:18+00:00 LangBridge: Multilingual Reasoning Without Multilingual Supervision Yoon, Dongkeun Jang, Joel Kim, Sungdong Kim, Seungone Shafayat, Sheikh Seo, Minjoon 2024-01-19 http://arxiv.org/abs/2401.10695 unknown http://arxiv.org/abs/2401.10695 Computer Science - Computation and Language text 2024 ftarxivpreprints 2024-01-28T02:06:58Z We introduce LangBridge, a zero-shot approach to adapt language models for multilingual reasoning tasks without multilingual supervision. LangBridge operates by bridging two models, each specialized in different aspects: (1) one specialized in understanding multiple languages (e.g., mT5 encoder) and (2) one specialized in reasoning (e.g., Orca 2). LangBridge connects the two models by introducing minimal trainable parameters between them. Despite utilizing only English data for training, LangBridge considerably enhances the performance of language models on low-resource languages across mathematical reasoning, coding, and logical reasoning. Our analysis suggests that the efficacy of LangBridge stems from the language-agnostic characteristics of multilingual representations. We publicly release our code and models. Comment: Work in progress Text Orca ArXiv.org (Cornell University Library) |
institution |
Open Polar |
collection |
ArXiv.org (Cornell University Library) |
op_collection_id |
ftarxivpreprints |
language |
unknown |
topic |
Computer Science - Computation and Language |
spellingShingle |
Computer Science - Computation and Language Yoon, Dongkeun Jang, Joel Kim, Sungdong Kim, Seungone Shafayat, Sheikh Seo, Minjoon LangBridge: Multilingual Reasoning Without Multilingual Supervision |
topic_facet |
Computer Science - Computation and Language |
description |
We introduce LangBridge, a zero-shot approach to adapt language models for multilingual reasoning tasks without multilingual supervision. LangBridge operates by bridging two models, each specialized in different aspects: (1) one specialized in understanding multiple languages (e.g., mT5 encoder) and (2) one specialized in reasoning (e.g., Orca 2). LangBridge connects the two models by introducing minimal trainable parameters between them. Despite utilizing only English data for training, LangBridge considerably enhances the performance of language models on low-resource languages across mathematical reasoning, coding, and logical reasoning. Our analysis suggests that the efficacy of LangBridge stems from the language-agnostic characteristics of multilingual representations. We publicly release our code and models. Comment: Work in progress |
format |
Text |
author |
Yoon, Dongkeun Jang, Joel Kim, Sungdong Kim, Seungone Shafayat, Sheikh Seo, Minjoon |
author_facet |
Yoon, Dongkeun Jang, Joel Kim, Sungdong Kim, Seungone Shafayat, Sheikh Seo, Minjoon |
author_sort |
Yoon, Dongkeun |
title |
LangBridge: Multilingual Reasoning Without Multilingual Supervision |
title_short |
LangBridge: Multilingual Reasoning Without Multilingual Supervision |
title_full |
LangBridge: Multilingual Reasoning Without Multilingual Supervision |
title_fullStr |
LangBridge: Multilingual Reasoning Without Multilingual Supervision |
title_full_unstemmed |
LangBridge: Multilingual Reasoning Without Multilingual Supervision |
title_sort |
langbridge: multilingual reasoning without multilingual supervision |
publishDate |
2024 |
url |
http://arxiv.org/abs/2401.10695 |
genre |
Orca |
genre_facet |
Orca |
op_relation |
http://arxiv.org/abs/2401.10695 |
_version_ |
1792052710072123392 |