MEGA Hub
New

GigaAM Multilingual: Foundation Model for Underrepresented Languages

Published on Jul 10 Submitted by MEGAon Jul 21#2 Paper of the day ai-sage

Authors

Do you know Andrei Kuzmenko?You can claim authorship or link another user.Do you know Alexandr Maximenko?You can claim authorship or link another user.Do you know Aleksandr Kutsakov?You can claim authorship or link another user.Do you know Georgii Gospodinov?You can claim authorship or link another user.Do you know Dmitrii Bolotov?You can claim authorship or link another user.Do you know Oleg Kutuzov?You can claim authorship or link another user.Do you know Pavel Bogomolov?You can claim authorship or link another user.Do you know Fyodor Minkin?You can claim authorship or link another user.

Abstract

Despite recent scaling successes, multilingual ASR performance remains highly uneven, with long-tail languages suffering from severe data scarcity. This work addresses the challenge of building robust foundation models for underrepresented Central Asian languages (Kazakh, Kyrgyz, Uzbek). We present GigaAM Multilingual, a Conformer encoder pre-trained on 2M hours of audio using a HuBERT-style objective. Crucially, we introduce a cluster-level data balancing strategy during pre-training and a domain-aware sampling method during fine-tuning to mitigate head-language dominance. In controlled comparisons, our approach outperforms strong open pretrained encoders (Whisper Large v3, Omnilingual-1B) on target languages, achieving significant gains on spontaneous speech while maintaining efficiency. We release the foundation encoder and ASR model, offering a proven recipe for effective multilingual adaptation under realistic data imbalance.

Community

00