Publication Details:

This issue of the Periodical is dedicated to the 85-th anniversary of Hrant B. Marandjian,Doctor of Physical and Mathematical Sciences, Professor, Corresponding Member of NAS RA, Academician of the Russian Academy of Natural Sciences.

Journal or Publication Title:

Математические вопросы кибернетики и вычислительной техники=Կիբեռնետիկայի և հաշվողական տեխնիկայի մաթեմատիկական հարցեր=Mathematical problems of computer science

Date of publication:

2024

Volume:

62

ISSN:

2579-2784 ; e-2538-2788

Additional Information:

click here to follow the link

Coverage:

59-71

Abstract:

Training and deploying BERT models for specific languages, especially low-resource ones, presents a unique set of challenges. These challenges stem from the inherent data scarcity associated with languages like Armenian, the computational demands of training BERT models, often requiring extensive resources, and the inefficiencies in hosting and maintaining models for languages with limited digital traffic. In this research, we introduce a novel methodology that leverages the Armenian Wikipedia as a primary data source, aiming to optimize the performance of BERT for the Armenian language. Our approach demonstrates that, with strategic preprocessing and transfer learning techniques, it's possible to achieve performance metrics that rival those of models trained on more abundant datasets. Furthermore, we explore the potential of fine-tuning pre-trained multilingual BERT models, revealing that they can serve as robust starting points for training models for low-resource but significant languages like Armenian.

Publisher:

Изд-во НАН РА

Format:

pdf

Identifier:

oai:arar.sci.am:405410

Location of original object:

ՀՀ ԳԱԱ Հիմնարար գիտական գրադարան