Publication Details:

This issue of the Periodical is dedicated to the 85-th anniversary of Hrant B. Marandjian, Doctor of Physical and Mathematical Sciences, Professor, Corresponding Member of NAS RA, Academician of the Russian Academy of Natural Sciences.

Journal or Publication Title:

Математические вопросы кибернетики и вычислительной техники=Կիբեռնետիկայի և հաշվողական տեխնիկայի մաթեմատիկական հարցեր=Mathematical problems of computer science

Date of publication:

2024

Volume:

62

ISSN:

2579-2784 ; e-2538-2788

Additional Information:

click here to follow the link

Coverage:

82-92

Abstract:

Hebbian learning, an important concept in neural networks, is the basis for various learning algorithms that model the adaptation of neural connections, also known as synapses. Among these models, Oja’s rule stands out as an important example, giving valuable insights into the dynamics of unsupervised learning algorithms. The fact that the final steady-state solution of a single-layer network that learns using Oja’s rule equals the solution of Principal component analysis is well known. However, the way in which the learning rate can affect the variance of the final parameters is less explored. In this paper, we investigate how different learning rates can influence the variance of parameters in Oja’s rule, utilizing the moment closure approximation. By focusing on the variance, we offer new perspectives on the behavior of Oja’s rule under varying conditions. We derive a closed-form equation that connects the parameter variance with the learning rate and shows that the relationship between these is linear. This gives valuable insights that may help to optimize the learning process of Hebbian models.

Publisher:

Изд-во НАН РА

Format:

pdf

Identifier:

oai:arar.sci.am:406428

Location of original object:

ՀՀ ԳԱԱ Հիմնարար գիտական գրադարան