Ամսագրի կամ հրապարակման վերնագիր:
Հրապարակման ամսաթիվ:
Հատոր:
Լրացուցիչ տեղեկություն:
սեղմիր այստեղ կապին հետևելու համար
Վերնագիր:
Information Theory Tools and Techniques to Overcome Machine Learning Challenges
Ստեղծողը:
Haroutunian, Mariam E. ; Gharagyozyan, Gor A.
Խորագիր:
Չվերահսկվող բանալի բառեր:
Information Bottleneck ; Entropy-Based regularization ; Mutual information ; Feature selection
Ծածկույթ:
Ամփոփում:
In this survey, we explore the broad applications of Information Theory in Machine Learning, highlighting how core concepts like entropy, Mutual Information, and KLdivergence are used to enhance learning algorithms. Since its inception by Claude Shannon, Information Theory has provided mathematical tools to quantify uncertainty, optimize decision-making, and manage the trade-off between model flexibility and generalization. These principles have been integrated across various subfields of Machine Learning, including neural networks, where the Information Bottleneck offers insights into data representation, and reinforcement learning, where entropy-based methods improve exploration strategies. Additionally, measures like Mutual Information are critical in feature selection and unsupervised learning. This survey bridges foundational theory with its practical implementations in modern Machine Learning by providing both historical context and a review of contemporary research.. We also discuss open challenges and future directions, such as scalability and interpretability, highlighting the growing importance of these techniques in next-generation models.