scispace - formally typeset
Search or ask a question
Posted ContentDOI

Feasibility of Transfer Learning: A Mathematical Framework

22 May 2023-
TL;DR: In this article , the feasibility issue of transfer learning is addressed and a mathematical framework for transfer learning based on three-step optimization problem is proposed, which is shown to be optimal under certain technical conditions, such as appropriate choice of loss functions and data sets.
Abstract: Transfer learning is a popular paradigm for utilizing existing knowledge from previous learning tasks to improve the performance of new ones. It has enjoyed numerous empirical successes and inspired a growing number of theoretical studies. This paper addresses the feasibility issue of transfer learning. It begins by establishing the necessary mathematical concepts and constructing a mathematical framework for transfer learning. It then identifies and formulates the three-step transfer learning procedure as an optimization problem, allowing for the resolution of the feasibility issue. Importantly, it demonstrates that under certain technical conditions, such as appropriate choice of loss functions and data sets, an optimal procedure for transfer learning exists. This study of the feasibility issue brings additional insights into various transfer learning problems. It sheds light on the impact of feature augmentation on model performance, explores potential extensions of domain adaptation, and examines the feasibility of efficient feature extractor transfer in image classification.