loading page

Dynamic Fine-tuning Layer Selection Using Kullback-Leibler Divergence
  • Raphael Wanjiku,
  • Michael Kimwele,
  • Lawrence Nderu
Raphael Wanjiku
Jomo Kenyatta University of Agriculture and Technology

Corresponding Author:phaelgi@gmail.com

Author Profile
Michael Kimwele
Jomo Kenyatta University of Agriculture and Technology
Author Profile
Lawrence Nderu
Jomo Kenyatta University of Agriculture and Technology
Author Profile

Abstract

The selection of layers in the transfer learning fine-tuning process ensures a pre-trained model’s accuracy and adaptation in a new target domain. However, the selection process is still manual and without clearly defined criteria. If the wrong layers in a neural network are selected and used, it could lead to poor accuracy and model generalisation in the target domain. This paper introduces the use of Kullback-Leibler divergence on the weight correlations of the model’s convolutional neural network layers. The approach identifies the positive and negative weights in the ImageNet initial weights selecting the best-suited layers of the network depending on the correlation divergence. We experiment on four publicly available datasets and four ImageNet pre-trained models that have been used in past studies for results comparisons. This proposed approach method yields better accuracies than the standard fine-tuning baselines with a margin accuracy rate of 10.8% to 24%, thereby leading to better model adaptation for target transfer learning tasks.
16 Mar 2022Submitted to Engineering Reports
17 Mar 2022Submission Checks Completed
17 Mar 2022Assigned to Editor
25 Mar 2022Reviewer(s) Assigned
25 May 2022Editorial Decision: Revise Major
07 Jul 20221st Revision Received
08 Jul 2022Submission Checks Completed
08 Jul 2022Assigned to Editor
14 Jul 2022Reviewer(s) Assigned
10 Aug 2022Editorial Decision: Revise Major
28 Aug 20222nd Revision Received
29 Aug 2022Submission Checks Completed
29 Aug 2022Assigned to Editor
13 Sep 2022Reviewer(s) Assigned
28 Sep 2022Editorial Decision: Revise Minor
25 Oct 20223rd Revision Received
25 Oct 2022Submission Checks Completed
25 Oct 2022Assigned to Editor
25 Oct 2022Review(s) Completed, Editorial Evaluation Pending
26 Oct 2022Reviewer(s) Assigned
07 Nov 2022Editorial Decision: Accept