Friendly sharpness-aware minimization

• Main Authors: LI, Tao, ZHOU, Pan, HE, Zhengbao, CHENG, Xinwen, HUANG, Xiaolin
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2024
• Subjects: Theory and Algorithms
• Online Access: https://ink.library.smu.edu.sg/sis_research/9018; https://ink.library.smu.edu.sg/context/sis_research/article/10021/viewcontent/2024_CVPR_FSAM.pdf
• Institution: Singapore Management University

Sharpness-Aware Minimization (SAM) has been instrumental in improving deep neural network training by minimizing both training loss and loss sharpness. Despite the practical success, the mechanisms behind SAM’s generalization enhancements remain elusive, limiting its progress in deep learning optimi...