Decoupled neural network training with re-computation and weight prediction

• Main Authors: Peng, Jiawei, Xu, Yicheng, Lin, Zhiping, Weng, Zhenyu, Yang, Zishuo, Zhuang, Huiping
• Other Authors: School of Electrical and Electronic Engineering
• Format: Article
• Language: English
• Published: 2023
• Subjects: Engineering::Electrical and electronic engineering; Back Propagation; Convolutional Neural Network
• Online Access: https://hdl.handle.net/10356/169712
• Institution: Nanyang Technological University

To break the three lockings during backpropagation (BP) process for neural network training, multiple decoupled learning methods have been investigated recently. These methods either lead to significant drop in accuracy performance or suffer from dramatic increase in memory usage. In this paper, a n...