Fully decoupled neural network learning using delayed gradients

• Main Authors: Zhuang, Huiping, Wang, Yi, Liu, Qinglai, Lin, Zhiping
• Other Authors: School of Electrical and Electronic Engineering
• Format: Article
• Language: English
• Published: 2024
• Subjects: Computer and Information Science; Decoupled learning; Delayed gradients
• Online Access: https://hdl.handle.net/10356/174476
• Institution: Nanyang Technological University

Training neural networks with back-propagation (BP) requires a sequential passing of activations and gradients. This has been recognized as the lockings (i.e., the forward, backward, and update lockings) among modules (each module contains a stack of layers) inherited from the BP. In this paper,...