Fully decoupled neural network learning using delayed gradients

• المؤلفون الرئيسيون: Zhuang, Huiping, Wang, Yi, Liu, Qinglai, Lin, Zhiping
• مؤلفون آخرون: School of Electrical and Electronic Engineering
• التنسيق: مقال
• اللغة: English
• منشور في: 2024
• الموضوعات: Computer and Information Science; Decoupled learning; Delayed gradients
• الوصول للمادة أونلاين: https://hdl.handle.net/10356/174476

Training neural networks with back-propagation (BP) requires a sequential passing of activations and gradients. This has been recognized as the lockings (i.e., the forward, backward, and update lockings) among modules (each module contains a stack of layers) inherited from the BP. In this paper,...