Accumulated decoupled learning with gradient staleness mitigation for convolutional neural networks

• المؤلفون الرئيسيون: Zhuang, Huiping, Weng, Zhenyu, Luo, Fulin, Toh, Kar-Ann, Li, Haizhou, Lin, Zhiping
• مؤلفون آخرون: School of Electrical and Electronic Engineering
• التنسيق: Conference or Workshop Item
• اللغة: English
• منشور في: 2024
• الموضوعات: Computer and Information Science; Convolutional neural networks; Delayed gradients-based methods
• الوصول للمادة أونلاين: https://hdl.handle.net/10356/174480; https://icml.cc/virtual/2021/index.html
• المؤسسة: Nanyang Technological University

Gradient staleness is a major side effect in decoupled learning when training convolutional neural networks asynchronously. Existing methods that ignore this effect might result in reduced generalization and even divergence. In this paper, we propose an accumulated decoupled learning (ADL), wh...