ScaleLong: Towards more stable training of diffusion model via scaling network long skip connection

• Main Authors: HUANG, Zhongzhan, ZHOU, Pan, YAN, Shuicheng, LIN, Liang
• 格式: text
• 語言: English
• 出版: Institutional Knowledge at Singapore Management University 2023
• 主題: OS and Networks
• 在線閱讀: https://ink.library.smu.edu.sg/sis_research/9025; https://ink.library.smu.edu.sg/context/sis_research/article/10028/viewcontent/2023_NeurIPS_scalelong.pdf
• 機構: Singapore Management University

In diffusion models, UNet is the most popular network backbone, since its long skip connects (LSCs) to connect distant network blocks can aggregate long-distant information and alleviate vanishing gradient. Unfortunately, UNet often suffers from unstable training in diffusion models which can be all...