ScaleLong: Towards more stable training of diffusion model via scaling network long skip connection

ScaleLong: Towards more stable training of diffusion model via scaling network long skip connection

In diffusion models, UNet is the most popular network backbone, since its long skip connects (LSCs) to connect distant network blocks can aggregate long-distant information and alleviate vanishing gradient. Unfortunately, UNet often suffers from unstable training in diffusion models which can be all...

Saved in:

書目詳細資料
Main Authors:	HUANG, Zhongzhan, ZHOU, Pan, YAN, Shuicheng, LIN, Liang
格式:	text
語言:	English
出版:	Institutional Knowledge at Singapore Management University 2023
主題:	OS and Networks
在線閱讀:	https://ink.library.smu.edu.sg/sis_research/9025 https://ink.library.smu.edu.sg/context/sis_research/article/10028/viewcontent/2023_NeurIPS_scalelong.pdf
標簽:	添加標簽沒有標簽, 成為第一個標記此記錄!
機構:	Singapore Management University
語言:	English

相似書籍

Simple and effective curriculum pointer-generator networks for reading comprehension over long narratives
由: TAY, Yi, et al.
出版: (2019)

Scaling human activity recognition via deep learning-based domain adaptation
由: KHAN, Md Abdullah Hafiz, et al.
出版: (2018)

Dark hazard: Large-scale discovery of unknown hidden sensitive operations in Android apps
由: PAN, Xiaorui, et al.
出版: (2017)

CRRN: Multi-scale guided concurrent reflection removal network
由: WAN, Renjie, et al.
出版: (2018)

Quantization-aware interval bound propagation for training certifiably robust quantized neural networks
由: LECHNER, Mathias, et al.
出版: (2023)

Stability verification in stochastic control systems via neural network supermartingales
由: LECHNER, Mathias, et al.
出版: (2024)

Learning and Controlling Network Diffusion in Dependent Cascade Models
由: DU, Jiali, et al.
出版: (2015)

4-bit shampoo for memory-efficient network training
由: WANG, Sike, et al.
出版: (2024)

Modeling route representation with mixed-scale hierarchical transformer
由: ZHANG, Hanyuan, et al.
出版: (2024)

New insight into hybrid stochastic gradient descent: Beyond with-replacement sampling and convexity
由: ZHOU, Pan, et al.
出版: (2018)

Urban scale trade area characterization for commercial districts with cellular footprints
由: ZHAO, Yi, et al.
出版: (2020)

Pre-training graph neural networks for link prediction in biomedical networks
由: LONG, Yahui, et al.
出版: (2022)

Adaptive scaling of cluster boundaries for large-scale social media data clustering
由: MENG, Lei, et al.
出版: (2015)

Built to last, not to scale: the long run of decentralised autonomous organisations
由: Sheikh, Saqib, et al.
出版: (2024)

Solving the master equation for extremely long time scale calculations
由: Lee, H.K., et al.
出版: (2014)

Win: Weight-decay-integrated Nesterov acceleration for faster network training
由: ZHOU, Pan, et al.
出版: (2024)

Distributed representation learning with skip-gram model for trained random forests
由: MA, Chao, et al.
出版: (2023)

Cross-project defect prediction via ASTToken2Vec and BLSTM-based neural network
由: LI, Hao, et al.
出版: (2019)

Learning spatio-temporal representation with local and global diffusion
由: QIU, Zhaofan, et al.
出版: (2019)

Task relation networks
由: LI, Jianshu, et al.
出版: (2019)

Advanced Excel training: Functions, macros, V.B.A. programming logic
由: Echavez, Christian D.
出版: (2011)

Large-Scale Functional Organization of Long-Range Chromatin Interaction Networks
由: Sandhu, K.S., et al.
出版: (2013)

Geometric scaling and long-run reductions in cost: The case of wind turbines
由: Srikanth, N., et al.
出版: (2014)

On size-oriented long-tailed graph classification of graph neural networks
由: LIU, Zemin, et al.
出版: (2022)

Semantic overlay network for large-scale spatial information indexing
由: ZOU, Zhiqiang, et al.
出版: (2013)

Empirical risk landscape analysis for understanding deep neural networks
由: ZHOU, Pan, et al.
出版: (2018)

Clustering of snapping shrimp snaps on long time scales: A simulation study
由: Legg, M.W., et al.
出版: (2014)

Polarization retention on short, intermediate, and long time scales in ferroelectric thin films
由: Lou, X.J.
出版: (2014)

Processing of multiple long-running queries in large-scale geo-data repositories
由: Tok, W.H., et al.
出版: (2014)

Network Effects and Embedded Options: Decision-Making under Uncertainty for Network Technology Investments.
由: KAUFFMAN, Robert John, et al.
出版: (2008)

Efficient Data Compression with Error Bound Guarantee in Wireless Sensor Networks
由: Mohammad Abu Alsheikh,, et al.
出版: (2014)

Network Structure of Social Coding in GitHub
由: Thung, Ferdian, et al.
出版: (2013)

Optimization planning for 3D ConvNets
由: QIU, Zhaofan, et al.
出版: (2021)

Towards omni-generalizable neural methods for vehicle routing problems
由: ZHOU, Jianan, et al.
出版: (2023)

Infinite time horizon safety of Bayesian neural networks
由: LECHNER, Mathias, et al.
出版: (2021)

Social balance on networks: Local minima and best-edge dynamics
由: CHATTERJEE, Krishnendu, et al.
出版: (2024)

SleepMore: Inferring sleep duration at scale via multi-device WiFi sensing
由: ZAKARIA, Camellia, et al.
出版: (2023)

Efficient stochastic gradient hard thresholding
由: ZHOU, Pan, et al.
出版: (2018)

On the robustness of diffusion in a network under node attacks
由: LOGINS, Alvis, et al.
出版: (2022)

Theory-inspired path-regularized differential network architecture search
由: ZHOU, Pan, et al.
出版: (2020)