Spectral tensor train parameterization of deep learning layers

• Main Authors: OBUKHOV, A., RAKHUBA, M., LINIGER, A., HUANG, Zhiwu, GEORGOULIS, S., DAI, D., VAN Gool L.
• 格式: text
• 語言: English
• 出版: Institutional Knowledge at Singapore Management University 2021
• 主題: Artificial Intelligence and Robotics; Databases and Information Systems
• 在線閱讀: https://ink.library.smu.edu.sg/sis_research/6259; https://ink.library.smu.edu.sg/context/sis_research/article/7262/viewcontent/Spectral_Tensor_Train_Parameterization_of_Deep_Learning_Layers.pdf
• 機構: Singapore Management University

We study low-rank parameterizations of weight matrices with embedded spectral properties in the Deep Learning context. The low-rank property leads to parameter efficiency and permits taking computational shortcuts when computing mappings. Spectral properties are often subject to constraints in optim...