Spectral tensor train parameterization of deep learning layers

• المؤلفون الرئيسيون: OBUKHOV, A., RAKHUBA, M., LINIGER, A., HUANG, Zhiwu, GEORGOULIS, S., DAI, D., VAN Gool L.
• التنسيق: text
• اللغة: English
• منشور في: Institutional Knowledge at Singapore Management University 2021
• الموضوعات: Artificial Intelligence and Robotics; Databases and Information Systems
• الوصول للمادة أونلاين: https://ink.library.smu.edu.sg/sis_research/6259; https://ink.library.smu.edu.sg/context/sis_research/article/7262/viewcontent/Spectral_Tensor_Train_Parameterization_of_Deep_Learning_Layers.pdf
• المؤسسة: Singapore Management University

We study low-rank parameterizations of weight matrices with embedded spectral properties in the Deep Learning context. The low-rank property leads to parameter efficiency and permits taking computational shortcuts when computing mappings. Spectral properties are often subject to constraints in optim...