Spectral tensor train parameterization of deep learning layers

• Main Authors: OBUKHOV, A., RAKHUBA, M., LINIGER, A., HUANG, Zhiwu, GEORGOULIS, S., DAI, D., VAN Gool L.
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2021
• Subjects: Artificial Intelligence and Robotics; Databases and Information Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/6259; https://ink.library.smu.edu.sg/context/sis_research/article/7262/viewcontent/Spectral_Tensor_Train_Parameterization_of_Deep_Learning_Layers.pdf
• Institution: Singapore Management University

We study low-rank parameterizations of weight matrices with embedded spectral properties in the Deep Learning context. The low-rank property leads to parameter efficiency and permits taking computational shortcuts when computing mappings. Spectral properties are often subject to constraints in optim...