Robust factorization machine: A doubly capped norms minimization

• Main Authors: LIU, Chenghao, ZHANG, Teng, LI, Jundong, YIN, Jianwen, ZHAO, Peilin, SUN, Jianling, HOI, Steven C. H.
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2019
• Subjects: Factorization machines; Feature engineerings; Feature interactions; Loss minimization; Modeling process; Rank minimizations; Real-world datasets; State of the art; Artificial Intelligence and Robotics; Databases and Information Systems; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/4389; https://ink.library.smu.edu.sg/context/sis_research/article/5392/viewcontent/SDM19_RFM.pdf
• Institution: Singapore Management University

Factorization Machine (FM) is a general supervised learning framework for many AI applications due to its powerful capability of feature engineering. Despite being extensively studied, existing FM methods have several limitations in common. First of all, most existing FM methods often adopt the squared loss in the modeling process, which can be very sensitive when the data for learning contains noises and outliers. Second, some recent FM variants often explore the low-rank structure of the feature interactions matrix by relaxing the low-rank minimization problem as a trace norm minimization, which cannot always achieve a tight approximation to the original one. To address the aforementioned issues, this paper proposes a new scheme of Robust Factorization Machine (RFM) by exploring a doubly capped norms minimization approach, which employs both a capped squared trace norm in achieving a tighter approximation of the rank minimization and a capped ℓ1-norm loss to enhance the robustness of the empirical loss minimization from noisy data. We develop an efficient algorithm with a rigorous convergence proof of RFM. Experiments on public real-world datasets show that our method outperforms the state-of-the-art FM methods significantly.