Cornac-AB : An open-source recommendation framework with native A/B testing integration

• Main Authors: ONG, Rong Sheng, TRUONG, Quoc Tuan, LAUW, Hady Wirawan
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2024
• Subjects: Recommender systems; Collaborative filtering; Recommendation library; A/B testing; open-source framework; Artificial Intelligence and Robotics; Numerical Analysis and Scientific Computing
• Online Access: https://ink.library.smu.edu.sg/sis_research/9850; https://ink.library.smu.edu.sg/context/sis_research/article/10850/viewcontent/webconf24dm.pdf
• Institution: Singapore Management University

Recommender systems significantly impact user experience across diverse domains, yet existing frameworks often prioritize offline evaluation metrics, neglecting the crucial integration of A/B testing for forward-looking assessments. In response, this paper introduces a new framework seamlessly incorporating A/B testing into the Cornac recommendation library. Leveraging a diverse collection of model implementations in Cornac, our framework enables effortless A/B testing experiment setup from offline trained models. We introduce a carefully designed dashboard and a robust backend for efficient logging and analysis of user feedback. This not only streamlines the A/B testing process but also enhances the evaluation of recommendation models in an online environment. Demonstrating the simplicity of on-demand online model evaluations, our work contributes to advancing recommender system evaluation methodologies, underscoring the significance of A/B testing and providing a practical framework for implementation. The framework is open-sourced at https://github.com/PreferredAI/cornac-ab.