Neural inductive biases for sequential recommendation

• Main Author: He, Chaoyue
• Other Authors: Miao Chun Yan
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2025
• Subjects: Computer and Information Science; Recommender system
• Online Access: https://hdl.handle.net/10356/182754
• Institution: Nanyang Technological University

In today's data-driven world, recommendation systems are indispensable for personalizing content to user preferences. Within this field, sequential recommendation—focused on ordered or timestamped user interactions—plays an essential role in anticipating future actions. Despite significant advancements in this field, three key challenges remain underexplored: modeling variable-length user histories, capturing complex relationships in item sets, and addressing asynchronous temporal dynamics. Firstly, a significant limitation is the reliance on fixed-length user histories, which neglects the potential benefits of multi-scale or variable-length user subsequence interactions. This oversight hinders the effective capture of nuanced dynamics inherent in user interactions, ultimately creating a notable gap in the adaptability and accuracy of recommendations. Secondly, when transitioning from recommending single items to sets or baskets, the complexity of item relationships increases significantly. Contemporary systems struggle to effectively map and represent these intricate connections, underscoring a critical need for advanced methodologies to unravel and comprehend these multifaceted item relationships. These challenges are further compounded by the third gap - the mishandling of the temporal dimension. Real-world scenarios, characterized by asynchronous timestamps in user-item interactions, necessitate a refined approach. Existing methods, constrained by either solely focusing on absolute timestamps or limited to relative time intervals, fall short of capturing the intricate temporal dynamics that interweave with both variable-length sequences and complex item relationships. To address these interconnected challenges, this thesis investigates a central research question: How can neural inductive biases be developed to enhance sequential recommendation systems by effectively modeling variable interaction lengths, complex item relationships, and temporal dynamics? The thesis systematically examines the often-overlooked sequential dimensions of recommendation systems, particularly focusing on the order and timing of user actions. At the core of our approach is the implementation of innovatively designed neural inductive biases - carefully crafted assumptions embedded within neural architectures that guide the learning process toward more effective sequential recommendations. The thesis presents a progressive exploration of solutions, beginning with next-item recommendation through a multi-scale-based approach. By incorporating specialized convolution within RNN cells, this method captures user-item interactions at different scales of orders, effectively addressing the challenge of modeling variable-length sequences and demonstrating improved recommendation accuracy. Building upon this foundation, the thesis advances to next-basket recommendation, proposing a graph-attention-based method that models transitions between items across consecutive baskets. This approach solves basket-level recommendation by tackling more complex item relationships, utilizing an attention mechanism to capture intricate item dependencies and showing strong performance. Further evolving the research, the thesis progresses to next-items recommendation by integrating asynchronous timestamps and multi-scale interaction utilizing a transformer architecture. This solution combines insights from previous approaches while implementing temporal warping embedding on timestamp intervals within a transformer architecture enriched with multi-scale convolution, effectively handling both variable interaction lengths and temporal dynamics. Lastly, the thesis explores temporal sets recommendation through an innovative nested transformer architecture that simultaneously models inner-set item relationships and outer-set temporal dependencies. This final approach incorporates temporal quantization to convert continuous timestamps into discrete intervals and maps them to learnable encoding vectors. By leveraging a nested transformer structure, the model effectively addresses the dual challenges of complex item relationships and temporal dynamics in a unified and principled manner. Looking beyond the current research, the thesis concludes by identifying promising research directions that build upon this exploration. These directions aim to advance the field's understanding of variable-length sequences, complex item relationships, and temporal dynamics in sequential recommendation systems, while exploring emerging challenges as recommendation systems continue to evolve.