Advanced hawkes processes for practical event sequence analysis : models and accelerations

• Main Author: Li, Tianbo
• Other Authors: Ke Yiping, Kelly
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
• Online Access: https://hdl.handle.net/10356/152392
• Institution: Nanyang Technological University

Hawkes processes, first proposed in the name of self- and mutually exciting point process, have been widely used to model event sequences produced from natural and social systems. Hawkes processes can capture both individual and interactive behaviors and have achieved satisfactory results in a variety of disciplines. Ranging from recommender systems to earthquake prediction to neural activities, applications regarding Hawkes process have confirmed the effectiveness of the model as a competent tool for dealing with event sequences. In this thesis, I specifically focus on the extension of Hawkes processes to practical event sequences analysis. In particular, I make efforts to the study of Hawkes processes in two aspects: modeling and algorithm acceleration. For the modeling part of event sequences, I pay special attention to the features associated with events and mainly deal with two subproblems. First, I present a novel model named Tweedie Hawkes process (THP), which links features associated with heave-tailed excitation. The model is essentially an instance of probabilistic graphical models and is able to learn from the outbreaks of events and find out the dominant factors behind it. The model parameterizes the excitation parameter in Hawkes process with a Tweedie regression over event features. THP leverages the Tweedie distribution in capturing various excitation effects. A variational EM algorithm is developed for model inference. Some theoretical properties of THP, including the sub-criticality and convergence of the learning algorithm, are discussed. Second, I study the problem of learning from cross-domain event sequences for Hawkes processes. One of the most important characteristics of Hawkes processes is that they link the occurrence of events up to the network structure, which makes it possible to infer the network structure from nothing but the dynamics of the event. However, cross-domain and feature information, which is also instrumental in modeling, is always neglected in existing works. I explore the idea of network transfer for Hawkes processes to leverage cross-domain information. The idea is instantiated by two models trHLSH and BTHM, from parametric and Bayesian perspective, respectively. Both models augment Hawkes processes with features and cross-domain information. We also present effective learning algorithms for each model. Evaluation of both synthetic and real-world datasets demonstrates that the proposed models can jointly learn knowledge from the temporal, feature, and cross-domain information, and have better performance in terms of network recovery and prediction. I also study the problem of acceleration for the learning process of Hawkes processes in the second part of this thesis. Traditional maximum likelihood estimation and expectation-maximization methods often suffer from high computational complexities. To speed up the process of model inference, I propose two methods: a deep learning model, called Graph Convolutional Hawkes Processes (GCHP), and a generic downsampling method called thinning for not only Hawkes processes but more general point processes. Deep learning has been attempted to the learning of event sequences in recent years. Existing methods, however, suffer from the limitations of failing to consider continuous features, relatively time-consuming training process, and restricted intensity assumptions. GCHP eschews recurrent units and is able to learn a non-linear marked Hawkes process via graph convolutional layers. The model provides a general framework for feature embedding in attributed event sequences and learns a nonlinear intensity without pre-defined form. Our model learns point processes with only graph convolutional layers and therefore it can be easily accelerated by the parallel mechanism. The model shows great prediction accuracy and efficiency in the experiment. The second method discusses one of the most fundamental issues about point processes that what is the best sampling method for point processes. Thinning as a downsampling method can be used for accelerating the learning of point processes. I find that the thinning operation preserves the structure of intensity, and is able to estimate parameters with less time and without much loss of accuracy. Theoretical results including intensity, parameter, and gradient estimation on a thinned history are presented for point processes with decouplable intensities. A stochastic optimization algorithm based on the thinned gradient is proposed. Experimental results on synthetic and real-world datasets validate the effectiveness of thinning in the tasks of parameter and gradient estimation, as well as stochastic optimization.