Self-organizing neural networks for human activity modeling and analysis

• Main Author: Gao, Shan
• Other Authors: Lihui CHEN
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2020
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence; Engineering::Computer science and engineering::Computer applications::Social and behavioral sciences
• Online Access: https://hdl.handle.net/10356/143281
• Institution: Nanyang Technological University

Activities of Daily Living (ADLs) refer to self-care activities performed by an individual in his/her residence on a daily basis. ADLs are widely used to measure the functional status of elderly people by health professionals. Given the importance of ADLs as indicators of wellness status, there has been active research in ADL detection and recognition in the recent years. However, whereas ADL recognition technologies are fairly mature, modeling ADL patterns remains a challenge and currently, there are limited works in this field. This thesis is aimed at developing computational methods to model and analyze ADL patterns, specifically daily ADL routines that a person frequently performs in a day as well as inter-ADL patterns encoding temporal relationship among individual ADLs. This thesis firstly presents a multi-memory model designed to learn a person's ADL routines. By mimicking the human multi-memory systems, the model, named Activities of Daily Living Adaptive Resonance Theory (ADLART), encodes the user's activity routines through an episodic memory module and generalizes them into semantics ADL routines. Based on ADLART, another model, named Recommendation Activities of Daily Living Adaptive Resonance Theory (RADLART), is developed to recommend ADLs to users. By learning the association among users' profiles, daily ADL routines, and the wellness scores, RADLART provides ADL recommendations for a healthy lifestyle. The second part of the thesis looks at encoding the spatiotemporal information of ADLs explicitly. A neural network model, named spatiotemporal ADL adaptive resonance theory (STADLART), is proposed which explicitly encodes the spatiotemporal information of ADLs. Specifically, it utilises a spatiotemporal-ADL layer to encode spatiotemporally generalised ADLs. The results of the experiments show that STADLART could effectively learn the users' routines. Building upon STADLART, a more general model named spatiotemporal ART (ST-ART) is proposed, which extends STADLART with an added spatiotemporal layer encoding the user's preference of space and time combinations. ST-ART is applied to the ADL domain and has the potential to be used in other fields. Beyond daily ADL routines, inter-ADL patterns, specifically, the causal relationship among ADLs, is studied in this work. A fusion ART model named hypothesis adaptive resonance theory (H-ART) is developed, which is capable of constructing hypotheses rapidly from a small number of observations. Hypotheses are constructed based on observations of repeated pairs of events. Observing that human may change behaviour patterns over time, H-ART also dynamically rejects hypothesis when new observations contradict them. Though illustrated in the ADL domain, H-ART can also be applied to other time series and stream data mining problems. In summary, this thesis presents a family of neural network models for modeling and analysing different aspects of ADL patterns. These models are capable of handling real-time online data. This capability has been evaluated with synthetic and real-life data sets. Nevertheless, there are possible extensions which have not been worked through in this thesis and are left for future studies. Examples are long-term ADL monitoring, wherein the time horizon of analysis spans multiple years, and the association between health conditions and ADL patterns, which may help to unveil possible causes of illnesses.