Fast adaptation of activity sensing policies in mobile devices

• Main Authors: ALSHEIKH, Mohammad Abu, NIYATO, Dusit, LIN, Shaowei, TAN, Hwee-Pink, KIM, Dong In
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2017
• Subjects: Activity tracking; fast adaptation; Internet of Things; Markov decision processes; wireless charging; Computer Sciences; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/3858; https://ink.library.smu.edu.sg/context/sis_research/article/4860/viewcontent/161103202v1.pdf
• Institution: Singapore Management University

With the proliferation of sensors, such as accelerometers,in mobile devices, activity and motion tracking has become a viable technologyto understand and create an engaging user experience. This paper proposes afast adaptation and learning scheme of activity tracking policies when userstatistics are unknown a priori, varying with time, and inconsistent for differentusers. In our stochastic optimization, user activities are required to besynchronized with a backend under a cellular data limit to avoid overchargesfrom cellular operators. The mobile device is charged intermittently usingwireless or wired charging for receiving the required energy for transmission andsensing operations. Firstly, we propose an activity tracking policy byformulating a stochastic optimization as a constrained Markov decision process(CMDP). Secondly, we prove that the optimal policy of the CMDP has a thresholdstructure using a Lagrangian relaxation approach and the submodularity concept.We accordingly present a fast Q-learning algorithm by considering the policystructure to improve the convergence speed over that of conventionalQ-learning. Finally, simulation examples are presented to support thetheoretical findings of this paper.