Learning index policies for restless bandits with application to maternal healthcare

• Main Authors: BISWAS, Arpita, AGGARWAL, Gaurav, VARAKANTHAM, Pradeep, TAMBE, Milind
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2021
• Subjects: reinforcement learning; multi-armed bandits; unknown transition probabilities; Artificial Intelligence and Robotics; Databases and Information Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/6774; https://ink.library.smu.edu.sg/context/sis_research/article/7777/viewcontent/Learning_Index_Policies_for_Restless_Bandits_with_Application_to_Maternal_Healthcare.pdf
• Institution: Singapore Management University

In many community health settings, it is crucial to have a systematic monitoring and intervention process to ensure that the patients adhere to healthcare programs, such as periodic health checks or taking medications. When these interventions are expensive, they can be provided to only a fixed small fraction of the patients at any period of time. Hence, it is important to carefully choose the beneficiaries who should be provided with interventions and when. We model this scenario as a restless multi-armed bandit (RMAB) problem, where each beneficiary is assumed to transition from one state to another depending on the intervention provided to them. In practice, the transition probabilities are unknown a priori, and hence, we propose a mechanism for the problem of balancing the explore-exploit trade-off. Empirically, we find that our proposed mechanism outperforms the baseline intervention scheme maternal healthcare dataset.