Avoiding starvation of arms in restless multi-armed bandit

Restless multi-armed bandits (RMAB) is a popular framework for optimizing performance with limited resources under uncertainty. It is an extremely useful model for monitoring beneficiaries (arms) and executing timely interventions using health workers (limited resources) to ensure optimal benefit in...

Full description

Saved in:
Bibliographic Details
Main Authors: LI, Dexun, VARAKANTHAM, Pradeep
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2023
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/8593
https://ink.library.smu.edu.sg/context/sis_research/article/9596/viewcontent/Avoiding_Starvation_of_Arms_in_Restless_Multi_Armed_Bandits.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-9596
record_format dspace
spelling sg-smu-ink.sis_research-95962024-01-25T08:44:42Z Avoiding starvation of arms in restless multi-armed bandit LI, Dexun VARAKANTHAM, Pradeep Restless multi-armed bandits (RMAB) is a popular framework for optimizing performance with limited resources under uncertainty. It is an extremely useful model for monitoring beneficiaries (arms) and executing timely interventions using health workers (limited resources) to ensure optimal benefit in public health settings. For instance, RMAB has been used to track patients' health and monitor their adherence in tuberculosis settings, ensure pregnant mothers listen to automated calls about good pregnancy practices, etc. Due to the limited resources, typically certain individuals, communities, or regions are starved of interventions, which can potentially have a significant negative impact on the individual/community in the long term. To that end, we first define a soft fairness objective which entails an algorithm never probabilistically favors one arm over another if the long-term cumulative reward of choosing the latter arm is higher. Then we provide a scalable approach to ensure long-term optimality while satisfying the proposed fairness constraints in RMAB. Our method, referred to as SoftFair, can balance the tradeoff between the goal of having resources uniformly distributed and maximizing cumulative rewards. SoftFair also provides theoretical performance guarantees and is asymptotically optimal. Finally, we demonstrate the utility of our approaches on simulated benchmarks and show that the soft fairness objective can be handled without a significant sacrifice on the optimal value. 2023-06-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/8593 https://ink.library.smu.edu.sg/context/sis_research/article/9596/viewcontent/Avoiding_Starvation_of_Arms_in_Restless_Multi_Armed_Bandits.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Automated calls Community OR Fairness Optimizing performance Patient health Restless multi-armed bandit Softmax Uncertainty Whittle indexs Workers' Databases and Information Systems
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Automated calls
Community OR
Fairness
Optimizing performance
Patient health
Restless multi-armed bandit
Softmax
Uncertainty
Whittle indexs
Workers'
Databases and Information Systems
spellingShingle Automated calls
Community OR
Fairness
Optimizing performance
Patient health
Restless multi-armed bandit
Softmax
Uncertainty
Whittle indexs
Workers'
Databases and Information Systems
LI, Dexun
VARAKANTHAM, Pradeep
Avoiding starvation of arms in restless multi-armed bandit
description Restless multi-armed bandits (RMAB) is a popular framework for optimizing performance with limited resources under uncertainty. It is an extremely useful model for monitoring beneficiaries (arms) and executing timely interventions using health workers (limited resources) to ensure optimal benefit in public health settings. For instance, RMAB has been used to track patients' health and monitor their adherence in tuberculosis settings, ensure pregnant mothers listen to automated calls about good pregnancy practices, etc. Due to the limited resources, typically certain individuals, communities, or regions are starved of interventions, which can potentially have a significant negative impact on the individual/community in the long term. To that end, we first define a soft fairness objective which entails an algorithm never probabilistically favors one arm over another if the long-term cumulative reward of choosing the latter arm is higher. Then we provide a scalable approach to ensure long-term optimality while satisfying the proposed fairness constraints in RMAB. Our method, referred to as SoftFair, can balance the tradeoff between the goal of having resources uniformly distributed and maximizing cumulative rewards. SoftFair also provides theoretical performance guarantees and is asymptotically optimal. Finally, we demonstrate the utility of our approaches on simulated benchmarks and show that the soft fairness objective can be handled without a significant sacrifice on the optimal value.
format text
author LI, Dexun
VARAKANTHAM, Pradeep
author_facet LI, Dexun
VARAKANTHAM, Pradeep
author_sort LI, Dexun
title Avoiding starvation of arms in restless multi-armed bandit
title_short Avoiding starvation of arms in restless multi-armed bandit
title_full Avoiding starvation of arms in restless multi-armed bandit
title_fullStr Avoiding starvation of arms in restless multi-armed bandit
title_full_unstemmed Avoiding starvation of arms in restless multi-armed bandit
title_sort avoiding starvation of arms in restless multi-armed bandit
publisher Institutional Knowledge at Singapore Management University
publishDate 2023
url https://ink.library.smu.edu.sg/sis_research/8593
https://ink.library.smu.edu.sg/context/sis_research/article/9596/viewcontent/Avoiding_Starvation_of_Arms_in_Restless_Multi_Armed_Bandits.pdf
_version_ 1789483282486788096