Optimizing expectation with guarantees in POMDPs
A standard objective in partially-observable Markov decision processes (POMDPs) is to find a policy that maximizes the expected discounted-sum payoff. However, such policies may still permit unlikely but highly undesirable outcomes, which is problematic especially in safety-critical applications. Re...
محفوظ في:
| المؤلفون الرئيسيون: | , , , |
|---|---|
| التنسيق: | text |
| اللغة: | English |
| منشور في: |
Institutional Knowledge at Singapore Management University
2017
|
| الموضوعات: | |
| الوصول للمادة أونلاين: | https://ink.library.smu.edu.sg/sis_research/9071 https://ink.library.smu.edu.sg/context/sis_research/article/10074/viewcontent/11046_13_14574_1_2_20201228.pdf |
| الوسوم: |
إضافة وسم
لا توجد وسوم, كن أول من يضع وسما على هذه التسجيلة!
|
| المؤسسة: | Singapore Management University |
| اللغة: | English |
| الملخص: | A standard objective in partially-observable Markov decision processes (POMDPs) is to find a policy that maximizes the expected discounted-sum payoff. However, such policies may still permit unlikely but highly undesirable outcomes, which is problematic especially in safety-critical applications. Recently, there has been a surge of interest in POMDPs where the goal is to maximize the probability to ensure that the payoff is at least a given threshold, but these approaches do not consider any optimization beyond satisfying this threshold constraint. In this work we go beyond both the "expectation" and "threshold" approaches and consider a "guaranteed payoff optimization (GPO)" problem for POMDPs, where we are given a threshold t and the objective is to find a policy σ such that a) each possible outcome of σ yields a discounted-sum payoff of at least t, and b) the expected discounted-sum payoff of σ is optimal (or near-optimal) among all policies satisfying a). We present a practical approach to tackle the GPO problem and evaluate it on standard POMDP benchmarks. |
|---|