A smoothed Q-learning algorithm for estimating optimal dynamic treatment regime

• Main Authors: FAN, Yanqin, HE, Ming, SU, Liangjun, ZHOU, Xiao-Hua
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2019
• Subjects: Asymptotic normality; Exceptional law; Optimal smoothing parameter; Sequential randomization; Wald-type inference; Econometrics
• Online Access: https://ink.library.smu.edu.sg/soe_research/2044; https://ink.library.smu.edu.sg/context/soe_research/article/3043/viewcontent/Smoothed_Q_learning_algorithm_for_estimating_optimal_2016_pp.pdf
• Institution: Singapore Management University

In this paper we propose a smoothed Q-learning algorithm for estimating optimal dynamic treatment regimes. In contrast to the Q-learning algorithm in which non-regular inference is involved, we show that under assumptions adopted in this paper, the proposed smoothed Q-learning estimator is asymptotically normally distributed even when the Q-learning estimator is not and its asymptotic variance can be consistently estimated. As a result, inference based on the smoothed Q-learning estimator is standard. We derive the optimal smoothing parameter and propose a data-driven method for estimating it. The finite sample properties of the smoothed Q-learning estimator are studied and compared with several existing estimators including the Q-learning estimator via an extensive simulation study. We illustrate the new method by analyzing data from the Clinical Antipsychotic Trials of Intervention EffectivenessAlzheimer’s Disease (CATIE-AD) study.