Watermarking deep reinforcement learning

• Main Author: Sim, Ming Jie
• Other Authors: Zhang Tianwei
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Computer science and engineering::Software
• Online Access: https://hdl.handle.net/10356/148747
• Institution: Nanyang Technological University

Deep Reinforcement Learning (DRL) is becoming more widely researched on as it is increasingly useful in solving several complicated problems, such as robotics control and autonomous driving. DRL models are usually built with the help of enormous computational resources that process large amount of proprietary data. The models produced are valuable Intellectual Property (IP) to the designer of the model and need to be secured to preserve the owner’s competitive edge. This report presents a watermarking scheme on DRL as well as evaluates how the model parameters of the Deep Q-Network (DQN) policy used in DRL can affect the watermark performance. The watermarking scheme involves embedding a unique identifier within the policy where a unique sequence of state transitions is produced, while having minimum influence on the policy performance. The digital watermark can help to detect unauthorized duplications of proprietary policies. The watermarking is done on DQN policy in DRL and is trained in the Cartpole environment.