Deep learning in virtual reality

• Main Author: Feng, Chengxuan
• Other Authors: Lin Feng
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
• Online Access: https://hdl.handle.net/10356/148133
• Institution: Nanyang Technological University

Physical rehabilitation may be a vital component for patients who are recuperating from a surgery, injury, or a disabling medical condition. During the treatment session, the patient relies greatly on the physiotherapist’s verbal feedback. However, in the event that the patient has to undergo long periods of rehabilitation, exercises done outside of the physiotherapist’s guidance could be ineffective as the patient is unable to visualize and attain immediate feedback. To better facilitate the patient’s recovery process, this project applies deep reinforcement learning on a humanoid model using Unity game engine and Unity’s ML-agents such that it is able to imitate a given training animation. The project is tested within the premise of a golf swing – an exercise that aims to benefit patients that suffer from shoulder arthritis. The implemented deep reinforcement learning algorithm proves to be a promising step in the right direction towards developing a real-time feedback system that could playback the patient’s movement and provide instant feedback to the user. Partial results were published in: Raymond Tan Rui Ming, Chengxuan Feng, Hock Soon Seah, Feng Lin, Movability Assessment on Physiotherapy for Shoulder Periarthritis via Fine-Grained 3D ResNet Deep Learning, SPIE Proceedings of International Forum on Medical Imaging Asia (IFMIA’21), Taiwan (Online), 24-27 January 2021