Dynamic flexible job shop scheduling using double deep Q-network

• Main Author: Chua, Hui Shun
• Other Authors: Rajesh Piplani
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Industrial engineering::Supply chain; Engineering::Mechanical engineering
• Online Access: https://hdl.handle.net/10356/150878
• Institution: Nanyang Technological University

The job shop scheduling problem (JSP) is a scheduling problem that aims to generate a near-optimal production schedule in a job shop. The flexible job shop scheduling problem (FJSP) is an extension of the JSP where each operation can be assigned on one or more available machines. The FJSP involves 2 problems: the machine selection problem and the job sequencing problem. The dynamic FJSP (DFJSP) considers dynamic events like job arrivals, job insertions, machine breakdowns and irregular processing times. Many methods such as metaheuristics, traditional reinforcement learning and deep reinforcement learning, have been used to solve dynamic scheduling problems like the DFJSP. However, research on the application of double deep Q-Network (DDQN)-based methods is limited. This report uses a DDQN-based approach to solve the DFJSP with new job arrivals. The objective is to minimize the total tardiness of the jobs at the flexible job shop. This approach uses two agents to solve the DFJSP. The design of the reward mechanism for the two agents is discussed. The performances of the individual agents and the combined model are benchmarked against some popular combined routing and sequencing rules. The results show that the combined model reduces the total tardiness of jobs by 4.60% compared to the best performing individual heuristic rules, resulting in a more optimal schedule and robust solution to the DFJSP. This report also explores how the percentage of tardy jobs and maximum job tardiness contribute to the total tardiness of jobs. Overall, this report provides insights on how DDQN-based approaches to the DFJSP can be improved.