Studying and fixing buggy code using ChatGPT

• Main Author: Ling, Yin
• Other Authors: Liu Yang
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Computer and Information Science
• Online Access: https://hdl.handle.net/10356/175119
• Institution: Nanyang Technological University

As software projects become more and more complex, bugs are inevitably introduced. Bugs may lead to undesirable consequences such as software reliability issues or vulnerabilities in the code. Which may then lead to other undesirable financial issues. Hence, bug fixing tasks are important in the software development cycle as this is an important step in identifying defects or undesirable behaviour in the software. However, bug fixing is a difficult task, typically due to the complexity of software that is built over time. Hence, many automated methods have been experimented with to improve the efficiency of bug fixing tasks. In this project, ChatGPT’s ability to do bug fixing is examined. The results show that ChatGPT performs well in bug fixing tasks, with 85% of the programs in the QuixBugs dataset being fixed correctly. However, the performance in test case generation tasks is subpar. In addition, a novel approach that includes test case generation to do bug fixing is played around with. Yet, it was found that generating test cases before generating bug fixes does improve the bug fixing performance of the model.