AI unveiled: a comparative analysis of code accuracy and reliability across various chatbot platforms

With the increasing number of AI chatbots being introduced to the market, they are now being utilised across various industries. AI chatbots can perform a wide range of tasks, including planning, designing, mathematics, engineering, and scientific programming. The surge in popularity of AI chatbots...

Full description

Saved in:
Bibliographic Details
Main Author: Choy, Ji En
Other Authors: Felix Lena Stephanie
Format: Final Year Project
Language:English
Published: Nanyang Technological University 2024
Subjects:
AI
Online Access:https://hdl.handle.net/10356/176352
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-176352
record_format dspace
spelling sg-ntu-dr.10356-1763522024-05-29T09:27:54Z AI unveiled: a comparative analysis of code accuracy and reliability across various chatbot platforms Choy, Ji En Felix Lena Stephanie School of Mechanical and Aerospace Engineering lsfelix@ntu.edu.sg Computer and Information Science Engineering AI ChatGPT Robotics With the increasing number of AI chatbots being introduced to the market, they are now being utilised across various industries. AI chatbots can perform a wide range of tasks, including planning, designing, mathematics, engineering, and scientific programming. The surge in popularity of AI chatbots began with the introduction of ChatGPT in November 2022. Subsequently, major technology companies such as Microsoft with Bing AI and Google with Google Bard followed suit, further expanding the landscape of AI chatbots. These AI chatbots have evolved to possess enhanced capabilities, such as image generation and puzzle solving. Due to the significant dependence on these chatbots, ensuring the reliability and accuracy of the solutions they provide is crucial. This project aims to investigate whether AI chatbots can deliver accurate and dependable programming code for engineering projects. The chosen engineering project for this study is an automatic drone inventory counting system. The objective is to generate code solely from AI chatbots and execute it to enable the drone to fly to a designated area, utilise its camera to scan products, and automatically record the inventory counts in an Excel sheet. To ensure the accuracy of the AI-generated code, the project will be divided into stages. Consistent starting input prompts will be used for all tested AI chatbots to ensure fairness in evaluation. Initially, a selection of popular and freely available AI chatbots was made, followed by a series of preliminary tests to narrow down the choices to three. Subsequently, the selected AI chatbots will undergo testing in stages, starting with object recognition and progressing to inventory counting. The tests will be recorded and evaluated based on criteria such as accuracy and completeness of the results. Recommendations for future research include investigating the impact of input prompt quality on the response quality. Additionally, utilizing a drone equipped with enhanced processing capabilities can ensure the accurate and immediate recording of captured frames. Another avenue for future research involves conducting experiments on premium AI chatbots that require subscription or payment to determine if their responses differ in quality from those provided by free chatbots. Exploring these recommendations can further validate the reliability and quality of answers provided by AI chatbots. Bachelor's degree 2024-05-29T09:27:54Z 2024-05-29T09:27:54Z 2024 Final Year Project (FYP) Choy, J. E. (2024). AI unveiled: a comparative analysis of code accuracy and reliability across various chatbot platforms. Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/176352 https://hdl.handle.net/10356/176352 en B346 application/pdf Nanyang Technological University
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Computer and Information Science
Engineering
AI
ChatGPT
Robotics
spellingShingle Computer and Information Science
Engineering
AI
ChatGPT
Robotics
Choy, Ji En
AI unveiled: a comparative analysis of code accuracy and reliability across various chatbot platforms
description With the increasing number of AI chatbots being introduced to the market, they are now being utilised across various industries. AI chatbots can perform a wide range of tasks, including planning, designing, mathematics, engineering, and scientific programming. The surge in popularity of AI chatbots began with the introduction of ChatGPT in November 2022. Subsequently, major technology companies such as Microsoft with Bing AI and Google with Google Bard followed suit, further expanding the landscape of AI chatbots. These AI chatbots have evolved to possess enhanced capabilities, such as image generation and puzzle solving. Due to the significant dependence on these chatbots, ensuring the reliability and accuracy of the solutions they provide is crucial. This project aims to investigate whether AI chatbots can deliver accurate and dependable programming code for engineering projects. The chosen engineering project for this study is an automatic drone inventory counting system. The objective is to generate code solely from AI chatbots and execute it to enable the drone to fly to a designated area, utilise its camera to scan products, and automatically record the inventory counts in an Excel sheet. To ensure the accuracy of the AI-generated code, the project will be divided into stages. Consistent starting input prompts will be used for all tested AI chatbots to ensure fairness in evaluation. Initially, a selection of popular and freely available AI chatbots was made, followed by a series of preliminary tests to narrow down the choices to three. Subsequently, the selected AI chatbots will undergo testing in stages, starting with object recognition and progressing to inventory counting. The tests will be recorded and evaluated based on criteria such as accuracy and completeness of the results. Recommendations for future research include investigating the impact of input prompt quality on the response quality. Additionally, utilizing a drone equipped with enhanced processing capabilities can ensure the accurate and immediate recording of captured frames. Another avenue for future research involves conducting experiments on premium AI chatbots that require subscription or payment to determine if their responses differ in quality from those provided by free chatbots. Exploring these recommendations can further validate the reliability and quality of answers provided by AI chatbots.
author2 Felix Lena Stephanie
author_facet Felix Lena Stephanie
Choy, Ji En
format Final Year Project
author Choy, Ji En
author_sort Choy, Ji En
title AI unveiled: a comparative analysis of code accuracy and reliability across various chatbot platforms
title_short AI unveiled: a comparative analysis of code accuracy and reliability across various chatbot platforms
title_full AI unveiled: a comparative analysis of code accuracy and reliability across various chatbot platforms
title_fullStr AI unveiled: a comparative analysis of code accuracy and reliability across various chatbot platforms
title_full_unstemmed AI unveiled: a comparative analysis of code accuracy and reliability across various chatbot platforms
title_sort ai unveiled: a comparative analysis of code accuracy and reliability across various chatbot platforms
publisher Nanyang Technological University
publishDate 2024
url https://hdl.handle.net/10356/176352
_version_ 1806059901082927104