Assessing the effectiveness of a chatbot workshop as experiential teaching and learning tool to engage undergraduate students

In this paper, we empirically examine and assess the effectiveness of a chatbot workshop as experiential teaching and learning tool to engage undergraduate students enrolled in an elective course “Doing Business with A.I.” in the Lee Kong Chian School of Business (LKCSB) at Singapore Management Univ...

Full description

Saved in:
Bibliographic Details
Main Authors: SHIM, Kyong Jin, MENKHOFF, Thomas, TEO, Ying Qian, ONG, Clement Shi Qi
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2023
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/8844
https://ink.library.smu.edu.sg/context/sis_research/article/9847/viewcontent/s10639_023_11795_5_pvoa_cc_by.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-9847
record_format dspace
spelling sg-smu-ink.sis_research-98472024-06-06T08:31:11Z Assessing the effectiveness of a chatbot workshop as experiential teaching and learning tool to engage undergraduate students SHIM, Kyong Jin MENKHOFF, Thomas TEO, Ying Qian ONG, Clement Shi Qi In this paper, we empirically examine and assess the effectiveness of a chatbot workshop as experiential teaching and learning tool to engage undergraduate students enrolled in an elective course “Doing Business with A.I.” in the Lee Kong Chian School of Business (LKCSB) at Singapore Management University. The chatbot workshop provides non-STEM students with an opportunity to acquire basic skills to build a chatbot prototype using the ‘Dialogflow’ program. The workshop and the experiential learning activity are designed to impart conversation and user-centric design know how and know why to students. A key didactical aspect which informs the design and flow of the chatbot workshop is that novice learners with no or very little knowledge about A.I. recognize and create the important linkage between knowledge inputs and outputs of conversational agents powered by natural language processing (NLP) so that user queries can be effectively addressed. According to the study results, 90.7% of all surveyed students (n = 43) were satisfied with the experiential learning chatbot workshop; 81.4% of the respondents felt engaged while 81.3% of the participants reported moderate to high levels of competencies (81.3%) as result of the hands-on workshop. Almost all students surveyed (97.7%) felt that the experiential chatbot workshop had met the expected learning outcomes. Besides presenting empirical data that underscore the pedagogical usefulness of conducting an experiential Chatbot workshop in basic A.I. classes with special reference to NLP, we are trying to corroborate a conceptual model developed from learning theories and technology mediated learning (TML) models aimed at measuring the effects of a chatbot practicum on students’ engagement and motivation as potential drivers of successfully acquiring basic NLP skills and learner satisfaction. The paper provides useful practical information for instructors interested in deploying a practical chatbot workshop as effective TML tool in a tertiary educational context aimed at making learners ‘future-ready’. 2023-01-01T08:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/8844 info:doi/10.1007/s10639-023-11795-5 https://ink.library.smu.edu.sg/context/sis_research/article/9847/viewcontent/s10639_023_11795_5_pvoa_cc_by.pdf http://creativecommons.org/licenses/by/3.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Chatbots Natural Language Processing Experiential Learning Student Engagement Learning Effectiveness Artificial Intelligence and Robotics Asian Studies Higher Education
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Chatbots
Natural Language Processing
Experiential Learning
Student Engagement
Learning Effectiveness
Artificial Intelligence and Robotics
Asian Studies
Higher Education
spellingShingle Chatbots
Natural Language Processing
Experiential Learning
Student Engagement
Learning Effectiveness
Artificial Intelligence and Robotics
Asian Studies
Higher Education
SHIM, Kyong Jin
MENKHOFF, Thomas
TEO, Ying Qian
ONG, Clement Shi Qi
Assessing the effectiveness of a chatbot workshop as experiential teaching and learning tool to engage undergraduate students
description In this paper, we empirically examine and assess the effectiveness of a chatbot workshop as experiential teaching and learning tool to engage undergraduate students enrolled in an elective course “Doing Business with A.I.” in the Lee Kong Chian School of Business (LKCSB) at Singapore Management University. The chatbot workshop provides non-STEM students with an opportunity to acquire basic skills to build a chatbot prototype using the ‘Dialogflow’ program. The workshop and the experiential learning activity are designed to impart conversation and user-centric design know how and know why to students. A key didactical aspect which informs the design and flow of the chatbot workshop is that novice learners with no or very little knowledge about A.I. recognize and create the important linkage between knowledge inputs and outputs of conversational agents powered by natural language processing (NLP) so that user queries can be effectively addressed. According to the study results, 90.7% of all surveyed students (n = 43) were satisfied with the experiential learning chatbot workshop; 81.4% of the respondents felt engaged while 81.3% of the participants reported moderate to high levels of competencies (81.3%) as result of the hands-on workshop. Almost all students surveyed (97.7%) felt that the experiential chatbot workshop had met the expected learning outcomes. Besides presenting empirical data that underscore the pedagogical usefulness of conducting an experiential Chatbot workshop in basic A.I. classes with special reference to NLP, we are trying to corroborate a conceptual model developed from learning theories and technology mediated learning (TML) models aimed at measuring the effects of a chatbot practicum on students’ engagement and motivation as potential drivers of successfully acquiring basic NLP skills and learner satisfaction. The paper provides useful practical information for instructors interested in deploying a practical chatbot workshop as effective TML tool in a tertiary educational context aimed at making learners ‘future-ready’.
format text
author SHIM, Kyong Jin
MENKHOFF, Thomas
TEO, Ying Qian
ONG, Clement Shi Qi
author_facet SHIM, Kyong Jin
MENKHOFF, Thomas
TEO, Ying Qian
ONG, Clement Shi Qi
author_sort SHIM, Kyong Jin
title Assessing the effectiveness of a chatbot workshop as experiential teaching and learning tool to engage undergraduate students
title_short Assessing the effectiveness of a chatbot workshop as experiential teaching and learning tool to engage undergraduate students
title_full Assessing the effectiveness of a chatbot workshop as experiential teaching and learning tool to engage undergraduate students
title_fullStr Assessing the effectiveness of a chatbot workshop as experiential teaching and learning tool to engage undergraduate students
title_full_unstemmed Assessing the effectiveness of a chatbot workshop as experiential teaching and learning tool to engage undergraduate students
title_sort assessing the effectiveness of a chatbot workshop as experiential teaching and learning tool to engage undergraduate students
publisher Institutional Knowledge at Singapore Management University
publishDate 2023
url https://ink.library.smu.edu.sg/sis_research/8844
https://ink.library.smu.edu.sg/context/sis_research/article/9847/viewcontent/s10639_023_11795_5_pvoa_cc_by.pdf
_version_ 1814047592147845120