Supporting novices author audio descriptions via automatic feedback

• Main Authors: NATALIE, Rosiana, TSENG, Joshua Shi-hao, KACORRI, Hernisa, HARA, Kotaro
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: accessibility; AI-supported writing; individuals with disabilities; assistive technologies; Artificial Intelligence and Robotics; Databases and Information Systems; Graphics and Human Computer Interfaces
• Online Access: https://ink.library.smu.edu.sg/sis_research/8318; https://ink.library.smu.edu.sg/context/sis_research/article/9321/viewcontent/Supporting_Novices_Author_Audio_Descriptions_via_Automatic_Feedback.pdf
• Institution: Singapore Management University

Audio descriptions (AD) make videos accessible to those who cannot see them. But many videos lack AD and remain inaccessible as traditional approaches involve expensive professional production. We aim to lower production costs by involving novices in this process. We present an AD authoring system that supports novices to write scene descriptions (SD)—textual descriptions of video scenes—and convert them into AD via text-to-speech. The system combines video scene recognition and natural language processing to review novice-written SD and feeds back what to mention automatically. To assess the effectiveness of this automatic feedback in supporting novices, we recruited 60 participants to author SD with no feedback, human feedback, and automatic feedback. Our study shows that automatic feedback improves SD’s descriptiveness, objectiveness, and learning quality, without affecting qualities like sufficiency and clarity. Though human feedback remains more effective, automatic feedback can reduce production costs by 45%.