Developing an accessible learning application for sign language (C)

Hearing loss is a health issue that is affecting over 1.5 billion people worldwide, and this number is projected to increase even more to 2.5 billion by the Year 2050. In the case of children who are born deaf or suffered from hearing loss during their early development stage, statistics has shown t...

Full description

Saved in:
Bibliographic Details
Main Author: Patricks, Anastasia
Other Authors: Wesley Tan Chee Wah
Format: Final Year Project
Language:English
Published: Nanyang Technological University 2022
Subjects:
Online Access:https://hdl.handle.net/10356/157530
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-157530
record_format dspace
spelling sg-ntu-dr.10356-1575302023-07-07T19:16:20Z Developing an accessible learning application for sign language (C) Patricks, Anastasia Wesley Tan Chee Wah School of Electrical and Electronic Engineering wesleytan@ntu.edu.sg Engineering::Computer science and engineering::Software::Software engineering Hearing loss is a health issue that is affecting over 1.5 billion people worldwide, and this number is projected to increase even more to 2.5 billion by the Year 2050. In the case of children who are born deaf or suffered from hearing loss during their early development stage, statistics has shown that more than 90 per cent of these deaf children are born from hearing parents. While these children would usually learn sign language to communicate at specialised institutions, their parents might not have the necessary resources to pick up sign language themselves. This predicament often left the hearing parents at loss on how to communicate meaningfully with their children. As such, they might grow reluctant to communicate beyond necessary, which leads to deaf people feeling disconnected and isolated from their hearing family members. In this project, the author, together with two other team members, aims to address the above issue through the development of an interactive sign language learning web application targeted at the hearing family members of deaf people. The web app, named Sign2Sign, utilises an interactive 3D avatar and real-time sign recognition AI to support conversational-focused sign language lessons. These are implemented using several Microsoft Azure technologies such as Azure App Service, Azure Cosmos DB, and Azure Machine Learning, coupled with other core technologies including MediaPipe and ThreeJS. Through providing such accessible avenue to learn sign language effectively, this project aims to help bridge the communication gap between the deaf and their hearing family members. Furthermore, this project also aims to promote inclusivity for the Deaf community by encouraging more hearing people to learn sign language. This report discusses the author's contribution in the development of the Sign2Sign web app, which includes curating the sign language lessons, building the lessons database and implementing the lessons APIs, generating signing landmarks for sign recognition, as well as developing the UI for the lessons' quiz components. The completed prototype, Sign2Sign, was also submitted to the Microsoft Imagine Cup 2022 competition. It successfully achieved a World Finalist result and won in the Lifestyle category of the Asia region in the World Finals. Bachelor of Engineering (Information Engineering and Media) 2022-05-19T07:19:06Z 2022-05-19T07:19:06Z 2022 Final Year Project (FYP) Patricks, A. (2022). Developing an accessible learning application for sign language (C). Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/157530 https://hdl.handle.net/10356/157530 en A3222-211 application/pdf Nanyang Technological University
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering::Computer science and engineering::Software::Software engineering
spellingShingle Engineering::Computer science and engineering::Software::Software engineering
Patricks, Anastasia
Developing an accessible learning application for sign language (C)
description Hearing loss is a health issue that is affecting over 1.5 billion people worldwide, and this number is projected to increase even more to 2.5 billion by the Year 2050. In the case of children who are born deaf or suffered from hearing loss during their early development stage, statistics has shown that more than 90 per cent of these deaf children are born from hearing parents. While these children would usually learn sign language to communicate at specialised institutions, their parents might not have the necessary resources to pick up sign language themselves. This predicament often left the hearing parents at loss on how to communicate meaningfully with their children. As such, they might grow reluctant to communicate beyond necessary, which leads to deaf people feeling disconnected and isolated from their hearing family members. In this project, the author, together with two other team members, aims to address the above issue through the development of an interactive sign language learning web application targeted at the hearing family members of deaf people. The web app, named Sign2Sign, utilises an interactive 3D avatar and real-time sign recognition AI to support conversational-focused sign language lessons. These are implemented using several Microsoft Azure technologies such as Azure App Service, Azure Cosmos DB, and Azure Machine Learning, coupled with other core technologies including MediaPipe and ThreeJS. Through providing such accessible avenue to learn sign language effectively, this project aims to help bridge the communication gap between the deaf and their hearing family members. Furthermore, this project also aims to promote inclusivity for the Deaf community by encouraging more hearing people to learn sign language. This report discusses the author's contribution in the development of the Sign2Sign web app, which includes curating the sign language lessons, building the lessons database and implementing the lessons APIs, generating signing landmarks for sign recognition, as well as developing the UI for the lessons' quiz components. The completed prototype, Sign2Sign, was also submitted to the Microsoft Imagine Cup 2022 competition. It successfully achieved a World Finalist result and won in the Lifestyle category of the Asia region in the World Finals.
author2 Wesley Tan Chee Wah
author_facet Wesley Tan Chee Wah
Patricks, Anastasia
format Final Year Project
author Patricks, Anastasia
author_sort Patricks, Anastasia
title Developing an accessible learning application for sign language (C)
title_short Developing an accessible learning application for sign language (C)
title_full Developing an accessible learning application for sign language (C)
title_fullStr Developing an accessible learning application for sign language (C)
title_full_unstemmed Developing an accessible learning application for sign language (C)
title_sort developing an accessible learning application for sign language (c)
publisher Nanyang Technological University
publishDate 2022
url https://hdl.handle.net/10356/157530
_version_ 1772827826530025472