ASL + AI = Learning and Communication (2nd place winner, hackathon)
Rafeed Arian ( Graduate -Computer Science ), Hoang Nguyen ( Undergraduate - Computer Science ), Khanh Le ( Undergraduate - Comptr Info Systems ), Ansh Tiwari ( Undergraduate - Computer Science ), Joan Emmily Bayega ( Graduate - Computer Science )
What's the Problem?
For elementary school students who use sign language, expressing thoughts and ideas can sometimes be a challenge. That's why we're decided to work on a project aimed at making communication easier for everyone.
Our project focuses on creating a simple yet effective tool that translates hand sign language into text and vice versa. By doing so, we hope to foster better understanding and connection between students and teachers, helping the students learn better and help the teacher convey the material more effectively.
But First...
How did our team get here?
At first, our team came up with a range of ideas. Some looked at using generative AI to create scenarios and questions; others wanted to use the technology to assess learning progress or find a way to prolong learning retention. After idea prioritization, we decided that an AI-powered sign language interpreter would be the most effective way to help students communicate.
Elementary school students who live and learn with disabilities could face significant challenges in studying, comprehending the alphabet, and effectively communicating with teachers and friends.
These roadblocks can be a barrier for academic progress and social development. For our team, this meant that there is an urgent need for solutions to facilitate learning for these students by empowering them to connect with others in the classroom.
Defining the problem for a better solution
We decided our solution would be a program that translates text to American Sign Language (ASL) and vice versa. Our team was focused on helping elementary students who are deaf and hard of hearing, especially those who might be struggling with reading comprehension or being exposed to the alphabet for the first time. Having a program that translated ASL to text would be a step towards closing this gap in communication.