Doctoral Student Quon Wins Microsoft Epic Challenge for ASL-Inspired Computer Programming Interface

Dec. 17, 2021 – Imagining a more inclusive future inspired Beverly Quon, UC Irvine doctoral candidate in computer engineering, to develop her passion project called Keyboardless, ASL-inspired Programming Interface (KAPI). The American Sign Language-inspired programming interface makes computer programming more accessible for people who have difficulty typing or using a keyboard.

KAPI employs emerging low-code platforms (PyCaret, Github Co-Pilot), computer vision and machine learning so that people can use ASL to code with only a webcam. KAPI means “monkey” in Sanskrit and has become the project’s logo, a cute, friendly creature meant to welcome anyone into computer programming.

Focusing on ASL and the use of gross motor skills instead of fine motor skills required for typing on a keyboard lets those whose primary language is ASL participate in this high-tech field. “Eliminating the need for a keyboard removes a barrier to entry for computer programming,” explained Quon. KAPI would also help anyone who has difficulty typing or using a keyboard, including children, elderly and disabled individuals.

This innovative approach won Quon and collaborator Asha Makwana, a Northeastern University graduate student in analytics, the 2022 Microsoft Imagine Cup Epic Challenge for the Americas region. The two met at the 2021 Grace Hopper celebration. They quickly realized they shared an interest in education technology (EdTech) and had even entered Microsoft’s Imagine Cup individually. They partnered to develop KAPI as a way to work toward their mutual dream of innovating EdTech.

Neither knew sign language at the time, so in order to gain valuable insights into the deaf community, they reached out to experts on LinkedIn. “It really shocked us when we messaged subject-matter experts, and they said yes,” Quon said. “Not only did they say yes, but they gave us an abundant amount of information. They were very encouraging and passionate about KAPI. They loved the idea, our passion and our goal of promoting inclusivity.”

“We didn’t just create a project,” added Quon. “The fact that we were looking for feedback showed that we really cared about the people we were creating it for. I think we won because of two things: identifying that the timing was right given the emerging open-source technologies and actively searching for critical feedback from the community that would benefit from this.”

Microsoft’s 2022 Imagine Cup is a global student developer competition in which teams propose an original technology solution in one of four categories: earth, education, health or lifestyle. For the first time, teams had the option of submitting their project to the Epic Challenge first to receive preliminary feedback on their ideas from professional judges. To enter, teams submitted a three-minute project pitch and proposal. The winning teams received $1,000 to continue developing their projects and advance to the Imagine Cup World Finals in May 2022.

Quon has advanced to the semifinals, taking place in late January, where competitors have to submit a working prototype. She plans to market KAPI as a software licensing and delivery model in which software is licensed on a subscription basis and is centrally hosted with the goal of bringing expertise and technical support into immersive programming by reaching out to businesses that rely on low-code platforms.

“Businesses who adopt KAPI will help promote diversity and inclusion by broadening their hiring scope to those who prefer to communicate via sign language,” explained Quon. “Regardless of the competition, we hope that this kind of reinvigorates and pushes more research toward sign language and other technology that promotes accessibility.”

– Tonya Becerra