Auto Captions and Deaf Students: Why Automatic Speech Recognition Technology Is Not the Answer (Yet)

Published on October 27, 2020

This is an image of a woman trying to use the voice-activated search or command on a mobile phone. The mobile phone display shows a " Speak Now" with a Mic icon.

With the rapid shift to online learning due to the pandemic, many colleges and schools are relying upon automatic captions as a quick and cheap way to convert spoken words into text for deaf students in classrooms, events, and extracurricular activities.

While this type of automatic speech recognition (ASR) technology may be effective for Alexa — to ask your home device to make a grocery list or set a reminder — it is a sub-standard option in educational settings and can have costly repercussions for institutions.

The National Deaf Center on Postsecondary Outcomes (NDC) has analyzed research on ASR’s impact on deaf students and outlines how colleges can reduce barriers, adhere to recent legal rulings, and provide equitable access.

Converting Speech to Text: Human vs. Machine

Real-time captioning. Closed captioning. Live transcription. CART.

There are many terms to describe the manual process of converting spoken language and auditory information into text in real-time. No matter which term you use, these are all speech-to-text services that require the skill of trained people. For many years, speech-to-text professionals have been providing real-time captioning access for classrooms, courtrooms, arts events, live television programming, and more.

With rapid developments in technology, the use of Automatic Speech Recognition (ASR) software has grown significantly. Now there is a growing trend of using ASR in educational settings, with the assumption that automatic captions are satisfactory and cost-effective. Yet as far as ASR technology has advanced, it is still not comparable to what speech-to-text professionals provide — and deaf students are paying the price.

Automatic Captions Are Not Equitable Access

Deaf students have long battled inequities in education, including lack of access to communication. Limited access to language and communication is one of the prominent root causes of gaps in postsecondary achievement and outcomes for deaf people.

Whether classes are being hosted online or in person, deaf students are being forced to decipher errors in ASR captions, without access to the original source of information. Additionally, the expectation is that ASR allows deaf students to perform academically at the same level of their peers. This false assumption is having a negative impact and taking a unique toll on deaf students during the pandemic.

The Pitfalls of ASR

To the untrained eye, ASR may seem “good enough” when testing its application in a quiet office with a single speaker. When you introduce environmental and other factors — such as accents, female speakers, multiple or overlapping speakers in group discussion, and audio distortion — the technology has yet to be proven comparable to a trained speech-to-text professional.

Research shows ASR often does not typically include:

Another ASR pitfall: errors cannot be corrected in real-time accuracy. All of these factors are what speech-to-text professionals can naturally adjust.

Recent research and legal rulings also emphasize ASR’s shortcomings and the concerns educational institutions should have with using the technology on a widespread basis on campus. They include:

Using ASR further deprives deaf students of their access to language, with intangible repercussions from the inability to attain a quality education. Various accessibility laws exist to ensure that schools, colleges, and universities fulfill their obligations in providing equitable access to their programs and services. For deaf people, the standard of effective communication measures whether or not an accommodation removes barriers to communication.

The True Cost of ASR

So, why are institutions choosing ineffective technology over trained human professionals? The main reason: cost.

The pandemic has certainly caused tighter budget constraints and pressures on educational institutions. When required under the Americans with Disabilities Act (ADA), claiming that an accommodation is an undue financial burden is often unsuccessful for educational institutions. To demonstrate an undue burden, all the resources available to pay for the accommodation must be considered, not just the resources of a single academic department or disability services office.

If the decision to use ASR for real-time access is purely driven by cost, consider the high costs of legal ramification, reputation, and the burden on deaf students to fight for access to education.

What If a Deaf Student Prefers ASR?

If a deaf student specifically requests the use of ASR as an accommodation, it is important to explain the risk of errors and misunderstanding, as well as discuss options to ensure effective communication is not compromised.

When applied appropriately to other scenarios, ASR can have some benefits:

  • It can be a stop-gap tool for one-on-one situations when deaf and hearing people need to communicate.

  • Use ASR to create transcripts that can be edited to write captions for produced videos. 

  • ASR may be useful for people with physical disabilities, in order to type independently.

  • Some students also like to use ASR apps as note-taking tools.

Deaf students are entitled to equal opportunities to achieve postsecondary success. Ensuring equal access means working with each deaf student on a case-by-case basis to determine what accommodations are necessary and by leaning into the ADA standard of effective communication access. Consider this objective when working with deaf students who require real-time speech-to-text access and why ASR is not up to the task.

Do you have further questions about the variety of accommodations used by deaf students? Contact the NDC | Help team for free one-on-one support.

Additional Resources:

Save This

Other News Items

Screenshot 2023-06-02 at 10.47.21
June 2, 2023
...Idaho Educational Services for the Deaf and the Blind (IESDB) converted an old school bus into a food truck, “The Ravenous Raptor,” managed by students aged 14-26. The program helps students learn essential live-enhancing skills and develop workplace readiness skills that may not otherwise be available to them....
May 10, 2023
Conferences and Events
...On March 16th, NDC hosted a Deaf Centered Interpreting Webinar wherein Dr. Thomas Holcomb, PhD, and Aundrea Love, NIC, aimed to answer the question “How can interpreting become more deaf-centered?” by proposing a deaf-centered interpreting model. This model emerged from an ongoing effort to improve access and participation for deaf participants in interpreted sessions. This presentation focused on the changing view of the delivery model from merely providing access to increasing the quality of interpreting work by removing barriers that have made it difficult for deaf participants to engage fully in interpreted sessions....
Screen Shot 2023-02-24 at 10.26.29 AM
April 19, 2023
...In response to the continued effects of the interpreter shortage, NDC has remained focused on creating opportunities for disability services professionals to connect both with us and with one another as they continue to provide support services to deaf students. On March 30, 2023, NDC hosted a second session of Exploring Interpreter Shortages in Postsecondary Settings Listening Session, wherein participants discussed a variety of potential solutions....

Need Help?

Fill out this form to get help from the NDC team.  Can’t see the form below? Click here to contact the NDC team.