Zoom: AI Sign-to-Speech Mode

Inclusive Design

Oct 10

Written By Jin Kim

Concept Design – Mockup

**Every Sign. Every Voice. One Table.**

**Beyond subtitles. Beyond barriers.**
Zoom’s **AI Sign-to-Speech Mode** translates signing into natural speech and voice in real time—so every participant can lead, share, and connect equally. Emotional tone, voice personalization, and multilingual support bring true inclusivity to life.
**Where everyone leads together.**

The Origin of the Project — Silence in a Digital World
This project began from a small but unforgettable moment.
During a Zoom class in 2021, one of my classmates had partial hearing loss. On campus, we communicated easily—through voice, gestures, and expression. But online, everything changed. With cameras off and tiny video boxes, lip movements disappeared, and gestures were hard to see. Suddenly, she was left behind in a space that was meant to connect us.
If even those with partial hearing loss struggled, what about those who rely entirely on sign language? The digital world, instead of leveling the field, had quietly built new walls. That day made me realize that accessibility isn’t just about listening—it’s about being able to speak, lead, and be heard.

🫱🏽‍🫲🏼 Empowering Deaf Voices

**Problem**

Imagine sitting in a meeting, full of ideas, but unable to make your voice heard. That’s the reality many Deaf professionals face in video conferences. A **2022 National Deaf Center study** shows Deaf individuals experience far lower employment in leadership and professional roles. They can see everyone else, but their signed thoughts remain invisible.
Remote work has only deepened the gap. A **Pew Research Center study** found Deaf remote workers are far more likely to experience communication breakdowns—especially in fast-paced group discussions. Yet even in 2024, Zoom supports only captions and interpreter assignments. **No AI translation.** This forces Deaf users to depend on intermediaries, creating delays that disrupt the flow and relegate them to passive roles.
The result? Their ideas often go unheard—and leadership opportunities slip away.

**Insight**

Zoom connects over 300 million people every day. But without AI-based sign translation, millions are still left out of real-time collaboration.
**Accessibility shouldn’t stop at comprehension; it should empower contribution.**
True inclusion means letting Deaf professionals **ask questions, pitch ideas, and lead discussions** without waiting for interpretation. By bringing AI-driven sign-to-speech into the platform, Zoom can move beyond “accessible meetings” to **meetings where every voice is equal.**

Solution — Zoom AI Sign-to-Speech Mode 🤖

A built-in, two-way translation system enabling instant communication between Deaf and hearing participants.

🎧 **1. Hearing → Deaf (Spoken → Signed)**

**Sign Language Avatar + Captions**
Spoken words appear both as text and through an expressive signing avatar.
**Ambient Sound Captions + Tone Visualizer**
Non-verbal sounds appear as captions—(👏 Applause) (😂 Laughter)—while voice tone is shown with subtle icons (🔥 passionate / 💧 calm).
**AI Emotion Overlays**
Group reactions appear as emojis beside each participant—😂 👏 😮 👍—so Deaf users instantly sense the emotional rhythm of the room.

👐 2. Deaf → Hearing (Signed → Spoken)

Adaptive Voice + Captions
AI translates signing into natural speech and captions in real time.
Personal Voice Profile
Deaf users choose their preferred tone—Professional Male, Casual Female, Neutral AI—deciding “how I want my voice to sound.”
Emotion Carried Voice
AI mirrors emotion, pace, and emphasis—turning signs into voices that feel alive.
Auto Phrasing Polisher
Converts sign grammar into fluid speech.Example: “Good. Can do.” → “Yes, I can do that.” (or) “I’d be happy to take that on.”
Live Translation Toggle
Real-time multilingual support (ASL → English + Korean/Spanish captions) for global teams.