This app is designed for deaf people — for situations where speech occurs, but hearing can no longer capture it. A conversation at a table, an announcement, a phone call.

What already works today

Live captions — what is spoken appears as text on the screen in real time
Typing→Speech — typed text is spoken aloud, for the opposite direction of the conversation
Conversation Transcript — an entire conversation is transcribed, allowing for reading back later instead of just listening in the moment

Three of the four planned modes are real and functional, not a mockup.

What is still pending

The fourth mode, Noise Alarm — detecting when, for example, a doorbell or a smoke detector goes off, and passing this on as a visual notification — is currently still a placeholder. The audio recognition model required for this (YAMNet) has not yet been integrated.

And one topic is still completely missing, which I am aware of: Sign Language. For many deaf people, it is their actual native language, not the text on a screen. Subtitles and transcripts are a start, but not a replacement.

Dabei geht es nicht nur um die Richtung „gesprochene Sprache → Text für den Gehörlosen“. Genauso wichtig ist die umgekehrte Richtung: Wenn jemand in Gebärdensprache antwortet und das Gegenüber sie nicht versteht, wird die Barriere für den Hörenden genauso real. Das Handy müsste also auch andersherum übersetzen können — Gebärde zu Sprache oder Text, live. Das ist technisch die deutlich größere Herausforderung, gehört aber auf die Liste. Ehrlich als offen markiert statt stillschweigend übergangen.

This series: Part 1 · Part 2 · Part 3 — Vision