All May Hear is a web-based service designed to provide near real-time interpretation of spoken language during live events. It enables event organizers to offer live interpretation to multilingual audiences through a simple web interface accessible on any device. The software is primarily designed for meetings held by the Church of Jesus Christ of Latter-day Saints, in particular Sacrament meetings.
Key capabilities:
- Enables live translation of audio during events
- Accessible through a web interface
- Supports both text and audio output for translated content
Key Components
Recorder application
The Recorder application is an interface designed for capturing audio and translating it into text and audio. The recorder interface provides a focused experience for audio capture with minimal distractions.
Key features:
- Receives audio input from microphones, headsets, or external sound systems
- Allows the recorder to start and stop audio capture independently
Reader application
The Reader application is a simple interface for attendees to access translations or the raw audio feed during a meeting.
Users can view the translated text and select their preferred language from the languages configured for the room. The reader provides visual feedback when new translations are available and can optionally play audio versions of translations.
Since the Reader application can pass through the unfiltered audio feed from the sound system attached to the Recorder. This allows the Reader to act as an assistive listening device for those whose hearing aids are connected to their mobile device.
Key features:
- Displays live translations in the user's selected language
- Allows users to switch between available interpretation languages
- Notifies users when new translated content is available
- Supports optional audio playback of translations using text-to-speech technology
- Provides a clean, distraction-free viewing experience optimized for reading translations
- Optionally displays the source language transcription alongside the translated text for those trying to learn the source language.
How All May Hear Works
The system uses AI to process and translate spoken language in real time using automatic transcription and translation.
Auto-Translate Mode
The system automatically transcribes and translates speech continuously. When audio capture is started, the system:
- Detects when someone is speaking using voice activity detection
- Transcribes speech to text in real time using AI
- Intelligently waits for complete thoughts or sentences before translating to avoid translating incomplete phrases
- Translates the transcribed text into the languages that readers need
- Sends translations to readers in real time via their web browsers
Translation Process
The following steps occur automatically:
- Audio Transcription: Audio is continuously transcribed to text in the source language using AI (OpenAI Realtime API)
- Completeness Detection: The system checks if the transcribed text represents a complete thought or sentence before proceeding
- Translation: Complete text segments are translated into the languages that readers need. The system only translates to languages that are both configured for the room and requested by connected readers, making it efficient and cost-effective
- Delivery: Translated content is sent to readers in real time via their web browsers
- Audio Output (optional): Translated content can be output as speech using text-to-speech (TTS) technology, either using local device voices or high-quality cloud-based voices