Project / 01

Interpretation
without the distance.

VoxBento is a browser-first, low-latency interpretation platform for virtual and hybrid events. It gives interpreters, listeners, organizers, and speech-processing services distinct jobs instead of asking one application to carry the whole event.

The problem

A live event should not have a language boundary.

Traditional interpretation setups depend on dedicated booths, specialist hardware, and difficult media routing. VoxBento moves the interpreter booth and listener experience into the browser, while preserving a direct media path built for responsive audio.

The design goal

Keep every concern in the right place.

Browser-based interpreter publishing and listener playback
Low-latency audio transport independent of application logic
Live captions, translation, and optional spoken output
Event, room, booth, and role-aware participation
Recovery paths when a WebRTC connection is unavailable

Architecture

The live audio path stays out of Python.

MediaMTX transports audio. FastAPI coordinates people, state, and processing. Jitsi provides floor monitoring. The boundaries are deliberate.

Live audio path

Interpreter browserWebRTC / WHIPMediaMTXWebRTC / WHEPListener browser

Direct media path: Python coordinates it but does not process audio bytes.

Interpretation pipeline

RTSP / FFmpegSpeech-to-Textfinal segmentsTranslationText-to-Speech

Captions, translations, and synthesized output are distributed by WebSocket.

FastAPI · async coordinationWebSocket booth statePostgreSQL / SQLiteJitsi floor monitoringDocker · Linux · Nginx

Pipeline / interpreter to listener

One stream, two complementary paths.

01 / Publish

Interpreter microphone

The interpreter booth creates a browser RTCPeerConnection, adds the microphone track, and publishes through WebRTC/WHIP.

02 / Transport

MediaMTX

MediaMTX receives the WHIP stream. Its WHEP endpoint serves active interpretation audio to listener browsers; its RTSP output provides a separate processing feed.

03 / Listen

Listener browser

The listener selects a language and opens a WHEP session. Browser playback receives the live stream, with an HLS fallback available when WebRTC cannot connect.

Speech pipeline

Speech becomes captions, translation, and audio.

STT / continuous

Transcription workers

For each enabled booth, a worker uses FFmpeg to read the MediaMTX RTSP stream. Streaming and chunk-based providers produce partial and final caption segments for connected clients.

Translate / on demand

Final segments

When a final transcript segment is stored, enabled target languages can be translated asynchronously. Translation is lazy: it only runs for languages with interested TTS listeners.

TTS / on demand

Scheduled playback

Translated segments can be synthesized server-side and sent in a binary WebSocket bundle. The browser audio scheduler plays the PCM output in sequence.

Speech-to-text

A worker per active booth.

VoxBento supports local faster-whisper and configured cloud providers including OpenAI, Deepgram, NVIDIA Riva, and ElevenLabs. Deepgram follows a streaming path; local and other chunk-oriented providers process short audio chunks. The worker keeps per-booth session state, tracks gaps and latency, retries connection failures with exponential backoff, and broadcasts captions over WebSocket.

Text-to-speech

Optional spoken interpretation.

The TTS worker synthesizes translated floor content with Deepgram Aura or the local Supertonic ONNX provider. Both normalize output as 24 kHz mono PCM. TTS is server-side and on demand, not a continuously running process; each translated segment is synthesized and delivered with its text metadata for synchronized client playback.

Translation

Preserve the difference between a partial thought and a final statement.

Partial transcription is useful for immediate captions. Finalized segments become durable records and the trigger for translation. If the source language matches a target language, VoxBento bypasses translation and broadcasts immediately.

Configured provider backends include OpenAI, Anthropic, Gemini, OpenRouter, and a local fallback. Per-language semaphores limit concurrent translation requests so a busy room does not overwhelm a provider.

Coordination and reliability

State is fast when it is transient, durable when it matters.

Transient state

BoothRegistry

Active interpreter status, chat, and booth presence live in an in-memory registry protected by per-booth asyncio.Lock instances.

Durable state

SQLAlchemy + Alembic

SQLite supports development and PostgreSQL supports production persistence for events, rooms, booths, memberships, transcripts, and webhook deliveries.

Handoff

Coordinator control

A coordinator changes the active interpreter through WebSocket state. The old WHIP publisher disconnects, MediaMTX accepts the new publisher, and listeners retain their endpoint.

Redis is not part of the VoxBento architecture. In-process coordination and database-backed durable records avoid an unnecessary cache or queue dependency for the current single-instance design.

Access and operations

A real event needs more than an audio stream.

Roles and access

JWT-backed roles distinguish super admins, event admins, coordinators, interpreters, and listeners. Permissions are checked in routes and WebSockets; only authorized roles can control active interpreters or mutate booth state. Single-use invite tokens provide time-bounded booth access.

Deployment

Docker Compose brings together the FastAPI portal, MediaMTX, an optional Chromium floor bot, and the self-hosted Jitsi services. API keys are encrypted at rest with Fernet, and Alembic migrations initialize persistent schema changes.

Future scope / personalization

A transcript can be personal without changing the shared record.

Today, transcript and translation state is scoped to events, rooms, and booths. A future personalization layer could let a listener save preferred languages, choose a supported TTS voice, control caption appearance, and manage transcript download or sharing preferences.

The natural implementation is a separate user-preference record and API surface, linked to the existing user and event-membership model. It should not alter the canonical transcript: personal display, language, and voice choices belong at the edge, while the source record remains attributable and auditable.

Work across FOSSASIA

The project is strengthened by upstream engineering.

VoxBento is the GSoC centerpiece, but the surrounding contribution record also includes platform work, an evolving business plugin, and community automation.

01 / VoxBento

Real-time interpretation infrastructure

Browser WHIP publishing, MediaMTX WHEP delivery, RTSP worker feeds, FastAPI coordination, live captions, translation, TTS, authentication, and role-aware booths form the project’s core. The implementation keeps media transport separate from application logic so each can fail and scale on its own terms.View VoxBento upstream ↗

02 / Eventyay Business

Tiers and entitlement foundations

Active work adds versioned tier configuration, monthly and annual pricing, typed capability entitlements, Free-tier assignment, usage records, and organizer plan visibility. Billing webhooks, charge collection, upgrade paths, and analytics remain work in progress, so this is represented as an evolving plugin rather than a finished billing product.View Eventyay Business upstream ↗

03 / Eventyay Core

Images, imports, and exports

Image handling improvements validate uploaded images, preserve intended orientation with EXIF transpose, remove unnecessary metadata, resize with aspect-ratio preservation, and use LANCZOS resampling. Event data work covers schedule and speaker import paths, export reliability, and the surrounding validation needed to move structured event data safely.Read the LANCZOS image optimization PR ↗

04 / Star-Me

FOSSASIA repository discovery automation

Star-Me is a browser-console script that uses the signed-in GitHub session to enumerate paginated FOSSASIA repositories and guide the user through starring them. The script handles popup blocking, polling for GitHub’s changing Star control, retry and timeout behavior, cross-origin failures, and rate-limit prompts. The broken selector and hanging behavior were fixed to keep the flow usable against the current GitHub interface.Read the Star-Me script fix ↗

Technology map

Interpretationwithout the distance.