Whisper In-Browser: Speech-to-Text on the Edge
REPORT_ABSTRACT
Whisper In-Browser Transcriber proves that a production-grade speech recognition stack no longer needs a server. Recorded voice or dropped audio files are decoded to 16 kHz mono PCM and transcribed by OpenAI Whisper (tiny) entirely on the visitor's own device. Inference runs inside a Web Worker via ONNX Runtime Web so the interface never stutters, and an int8-quantized ~39 MB model is fetched once from the Hugging Face Hub and cached for instant reloads.
01. OFF-THREAD INFERENCE ARCHITECTURE
The core design decision is that the model never touches the main thread. A dedicated Web Worker owns the ONNX Runtime Web session; the UI thread posts raw audio buffers across and receives decoded text events back. Because decode is asynchronous and off-thread, the interface keeps rendering at 60fps while a multi-second clip is being transcribed, and the tab never locks during model download or warmup.
02. THE DEVICE/DTYPE FALLBACK CHAIN
ONNX Runtime Web exposes multiple execution providers, and no single one works on every machine. The boot sequence walks a chain — WebGPU first, then WASM q8, then WASM fp32 — and keeps the first session that initializes successfully. The site ships Cross-Origin-Isolation headers so the WASM path can use SharedArrayBuffer-backed multi-threaded execution, which typically delivers a 3–5× speedup over single-threaded WASM.
03. MODEL DELIVERY AND CACHING
The model is onnx-community/whisper-tiny.en, quantized to int8 (~39 MB) — small enough that the entire weights blob is a single fetch. It is pulled once from the Hugging Face Hub and cached by the browser, so repeat visits transcribe instantly with no network dependency. The English-only checkpoint keeps vocabulary and decode complexity low while covering the vast majority of speech use cases.
04. THE AUDIO FRONT END
Before the model sees anything, raw audio is normalized: mic input is captured at the device sample rate and resampled down to the 16 kHz mono PCM that Whisper expects. A live waveform is drawn from the same buffer as it streams in, giving immediate visual feedback while recording. Dropped files bypass the mic path entirely and flow through the identical normalization stage, so both routes share one decode pipeline.
05. TRANSCRIPTION AND EXPORT UX
Decoded output streams in as per-segment, timestamped text. Each segment can be copied individually, the full transcript is available with one click, and a TXT export serializes the entire conversation. The whole pipeline — capture, decode, export — is a pure static SPA: no backend, no API keys, deployable anywhere a static host will serve it.