Voice Activity Detector
A Voice Activity Detector (VAD) answers the cheapest question in the speech pipeline: is there a human voice in this frame of audio? It outputs a per-frame speech probability rather than text — no transcription, no speaker identity (that is Speech to text inference and Diarization). Because it is so cheap, a VAD is placed in front of expensive stages and used to gate, segment, and trim: skip silence instead of decoding it through ASR, cut long audio into chunks at natural pause boundaries, or keep a streaming pipeline idle until speech actually starts.
Two families: energy/heuristic detectors (ffmpeg silencedetect, WebRTC VAD) measure signal level and zero-crossing statistics — fast, but fail on background noise and music. Neural detectors run a tiny model over each frame and are robust to noise; the de-facto standard is silero-vad (OSS) — a ~2 MB model processing a 30+ ms chunk in <1 ms on a single CPU thread, supporting 8 kHz and 16 kHz audio, shipped for PyTorch and ONNX runtimes (MIT repository).
Where it runs in this homelab
Inside voiceapi (Parakeet STT, :5092): the upstream service’s chunker.py runs Silero VAD on the ONNX backend (no torch) to auto-chunk uploads at speech-pause boundaries. Live configuration: speech-probability threshold=0.5, min_silence_duration_ms=400, speech_pad_ms=120 — chunks target a configurable length but always cut on detected silence, so words never straddle a chunk edge. Upstream’s own comment frames the win: this is the “intelligent chunker” the baseline attempted with ffmpeg silencedetect, but ~10x faster and without subprocesses. It replaced exactly that ffmpeg heuristic in an earlier version of the service.
This matters for CPU-bound STT: one long decode of 5 minutes of mostly-silence brain-dump audio wastes the same compute as speech, while VAD-chunked decode only pays for actual speech — one of the reasons Parakeet reaches 20–30x real-time on CPU here (see Speech to text inference).
Adjacent concepts
- SAD vs diarization: speech activity detection (is anyone speaking) is the VAD’s job; who is speaking is diarization. In the Voice Arena DER benchmark, Nemotron-3-Diarization was scored with system-generated SAD and no boundary collar — i.e. the diarizer’s own VAD-style front end, not hand annotations (see Diarization).
- Streaming ASR: VAD marks utterance end in online pipelines (end-of-utterance detection), deciding when a partial hypothesis should be finalized.
- Energy detectors remain the right choice where a neural model is too heavy — embedded wake-word devices, or gating a recorder.