Whisper (STT)

Overview

Faster-Whisper provides local speech-to-text (STT) capabilities for our voice assistant. It is optimized for CPU performance, allowing near real-time transcription on our homelab server.

Ports

Service	Port	Protocol	Description
Wyoming	10300	TCP	Wyoming protocol endpoint

Configuration

The service is configured in modules/nixos/services/whisper.nix using the base model balanced for low latency on the host CPU.

services.wyoming.faster-whisper.servers."es" = {
  enable = true;
  model = "base";
  language = "es";
  uri = "tcp://127.0.0.1:10300";
  device = "auto";
};

Model Tuning History

The Whisper model was iterated several times to find the sweet spot between latency and accuracy for live Assist in Spanish:

Attempt	Model	Result
First	medium	Good Spanish transcription, but too slow for live Assist.
Revert	small	Better latency, acceptable accuracy.
Second	base	Lowest latency for live Assist. Recordings are used to evaluate if small is worth it.

The current base model is the default. If transcription quality degrades, use the assist-recorder metrics (timing + audio normalization) to decide whether to bump back to small.

Troubleshooting

Restarting the service

sudo systemctl restart faster-whisper-es

Checking Logs

sudo journalctl -u faster-whisper-es -f