Speech to text and text to speech

hal0 exposes two voice directions through OpenAI-compatible endpoints:

• Speech-to-text (POST /v1/audio/transcriptions) — whisper-v3:turbo served by FastFlowLM (FLM) on the XDNA NPU, co-loaded with the chat model in one FLM process (the “NPU trio”: chat + STT + embed).
• Text-to-speech (POST /v1/audio/speech) — served from the single canonical tts slot, which can run either engine: Kokoro-82M ONNX on CPU, or Qwen3-TTS CustomVoice on the GPU (ROCm, native gfx1151). The voice.tts capability picker is an engine switch — choosing a device swaps which provider the tts slot runs, with no separate slot to configure.

Both are children of the voice capability: voice.stt maps to the stt slot and voice.tts maps to the tts slot.

Enable the voice capability

Voice is opt-in. Enable each direction from the dashboard’s capability picker, or by POSTing a partial selection to the capability API. The body accepts any subset of { backend, provider, model, enabled }:

Speech-to-text (NPU, FLM + whisper-v3:turbo)

curl -X POST http://localhost:8080/api/capabilities/voice/stt \
  -H 'Content-Type: application/json' \
  -d '{"enabled": true, "backend": "npu"}'

Text-to-speech (CPU, Kokoro)

curl -X POST http://localhost:8080/api/capabilities/voice/tts \
  -H 'Content-Type: application/json' \
  -d '{"enabled": true, "backend": "cpu", "provider": "kokoro"}'

Text-to-speech (GPU, Qwen3-TTS)

curl -X POST http://localhost:8080/api/capabilities/voice/tts \
  -H 'Content-Type: application/json' \
  -d '{"enabled": true, "backend": "gpu-rocm", "provider": "qwen3tts"}'

The response is { "ok": true, "selection": { ...current selection... } }.

Note

STT runs on the XDNA NPU via the FLM container — not on CPU. There is no CPU whisper.cpp backend in hal0. TTS runs from the single tts slot on either engine: Kokoro (CPU, ONNX) or Qwen3-TTS (GPU, ROCm) — picking a device in the voice.tts picker swaps the engine in place, it does not add a second slot.

The voice.tts engine switch

voice.tts isn’t a model picker in the usual sense — it’s a switch between two engines that both serve the same tts slot and the same POST /v1/audio/speech contract:

Device	Provider	Slot profile	Runs on
cpu	kokoro	tts	CPU (ONNX)
gpu-rocm	qwen3tts	tts-qwen3	Strix Halo iGPU (ROCm, native gfx1151)

Selecting a device rewrites which profile/provider the tts slot loads — the slot itself doesn’t move. Qwen3-TTS weights are operator-staged (not pulled through the hal0 registry) and the slot mounts a writable cache directory for the MIOpen kernel DB and HuggingFace codec/tokenizer cache.

How the NPU STT path works

Enabling voice.stt with backend=npu does not spawn a standalone process. It drives the FLM trio — one flm serve anchor process serving chat, transcription (whisper-v3:turbo), and embeddings together. The orchestrator toggles the anchor’s [npu].asr flag (which renders as --asr 1 on the FLM command line) and writes a type=transcription slot record for dispatch gating. The anchor is not auto-restarted — the response carries pending_reload: true, and the change only takes effect once you reload the FLM anchor yourself (Settings → NPU’s restart control, or hal0 slot restart <anchor-slot>). NPU transcription requires the FLM chat anchor to be loaded.

Transcribe audio (speech to text)

Send a multipart upload with the audio file and a required model field. The raw multipart bytes are forwarded unchanged to the upstream:

curl -X POST http://localhost:8080/v1/audio/transcriptions \
  -F 'file=@recording.wav' \
  -F 'model=<your-stt-model>'

The model form field is required; omitting it returns 400 (request.missing_model) rather than a misleading 404.

Synthesize speech (text to speech)

Send a JSON body with model and input. voice, speed, and response_format are optional — see request defaults below for what happens when you omit them:

curl -X POST http://localhost:8080/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "<your-tts-model>",
    "input": "Hello from hal0.",
    "voice": "<voice-id>"
  }' \
  --output speech.wav

As with transcription, model is required — a missing or empty model returns 400 (request.missing_model).

Note

Kokoro weights are operator-staged on the configured model store; Qwen3-TTS weights are likewise operator-staged (not pulled through the hal0 registry). Both engines run as containerized servers managed by hal0 as the tts slot — no manual container management needed — and both implement POST /v1/audio/speech, GET /v1/models, and GET /health.

TTS request defaults and live voice list

The tts slot can carry persisted request defaults that /v1/audio/speech seeds into a request whenever the body omits the matching field — an explicit value in the request always wins:

• default_voice — voice id injected when the request omits voice. When unset, the request falls through to the serving engine’s own default (Kokoro’s own default is af_bella).
• default_speed — playback speed (0.25–4.0) injected when the request omits speed. When unset, the engine default (1.0) applies.
• default_response_format — one of mp3 / wav / opus / flac / pcm, injected when the request omits response_format. When unset, the engine default (mp3) applies.

These are set from the dashboard’s Settings → Voice panel (which PUTs the tts slot config) and take effect immediately — no container restart. Slot selection for the seed prefers the tts slot bound to the requested model, then the per-type default tts slot, then the seeded tts slot name.

The dashboard’s voice picker also reads a live voice list from GET /api/slots/tts/voices, which proxies the serving engine’s own GET /v1/audio/voices (both Kokoro and Qwen3-TTS expose it). If the tts slot is cold or unreachable, the route fails soft with { "voices": [], "source": "offline" } and the picker falls back to a built-in Kokoro seed list; a previously-saved voice that’s missing from the live list stays selectable.

Note

Open WebUI’s Call mode is separately prewired (via its own env config, not the tts slot defaults above) to request kokoro-v1 with voice af_heart. That’s independent of what default_voice on the tts slot resolves to.

See also

• Capabilities and profiles — the capability overlay and the FLM NPU trio.
• Manage slots — the stt and tts slots.
• OpenAI-compatible API — the full /v1 surface.