Hardware matrix

hal0 targets AMD APU/GPU plus the AMD XDNA NPU, with a CPU fallback that is always available. NVIDIA GPUs are also supported via an experimental CUDA path (see below). This page lists what runs on each accelerator and the order hal0 prefers them in.

Hardware targets

Target	Backend / runtime	What runs there
AMD GPU (ROCm)	llama.cpp ROCm / Qwen3-TTS	chat / embed / rerank LLM inference (highest tps); multilingual TTS
AMD GPU (Vulkan)	llama.cpp Vulkan	chat / embed / rerank LLM inference (broad-compat)
NVIDIA GPU (CUDA)	llama.cpp CUDA — experimental	chat / embed / rerank LLM inference
AMD XDNA NPU	FastFlowLM (FLM)	chat + speech-to-text + embeddings (one process)
CPU	llama.cpp / ONNX	fallback LLM inference; Kokoro TTS

The reference platform is AMD Strix Halo: an 8060S-class iGPU, an XDNA NPU, and a unified-memory pool the GPU shares with the host via GTT. hal0’s profile flags are bench-tuned for that target, but the GPU/Vulkan and CPU paths run on any modern Linux AMD system.

Hardware monitoring dashboard for GPU, NPU, and system memory.

Backend availability order

hal0 probes the host and advertises the backends it can actually run, in this order:

1. NPU — only when an XDNA NPU is present and the FLM toolbox image is already pulled locally.
2. GPU (Vulkan) — whenever a GPU is detected (every modern Linux GPU has Mesa Vulkan).
3. GPU (ROCm) — only when the detected GPU is an AMD GPU with compute support (compute_capable).
4. CPU — always reachable; the guaranteed fallback.

Backend id	Label	Provider	Multiplex	Advertised when
npu	NPU	flm	yes	XDNA present + FLM image pulled
gpu-vulkan	GPU (Vulkan)	llama-server	no	any GPU detected
gpu-rocm	GPU (ROCm)	llama-server	no	AMD GPU with ROCm compute support
cpu	CPU	llama-server	no	always

The NPU is multiplex: a single flm serve process answers chat, STT, and embeddings at once, so one NPU slot can back three capabilities.

Note

gpu-cuda is not part of this auto-advertised list — /api/backends (the dashboard’s backend-availability footer) doesn’t probe for CUDA yet, so an NVIDIA GPU still only shows up here as gpu-vulkan. gpu-cuda is chosen by the installer’s hardware recommender on NVIDIA hosts where nvidia-container-toolkit (CDI) is detected, or set explicitly with device = "gpu-cuda" in a slot TOML.

The Inference Engine slot inventory — here 4 iGPU · 3 FLM — with the shared iGPU GTT carve-out above it.

NVIDIA / CUDA is experimental

hal0’s device enum is {gpu-rocm, gpu-vulkan, gpu-cuda, cpu, npu} — gpu-cuda runs NVIDIA GPUs through the upstream ghcr.io/ggml-org/llama.cpp:server-cuda image via CDI GPU passthrough (nvidia-container-toolkit required). It’s experimental: unbenched, not part of the AMD Strix Halo toolbox family, and not yet part of the /api/backends auto-probe above. Without nvidia-container-toolkit, an NVIDIA GPU still only surfaces through the generic Vulkan path (gpu-vulkan); there is no ROCm path for NVIDIA hardware. hal0 is built and tested on Linux x86_64.

Multi-GPU hosts

hal0’s hardware probe enumerates every detected GPU additively (index + PCI bus id), for both AMD and NVIDIA hosts. A slot can pin itself to one GPU via the gpu_index field — see Devices, providers & profiles for the per-backend visibility-env mapping. Leaving gpu_index unset exposes all GPUs to the slot, unchanged from prior single-GPU behaviour.

NPU host requirements

The NPU path runs FastFlowLM in a container, but the host must provide the XDNA driver and firmware:

• the amdxdna kernel driver (in-tree on kernel ≥ 6.14, or via the amdxdna-dkms package on kernel ≥ 6.10), and
• NPU firmware ≥ 1.1.0.0.

Note

The NPU host driver install is Ubuntu / apt-only — the FastFlowLM driver ships as a .deb. The FLM inference itself runs in the pinned flm toolbox container; only the host-side driver is distro-specific.

See also

• Devices, providers & profiles — the device → profile → image matrix.
• NPU & power REST routes — /api/npu, /api/backends, /api/stats/hardware.