Deployment contract

llama.cpp / GGUF standalone

Use the nine v0.2.0 release packages or source builds to run FunASR GGUF models on CPU, Vulkan, CUDA, and edge devices.

Start deployment Review benchmark method

Maturity: Production verified
FunASR: runtime-llamacpp-v0.2.0
Runtime: llama.cpp@803b7fca
Verified: 2026-08-11

Workload boundary

Decide whether it fits your production constraints

Good fit

No Python ML environment
Desktop applications and offline edge deployment
CPU, Vulkan, or CUDA packages for Linux, macOS, and Windows

Not a fit

vLLM-style large GPU batch scheduling
A universal prebuilt CUDA package for an unverified GPU architecture
Workflows that require the full Python model ecosystem

Models	SenseVoiceSmall-GGUF / Paraformer-GGUF / Fun-ASR-Nano-GGUF / FSMN-VAD-GGUF
Hardware	cpu / desktop-edge-gpu
Operating systems	Linux / macOS / Windows
Interfaces	CLI / local HTTP server

Release assets

Download a verified package for your operating system and backend

SHA-256 values come from the public GitHub Release. Verify the digest, then run a smoke test on the target hardware.

OS	Architecture	Backend	Download	SHA-256
Linux	arm64	CPU	funasr-llamacpp-linux-arm64.tar.gz	`c78987b2384c6aef339aea1bcd0e130070455d6394fa7ab7ca26840ead10d5da`
Linux	x64	CPU	funasr-llamacpp-linux-x64.tar.gz	`15e6407143b4fb91d90bb37f2a41c64c4d48ea0fbe6404b88a9b70269c84f240`
Linux	x64 AVX2	CPU	funasr-llamacpp-linux-x64-avx2.tar.gz	`02e10e9a46ea76a040c45d431efe51a3324e64f08c24d38e18c8a4d2781490cd`
Linux	x64	Vulkan	funasr-llamacpp-linux-x64-vulkan.tar.gz	`caf71b8c0b4c3249cebc4175e5406d3c588eb9e8966a00d571d4cc5070405385`
macOS	arm64	CPU	funasr-llamacpp-macos-arm64.tar.gz	`416cbb289e31cb7575365d382155074e922fd061807a37b9ca0247dabd9bc6f9`
Windows	x64	CPU	funasr-llamacpp-windows-x64.zip	`297c962346d7e30d7a7c2c860dfaab3ff07d01fddf15e6fc5212ca9545441a51`
Windows	x64 AVX2	CPU	funasr-llamacpp-windows-x64-avx2.zip	`4db0f11f603c324a63545cd7009cdd45bb45576efe282cec22796b5fd42d8ea1`
Windows	x64	Vulkan	funasr-llamacpp-windows-x64-vulkan.zip	`90b45240c6ccc9177c25490a11848de60a406e129391c8736b14521c0c28cdcb`
Windows	x64	CUDA	funasr-llamacpp-windows-x64-cuda.zip	`7f2f9ef4d7e0291b284a295ec74bbeca9ea635a7f5f42d0ad06eb780c0d6efc1`

Run path

From installation to known-audio verification

These commands come from the current verification registry. Pin dependencies, model, and hardware before rollout.

Install

curl -fLO https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.0/funasr-llamacpp-linux-x64.tar.gz

echo "15e6407143b4fb91d90bb37f2a41c64c4d48ea0fbe6404b88a9b70269c84f240  funasr-llamacpp-linux-x64.tar.gz" | sha256sum -c -

mkdir funasr-llamacpp && tar -xzf funasr-llamacpp-linux-x64.tar.gz -C funasr-llamacpp && cd funasr-llamacpp && bash download-funasr-model.sh sensevoice ./funasr-gguf f16

Launch

cd funasr-llamacpp && ./llama-funasr-sensevoice -m funasr-gguf/sensevoice-small-f16.gguf --vad funasr-gguf/fsmn-vad.gguf -a sample.wav

Health check

cd funasr-llamacpp && ./llama-funasr-sensevoice --help

Smoke test

Minimum verification before promotion

cd funasr-llamacpp && ./llama-funasr-sensevoice -m funasr-gguf/sensevoice-small-f16.gguf --vad funasr-gguf/fsmn-vad.gguf -a sample.wav | tee transcript.txt && test -s transcript.txt

Operations and capacity

Move from runnable to operable

Operational checks

Verify all nine release assets against the listed SHA-256 values
Keep models and binaries on the same release manifest
Retain the previous binary and model directory for rollback

Capacity variables

Audio duration, language, channels, and VAD segment distribution
Concurrency, queue time, warmup, and model-cache state
Exact hardware, driver, runtime, and thread configuration

Troubleshooting

Validate the q8 CPU path first
Match CUDA or Vulkan to the local driver
For Windows AMD Vulkan failures, record the GPU, driver, GGML_VK_MAX_NODES_PER_SUBMIT, and GGML_VK_SERIALIZE_SUBMISSIONS before reporting

Security boundary

Treat production ingress as untrusted

Read local audio and models by default
Bind the HTTP server to a private address and limit uploads
Do not load GGUF files from untrusted sources

Known limitation

Prebuilt GPU packages cover only the documented backend and architecture; the Windows AMD Vulkan crash fix still awaits reporter hardware confirmation, and every other target also requires hardware validation.

Public benchmarks are reproduction starting points, not substitutes for target-workload testing.

Evidence and feedback

Verify this contract against primary material

Report a deployment issue View related benchmark records