Deployment contract
audio.cpp native Fun-ASR-Nano and SenseVoice
Run Fun-ASR-Nano and SenseVoice Q8 with native C++ and GGML on CPU or GPU, using offline CLI, a local OpenAI-compatible API, and buffered SenseVoice streaming results.
Workload boundary
Decide whether it fits your production constraints
Good fit
- No Python inference environment
- Desktop, edge, offline batch, and low-dependency private deployment
- A local OpenAI-compatible endpoint or multilingual SenseVoice streaming partials are required
Not a fit
- Word-level timestamps or strict low-latency online streaming
- Production environments restricted to tagged releases
- Internet-facing service before capacity and accuracy tests on target hardware
| Models | Fun-ASR-Nano-2512 / SenseVoice-Small |
|---|---|
| Hardware | cpu / nvidia-gpu / desktop-edge-gpu |
| Operating systems | Linux / macOS / Windows |
| Interfaces | CLI / OpenAI-compatible HTTP / Buffered streaming CLI/SSE |
Run path
From installation to known-audio verification
These commands come from the current verification registry. Pin dependencies, model, and hardware before rollout.
Install
git clone https://github.com/0xShug0/audio.cpp.git && cd audio.cppgit checkout 979e070fc130bd499ad3fabeefc42b3884fff23abash scripts/build_linux.sh --backend cpu --model-set custom --models fun_asr_nano,sense_asr --target audiocpp_cli --target audiocpp_serverpython3 tools/model_manager_v2.py install fun_asr_nanopython3 tools/model_manager_v2.py install sensevoice_small_q8Launch
build/linux-cpu-release/bin/audiocpp_cli --task asr --family fun_asr_nano --model models/Fun-ASR-Nano-2512-GGUF/fun-asr-nano-2512-q8_0.gguf --backend cpu --audio speech.wav --text-out transcript.txtbuild/linux-cpu-release/bin/audiocpp_cli --task asr --family sense_asr --model models/SenseVoice-Small-GGUF/sensevoice-small-q8-audiocpp-v1.gguf --backend cpu --audio speech.wav --text-out sensevoice.txtbuild/linux-cpu-release/bin/audiocpp_cli --task asr --family sense_asr --model models/SenseVoice-Small-GGUF/sensevoice-small-q8-audiocpp-v1.gguf --backend cpu --mode streaming --audio - --request-option audio_chunk_duration_sec=5 --request-option audio_chunk_mode=none < 16k_s16.pcmprintf '%s\n' '{"host":"127.0.0.1","port":8080,"backend":"cpu","threads":4,"lazy_load":true,"models":[{"id":"fun-asr-nano","family":"fun_asr_nano","path":"models/Fun-ASR-Nano-2512-GGUF/fun-asr-nano-2512-q8_0.gguf","task":"asr","mode":"offline"},{"id":"sense-asr","family":"sense_asr","path":"models/SenseVoice-Small-GGUF/sensevoice-small-q8-audiocpp-v1.gguf","task":"asr","mode":"streaming"}]}' > server.jsonbuild/linux-cpu-release/bin/audiocpp_server --config server.jsonHealth check
curl -fsS http://127.0.0.1:8080/healthcurl -fsS http://127.0.0.1:8080/v1/modelsSmoke test
Minimum verification before promotion
curl -fsS http://127.0.0.1:8080/v1/audio/transcriptions -F model=fun-asr-nano -F language=auto -F file=@speech.wavcurl -fsS http://127.0.0.1:8080/v1/audio/transcriptions -F model=sense-asr -F language=auto -F file=@speech.wavOperations and capacity
Move from runnable to operable
Operational checks
- Pin the audio.cpp main commit and both GGUF revisions
- Verify known WAV transcripts with both models before business traffic
- Measure cold start, warm RTF, memory, streaming windows, and concurrent queueing separately
Capacity variables
- Audio duration, language, channels, and VAD segment distribution
- Concurrency, queue time, warmup, and model-cache state
- Exact hardware, driver, runtime, and thread configuration
Troubleshooting
- Start with CPU and Q8_0 to isolate GPU setup
- Match the compiled backend to the runtime argument
- For SenseVoice input failures, verify the schema-v1 GGUF revision plus WAV or 16 kHz mono PCM
Security boundary
Treat production ingress as untrusted
- Bind the service to 127.0.0.1 or a trusted private network
- Enforce upload size, MIME, authentication, and concurrency at the proxy
- Load only GGUF and configuration files verified with SHA-256
Known limitation
SenseVoice is merged into audio.cpp main@979e070f but is not yet in a tagged release. Pin that main commit until a release is published. Neither path currently provides word-level timestamps.
Public benchmarks are reproduction starting points, not substitutes for target-workload testing.
Evidence and feedback
