Deployment center
Choose a runtime from product constraints
Every path records fit, verified versions, startup commands, production responsibilities, and known limitations.
- Paths
- 10
- Languages
- 2
- Verified
- 2026-08-13
2026-08-13
Use vLLM's built-in FunASR architecture for multilingual and hotword-aware transcription through an OpenAI-compatible API on NVIDIA GPUs.
- Hardware
- nvidia-gpu / kubernetes
- Models
- Fun-ASR-Nano-2512 (community vLLM conversion)
Limitation
The native vLLM path currently depends on a community-converted checkpoint; use the official FunASR split-engine when an official-weight chain is required. Evidence covers one H100 only, so retest the target GPU, audio, languages, hotwords, and traffic.
View deployment contract
Production verified
2026-08-04
Build SenseVoiceSmall as a native FP16 TensorRT engine and serve batched GPU inference through Triton.
- Hardware
- nvidia-gpu / kubernetes
- Models
- SenseVoiceSmall
Limitation
A TensorRT plan is not portable across arbitrary GPU architectures or TensorRT versions; rebuild and revalidate accuracy, memory, and capacity on the target stack.
View deployment contract
Production verified
2026-08-11
Use the nine v0.2.0 release packages or source builds to run FunASR GGUF models on CPU, Vulkan, CUDA, and edge devices.
- Hardware
- cpu / desktop-edge-gpu
- Models
- SenseVoiceSmall-GGUF / Paraformer-GGUF
Limitation
Prebuilt GPU packages cover only the documented backend and architecture; the Windows AMD Vulkan crash fix still awaits reporter hardware confirmation, and every other target also requires hardware validation.
View deployment contract
Production verified
2026-08-13
Use one C++ binary with no Python runtime to serve SenseVoice over REST, SSE, SRT/VTT, and OpenAI realtime WebSocket transcription.
- Hardware
- cpu / nvidia-gpu
- Models
- SenseVoiceSmall-GGUF / FSMN-VAD-GGUF
Limitation
Official verification currently covers Linux source builds; the server does not include production authentication, TLS, tenant quotas, or durable queues, so provide them at the gateway and infrastructure layers.
View deployment contract
2026-08-13
Run Fun-ASR-Nano and SenseVoice Q8 with native C++ and GGML on CPU or GPU, using offline CLI, a local OpenAI-compatible API, and buffered SenseVoice streaming results.
- Hardware
- cpu / nvidia-gpu / desktop-edge-gpu
- Models
- Fun-ASR-Nano-2512 / SenseVoice-Small
Limitation
SenseVoice is merged into audio.cpp main@979e070f but is not yet in a tagged release. Pin that main commit until a release is published. Neither path currently provides word-level timestamps.
View deployment contract
Production verified
2026-07-26
Serve FunASR models through `/v1/audio/transcriptions` for existing SDKs, agents, and workflows.
- Hardware
- cpu / nvidia-gpu / kubernetes
- Models
- sensevoice / paraformer
Limitation
The example server does not include production authentication, tenant quotas, or full observability; provide them at the gateway and infrastructure layers.
View deployment contract
2026-08-17
Use online or two-pass WebSocket protocols for microphones, meeting captions, and continuous speech streams.
- Hardware
- cpu / nvidia-gpu / kubernetes
- Models
- Paraformer-online / FSMN-VAD
Limitation
Capacity depends on realtime ratio, chunks, VAD, connection count, and client backpressure; file RTFx does not replace concurrency tests.
View deployment contract
2026-07-26
Validate with Docker Compose, then deploy with the Kubernetes template, persistent model cache, probes, and private ClusterIP.
- Hardware
- kubernetes / cpu / nvidia-gpu
- Models
- sensevoice / paraformer
Limitation
The CPU example is not a CUDA production image; GPU drivers, image, scheduling, and capacity still require platform validation.
View deployment contract
2026-07-26
Deploy offline, online, and two-pass ASR on general CPUs with ONNX Runtime and C++ paths.
- Hardware
- cpu
- Models
- Paraformer / Paraformer-online
Limitation
Model conversion, operator support, and thread settings vary by platform; measure capacity on the target CPU with real audio.
View deployment contract
Production verified
2026-07-26
Move from a runnable model to an observable, scalable, reversible production service with explicit boundaries.
- Hardware
- cpu / nvidia-gpu / desktop-edge-gpu / kubernetes
- Models
- All supported deployment models
Limitation
A checklist does not replace workload, accuracy, failure, and security exercises for the target product.
View deployment contract