Choose the right checkpoint for Transformers
If you are adding speech recognition to an existing Transformers application, choose the checkpoint for that interface first. A shared model name does not make toolkit, serving and native Python weights interchangeable.
For example, a checkpoint may download successfully while the application reports an unknown model type. Check both the installed implementation and the selected artifact before assuming the weights are damaged. The native path here uses the official -hf checkpoint.
This is a text-generation interface, not a ready-made service. It does not supply word timestamps, speaker separation or streaming simply because loading succeeds.
Ready to try it? Follow the Transformers 5.17.0 quickstart, open the Space or run the notebook. Start with one recording, then add batches or a deployment backend.
One family, four different paths
- FunASR AutoModel:
Fun-ASR-Nano-2512. Original toolkit path; split-engine has its own guide. - Transformers processor / generate:
Fun-ASR-Nano-2512-hf. The native path described here, not a vLLM conversion. - Native vLLM HTTP service:
Fun-ASR-Nano-2512-vllm. Separate runtime, checkpoint format and service options. - C++ / GGML runtime: Matching converted GGUF files. Not a renamed Transformers directory.
The official model repositories are under FunAudioLLM. Native Transformers loads the official -hf checkpoint, while the model and processor code come from the pinned Transformers installation. It does not need to execute checkpoint-provided remote Python code. That removes one integration dependency; it does not supply authentication, queues or service monitoring.
For concurrent clients, inspect the native vLLM deployment. For offline C++, inspect llama.cpp deployment. A composable Python interface is not a replacement for every serving stack.
What did the short-recording check establish?
With released Transformers 5.17.0, official -hf revision and CPU float32, we ran a Chinese recording, an English recording, a mixed Chinese/English batch and a Chinese keyword request. All four requests returned text and EOS before the limit, with the batch in input order. These are functional checks on two public short recordings, not an accuracy ranking.
| The same Chinese audio | Raw output |
|---|---|
| No keywords | 开饭时间早上九点至下午五点。 |
| Candidate keyword: 开放时间 | 开饭时间：早上九点至下午五点。 |
The hint did not force the requested spelling. Accepting a parameter, changing the output and recovering the intended business term are different conclusions. Keep the raw result and check it against a human reference; prompts do not replace acceptance testing. See the verification record for environment, sample provenance and resampling details.
What remains after the interface works?
- Output scope. This -hf path is generation-based transcription and omits the native CTC branch. Text is not a word-timestamp or speaker-diarization result.
- Completeness. The short-file example allows 128 new tokens. Reaching the limit can truncate output; raising it does not prove complete long-recording coverage.
- Resources. A CPU example is not a CUDA, concurrent-serving or vLLM performance test. Measure download, model loading and generation separately. One short file is not capacity planning.
- Quality and privacy. Review important numbers, negation and the recording's end in authorized audio. Preserve fixed versions and raw output. Never attach customer recordings, tokens or identity data to a public report.
For segment timestamps and anonymous speakers, compare the third-party OpenMOSS MOSS unified transcription and diarization path. Anonymous labels are not known-person identities. Choose the output the application actually needs.
Appendix: environment and interface details
Check the package, not just the merge
PR #46180 merged on 2026-09-09, and released Transformers 5.17.0 now contains the native implementation. We inspected the published wheel and repeated real Chinese/English inference, keywords and batching on CPU. Use the stable-package quickstart; a source build is no longer required.
This distinction changes troubleshooting: an unknown model type may mean that the installed package lacks the implementation, not that the checkpoint is corrupt. A successful model download does not prove that the selected runtime can load it.
What happens between audio and text?
- Audio samples. Sample rate and channel layout are part of the data's meaning. Labeling a 48 kHz array as 16 kHz changes the time scale seen by the model. The guide requires an explicitly prepared mono 16 kHz WAV.
- Audio features. The native extractor computes Kaldi fbank through torchaudio, then applies low-frame-rate stacking and subsampling. Matching torchaudio is required here, regardless of the toolkit's optional-dependency policy.
- The transcription request.
apply_transcription_requestprepares the checkpoint's chat template from audio, language, context and keywords, aligning audio placeholder tokens with features. - Generation and decoding.
AutoModelForSpeechSeq2Seqreturns tokens. Remove the input prompt width before decoding to avoid mixing template framing into recognized speech.
A synthetic-silence preprocessing check separates dependency and template failures from weight loading. It does not run the generation model, so it cannot establish successful speech recognition.
Where does application context belong?
Pass a vocabulary through native keywords and relevant background through prompt. These are not the toolkit's hotword argument or HTTP fields. For separate recording contexts, language, prompt and nested keyword lists must match the number of audio inputs. Map results back to the same input manifest in order.
Customer-service recordings may benefit from candidate product or person names. But “the template includes this keyword” and “the recognizer recovered it in noise” are different tests. Do not present a spelling correction or summary rewrite as raw ASR output.
The pinned checkpoint defaults to left padding; explicit padding still makes a batch example easier to inspect. Remove the full input tensor width from generated sequences, not each row's valid attention-mask length. Reject empty recordings and empty batches at your application boundary.
Primary sources: official model documentation at the merged commit and the pinned model card. Explore the model and contribute through Fun-ASR and FunASR.
Try one recording
Open the pinned native Transformers guide and complete its short-recording check in a separate environment. Keep the raw result and verify that the output matches your application before expanding the workload.