paraformer English model available?

[runner995]

I tried sherpa-ncnn English models, seems like error rate for spoken English (everyday chatting, 3-4 words) is kind of high, want to give ONNX a try, I tried the sherpa-onnx-streaming-paraformer-bilingual-zh-en but 50% of the English words are mistaken to Chinese.

Do you have English only paraformer onnx model available? online or offline, I am unable to find any in the pre-trained model download site.

[dosubot]

Hi @runner995! I'm Dosu and I’m helping the sherpa-onnx team.

There is an English-only Paraformer ONNX model available for offline use: csukuangfj/sherpa-onnx-paraformer-en-2024-03-09. You can download it here and find usage instructions in the docs.

Currently, there isn’t an English-only streaming Paraformer ONNX model listed—only bilingual or trilingual streaming models are available, which may confuse English and Chinese words as you’ve seen source.

If you need streaming English recognition, consider trying the Zipformer or LSTM transducer models, which are English-only and support streaming.

For short utterances, you may get better results by adjusting endpointing settings or trying different decoding methods (like greedy search or beam search) as described in the docs.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

  

[csukuangfj]

Please use
https://k2-fsa.github.io/sherpa/onnx/pretrained_models/offline-transducer/nemo-transducer-models.html#sherpa-onnx-nemo-parakeet-tdt-0-6b-v2-int8-english
for English.

[runner995]

@dosu Thanks a lot! will try the offline model. I noticed the example android project code are using all streaming models, do you have example code using non-streaming model? I guess the getModelConfig() implementation will be different.

[dosubot]

You’re right—the configuration for non-streaming (offline) models is different from streaming models in sherpa-onnx Android projects. While the Android examples focus on streaming models, you can adapt the offline model setup using the same principles as in the C# and Go API examples.

For a non-streaming Paraformer model, you’ll need to use an OfflineModelConfig and set the relevant fields for the offline model type. Here’s a pseudocode template for what your getModelConfig() might look like for an offline Paraformer model:

OfflineModelConfig config = new OfflineModelConfig();
config.setParaformerModel("/path/to/model.onnx");
config.setTokens("/path/to/tokens.txt");
config.setNumThreads(2); // or as needed
config.setProvider("cpu"); // or "nnapi", "cuda", etc.
config.setModelType("paraformer");
config.setDebug(false);

OfflineRecognizerConfig recognizerConfig = new OfflineRecognizerConfig();
recognizerConfig.setModelConfig(config);
recognizerConfig.setDecodingMethod("greedy_search"); // or "modified_beam_search"
recognizerConfig.setMaxActivePaths(4); // for beam search
recognizerConfig.setFeatureConfig(new OfflineFeatureExtractorConfig(16000, 80)); // sample rate, feature dim

Key differences from streaming config:

• No chunk size, left context, or endpointing parameters.
• Use OfflineModelConfig instead of OnlineModelConfig.
• Specify the model file, tokens, and decoding method for offline recognition.

For reference, see how offline models are configured in Go and C# API examples. The Android integration will follow the same logic—just adapt the config object to your platform.

If you want to see how two-pass recognition works (streaming first, then offline for final result), check out the Android two-pass APK docs—these use both streaming and non-streaming models, though most examples are for Chinese.

Let me know if you need more details or a specific Java/Kotlin snippet!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

  

[csukuangfj]

I noticed the example android project code are using all streaming models

are you using sherpa-ncnn? can you try also sherpa-onnx?

[runner995]

I noticed the example android project code are using all streaming models

are you using sherpa-ncnn? can you try a

yes, I am using the onnx android example project now, trying to figure out how to change the kotlin code to use the offline model recommended.

[csukuangfj]

Please choose the correct android demo, e.g.,

• https://github.com/k2-fsa/sherpa-onnx/tree/master/android/SherpaOnnxSimulateStreamingAsr
• https://github.com/k2-fsa/sherpa-onnx/tree/master/android/SherpaOnnxVadAsr