Regarding the fine-tuning of the Whisper model using the "pytorch_model.bin" file generated by DeepSpeed.

[scufan1990]

How to convert the pytorch_model.bin generated by DeepSpeed during the fine-tuning of the whisper model into a .pt format that can be used by wenet?

[Mddct]

pytorch_model.bin is already pt format file

[scufan1990]

pytorch_model.bin is already pt format file

When I am decoding using the fine-tuned pytorch_model.bin model, it prompts that the parameters related to the decoder are {INFO missing tensor:}

Also, the decoding results are all {|notimestamps|} this one token.

I am using the configuration {conf/finetune_whisper_largev3.yaml} for fine-tuning, may I ask what the reason for this is?