Nothing is generated

[lacmuch]

Context

I downloaded and created the tar.gz model package. Than i loaded (create command) the following model file:

FROM DeepSeek-R1-Distill-Qwen-8B-int4-ov.tar.gz
ModelType "OpenVINO"
InferDevice "NPU"
PARAMETER repeat_penalty 1.0
PARAMETER top_p 1.0
PARAMETER temperature 1.0

The model started and did not generate any output, here is the ollama console output:

time=2025-09-06T20:20:34.231+02:00 level=INFO source=runner.go:240 msg="Request info :\nPOST /completion HTTP/1.1\r\nHost: 127.0.0.1:53910\r\nAccept-Encoding: gzip\r\nContent-Length: 385\r\nContent-Type: application/json\r\nUser-Agent: Go-http-client/1.1\r\n\r\n{"cache_prompt":true,"frequency_penalty":0,"image_data":null,"main_gpu":0,"max_new_token":2048,"min_p":0,"mirostat":0,"mirostat_eta":0.1,"mirostat_tau":5,"n_keep":4,"n_predict":-1,"presence_penalty":0,"prompt":"write a hello world application","repeat_last_n":64,"repeat_penalty":1,"seed":-1,"stop":null,"stop_id":null,"stream":true,"temperature":1,"top_k":40,"top_p":1,"typical_p":1}"
time=2025-09-06T20:20:34.231+02:00 level=INFO source=genai.go:253 msg="Sampling Parameters - Temperature: 1.00, TopP: 1.00, TopK: 40, RepeatPenalty: 1.00"
time=2025-09-06T20:20:34.231+02:00 level=INFO source=genai.go:208 msg="Genai Metrics info:"
time=2025-09-06T20:20:34.231+02:00 level=INFO source=genai.go:212 msg="Load time: 0.00"
time=2025-09-06T20:20:34.231+02:00 level=INFO source=genai.go:217 msg="Generate time: 0.00 ± 0.00 ms"
time=2025-09-06T20:20:34.231+02:00 level=INFO source=genai.go:222 msg="Tokenization time: 0.00 ± 0.00 ms"
time=2025-09-06T20:20:34.231+02:00 level=INFO source=genai.go:227 msg="Detokenization time: 0.00 ± 0.00 ms"
time=2025-09-06T20:20:34.231+02:00 level=INFO source=genai.go:232 msg="TTFT: 0.00 ± 0.00 ms"
time=2025-09-06T20:20:34.231+02:00 level=INFO source=genai.go:237 msg="TPOT: 0.00 ± 0.00 ms/token"
time=2025-09-06T20:20:34.231+02:00 level=INFO source=genai.go:241 msg="Num of generation tokens: 0"
time=2025-09-06T20:20:34.231+02:00 level=INFO source=genai.go:246 msg="Throughput: 0.00 ± 0.00 tokens/s"

My naive question: what could be the problem? Did you get the wrong input prompt?

What needs to be done?

Generete some output.

Example Pull Requests

No response

Resources

• Contribution guide - start here!
• Intel DevHub Discord channel - engage in discussions, ask questions and talk to OpenVINO developers
• How to link your Pull Request to an issue

Contact points

IDK

Ticket

No response