Skip to content

Conversation

@ArCyanic
Copy link
Contributor

The origin timing is wrong, this commit fixed that and added extra significant information for profiling.
Now it looks like this:

[Total time] 332.403s
[Prefill time] 36.8335s
[KV cache copy time] 7.6678e-05s
[Decode time] 295.569s
[Prefilling] 0.271492 tokens/s
[Decoding] 2.85213 tokens/s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant