Skip to content

[BUG]: No usage in response when tool calling #4440

@rmccorm4

Description

@rmccorm4

When a tool calling response is returned, there is no token usage in the response

  • with tool calling
  "usage": null
  • without tool calling
  "usage": {
    "prompt_tokens": 21,
    "completion_tokens": 406,
    "total_tokens": 427
  }

vllm serve has usage for both with / without tool calling

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingfrontend`python -m dynamo.frontend` and `dynamo-run in=http|text|grpc`observabilityRelated to metrics, tracing, logging

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions