GenAI Utils | Add metrics to LLMInvocations #3891
Conversation
util/opentelemetry-util-genai/src/opentelemetry/util/genai/handler.py
Outdated
Show resolved
Hide resolved
util/opentelemetry-util-genai/src/opentelemetry/util/genai/handler.py
Outdated
Show resolved
Hide resolved
util/opentelemetry-util-genai/src/opentelemetry/util/genai/metrics.py
Outdated
Show resolved
Hide resolved
util/opentelemetry-util-genai/src/opentelemetry/util/genai/metrics.py
Outdated
Show resolved
Hide resolved
util/opentelemetry-util-genai/src/opentelemetry/util/genai/metrics.py
Outdated
Show resolved
Hide resolved
util/opentelemetry-util-genai/src/opentelemetry/util/genai/metrics.py
Outdated
Show resolved
Hide resolved
util/opentelemetry-util-genai/src/opentelemetry/util/genai/metrics.py
Outdated
Show resolved
Hide resolved
| return time.time_ns() | ||
|
|
||
|
|
||
| def _get_span_start_time_ns(span: Optional[Span]) -> Optional[int]: |
There was a problem hiding this comment.
We generally measure the durations separately with default_timer() which is monotonic clock and more accurate when absolute timestamps are not needed. https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-python-contrib%20default_timer&type=code
It would also work even if the Tracing SDK is not set up and start_time can't be read.
There was a problem hiding this comment.
Well, that led me down a rabbit hole of monotonic clocks for duration versus wall clocks for times. Interesting stuff.
I used to calculate the duration based on the span start. That used a wall clock based on the tracer setting it and metrics should use the monotonic clock, I had to add a property and set it on start_llm.
Take a look at these changes and let me know if you think they are too much.
4 participants
Description
Add duration and token metrics to LLMInvocation traces.
Fixes # (issue)
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Unit tests show metrics exporting.
Does This PR Require a Core Repo Change?
Checklist:
See contributing.md for styleguide, changelog guidelines, and more.