File tree Expand file tree Collapse file tree 1 file changed +3
-0
lines changed
Expand file tree Collapse file tree 1 file changed +3
-0
lines changed Original file line number Diff line number Diff line change @@ -16,12 +16,14 @@ GuideLLM's Over-Saturation Detection (OSD) algorithm uses statistical slope dete
16162 . ** Time-to-First-Token (TTFT)** : The latency for the first token of each response
1717
1818For each metric, the algorithm:
19+
1920- Maintains a sliding window of recent data points
2021- Calculates the linear regression slope using online statistics
2122- Computes the margin of error (MOE) using t-distribution confidence intervals
2223- Detects positive slopes with low MOE, indicating degradation
2324
2425Over-saturation is detected when:
26+
2527- Both concurrent requests and TTFT show statistically significant positive slopes
2628- The minimum duration threshold has been met
2729- Sufficient data points are available for reliable slope estimation
@@ -132,6 +134,7 @@ guidellm benchmark \
132134```
133135
134136This example:
137+
135138- Runs a concurrent benchmark with 16 simultaneous requests
136139- Uses synthetic data with 256 prompt tokens and 128 output tokens
137140- Enables over-saturation detection with custom timing parameters
You can’t perform that action at this time.
0 commit comments