Skip to content

Commit 4c944de

Browse files
fix: mdformat over-saturation docs
Signed-off-by: Alon Kellner <[email protected]>
1 parent 72823a9 commit 4c944de

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

docs/guides/over_saturation_stopping.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,12 +16,14 @@ GuideLLM's Over-Saturation Detection (OSD) algorithm uses statistical slope dete
1616
2. **Time-to-First-Token (TTFT)**: The latency for the first token of each response
1717

1818
For each metric, the algorithm:
19+
1920
- Maintains a sliding window of recent data points
2021
- Calculates the linear regression slope using online statistics
2122
- Computes the margin of error (MOE) using t-distribution confidence intervals
2223
- Detects positive slopes with low MOE, indicating degradation
2324

2425
Over-saturation is detected when:
26+
2527
- Both concurrent requests and TTFT show statistically significant positive slopes
2628
- The minimum duration threshold has been met
2729
- Sufficient data points are available for reliable slope estimation
@@ -132,6 +134,7 @@ guidellm benchmark \
132134
```
133135

134136
This example:
137+
135138
- Runs a concurrent benchmark with 16 simultaneous requests
136139
- Uses synthetic data with 256 prompt tokens and 128 output tokens
137140
- Enables over-saturation detection with custom timing parameters

0 commit comments

Comments
 (0)