Conversation
…king non-streaming Anthropic SDK clients Wei-Shaw#867
There was a problem hiding this comment.
Pull request overview
This PR fixes issue #867 where a single isStream flag was incorrectly used to control both upstream streaming (SSE) and downstream client streaming, causing OAuth accounts (which force upstream SSE) to return SSE even when the client requested stream:false.
Changes:
- Split streaming intent into
clientWantsStream(downstream behavior) andisStream(upstream request behavior, OAuth may force to true). - Update response handling to branch on
clientWantsStream, adding an SSE→JSON aggregation path when upstream must stream but client does not. - Add
handleAnthropicStreamToNonStreamingResponseto read the upstream Responses SSE stream and assemble a single Anthropic Messages JSON response.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
| // Assemble content blocks in index order. | ||
| var content []apicompat.AnthropicContentBlock | ||
| for idx := 0; idx < len(blocks); idx++ { | ||
| b, ok := blocks[idx] | ||
| if !ok { | ||
| continue | ||
| } |
There was a problem hiding this comment.
When assembling content from blocks, iterating idx := 0; idx < len(blocks); idx++ can drop blocks if indices are non-contiguous or don’t start at 0 (e.g., keys {0,2} => len=2, index 2 never visited). Track the max index or collect/sort the map keys and iterate in order to ensure all blocks are included.
|
|
||
| if s.responseHeaderFilter != nil { | ||
| responseheaders.WriteFilteredHeaders(c.Writer.Header(), resp.Header, s.responseHeaderFilter) | ||
| } |
There was a problem hiding this comment.
WriteFilteredHeaders forwards upstream Content-Type by default (and upstream is text/event-stream in this path). Gin’s c.JSON won’t overwrite an existing Content-Type, so clients may receive a JSON body with text/event-stream header. Explicitly Set the downstream Content-Type to application/json; charset=utf-8 (ideally after copying headers, or ensure the filter doesn’t forward Content-Type here).
| } | |
| } | |
| // Ensure JSON responses are served with the correct Content-Type, even if | |
| // the upstream forwarded a different Content-Type (e.g., text/event-stream). | |
| c.Writer.Header().Set("Content-Type", "application/json; charset=utf-8") |
| switch { | ||
| case clientWantsStream: | ||
| // Client wants SSE; upstream is also streaming — pass through directly. | ||
| result, handleErr = s.handleAnthropicStreamingResponse(resp, c, originalModel, mappedModel, startTime) | ||
| } else { | ||
| case isStream: | ||
| // Upstream is streaming (OAuth forced) but client wants a single JSON | ||
| // object — collect the SSE and assemble the non-streaming response. | ||
| result, handleErr = s.handleAnthropicStreamToNonStreamingResponse(resp, c, originalModel, mappedModel, startTime) | ||
| default: |
There was a problem hiding this comment.
This introduces a new behavioral branch (upstream SSE → client JSON via handleAnthropicStreamToNonStreamingResponse) that isn’t covered by tests. There are existing tests for OAuth SSE→JSON conversion in openai_gateway_service_test.go; adding a similar unit test for ForwardAsAnthropic/handleAnthropicStreamToNonStreamingResponse would help prevent regressions (e.g., client stream:false with OAuth account returns a single JSON body and correct Content-Type/usage).
修改文件：backend/internal/service/openai_gateway_messages.go
问题根因：isStream 变量身兼两职——同时控制"上游请求是否流式"和"客户端响应是否流式"。OAuth 账号为满足上游要求强制将其设为 true，导致即使客户端发送 stream: false，响应仍以 SSE 格式返回。
修复思路：拆分两个变量的职责。 1.clientWantsStream：只记录客户端的原始意图，全程不变
2.isStream：只控制上游请求格式，OAuth 时可被覆盖为 true
响应阶段改用 clientWantsStream 做分支判断，并新增 handleAnthropicStreamToNonStreamingResponse，处理"上游 SSE → 客户端 JSON"的转换（读完 SSE 流后组装成单个 JSON 对象返回）。