-
Notifications
You must be signed in to change notification settings - Fork 130
fix: make usage chunk in stream mode of gemini compatible with openai #1503
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
3fc0bf1 to
35bff4c
Compare
Codecov Report❌ Patch coverage is
❌ Your patch status has failed because the patch coverage (61.90%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #1503 +/- ##
==========================================
- Coverage 84.03% 84.01% -0.02%
==========================================
Files 141 141
Lines 13039 13049 +10
==========================================
+ Hits 10957 10963 +6
- Misses 1460 1462 +2
- Partials 622 624 +2 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
/retest |
|
@yuzisun can you take a look at this one? |
Signed-off-by: yxia216 <[email protected]>
… a tool call (envoyproxy#1486) **Description** Finish reason should be tool calls if the model returns a tool call response. In vertex api, there is no tool call finish reason, thus need a work around to make it compatible. --------- Signed-off-by: yxia216 <[email protected]> Co-authored-by: Dan Sun <[email protected]> Signed-off-by: yxia216 <[email protected]>
…oxy#1491) **Description** This decouples backendauth & headermutator packages from extproc specifics. As we are looking to migrate to dynamic modules, this is a necessary refactoring work to make the code as reusable as possible. **Related Issues/PRs (if applicable)** Preliminary for envoyproxy#90 --------- Signed-off-by: Takeshi Yoneda <[email protected]> Signed-off-by: yxia216 <[email protected]>
Signed-off-by: yxia216 <[email protected]>
Signed-off-by: yxia216 <[email protected]>
Signed-off-by: yxia216 <[email protected]>
66eaa8c to
30a3391
Compare
Description
Users found the simply use "usage" information does not work for streaming responses of gemini models.

This is because for openai models, the usage chunk would be a separate chunk. For example, this is an example response from gpt-4o:
There is a finish_reason chunk, and then a usage chunk.
Thus, want to make it compatible with Openai. (Actually, in anthropic translation, it's already compatible)