beta wrap_gemini ts wrapper #2121
Conversation
EugeneJinXin
force-pushed
the
xin/wrap_gemini_typescript
branch
3 times, most recently
from
eb00750 to
15065e4
Compare
jacoblee93
left a comment
jacoblee93
left a comment
There was a problem hiding this comment.
Looks mostly good, some nits and questions
js/src/wrappers/gemini.ts
Outdated
| Object.assign(currentRun.extra.metadata.usage_metadata, usageMetadata); | ||
|
|
||
| // Fire-and-forget patch to persist changes asynchronously | ||
| currentRun.patchRun().catch(() => {}); |
There was a problem hiding this comment.
This is already backgrounded, just await it
There was a problem hiding this comment.
Actually - why do you need this at all?
There was a problem hiding this comment.
we need the usageMetadata to store in currentRun.extra.metadata.usage_metadata since that's new requirement from the DB team.
| openai-api-key: | ||
| description: "OpenAI API key" | ||
| required: false | ||
| gemini-api-key: |
There was a problem hiding this comment.
Is this populated in CI? I don't think so?
Let's just omit for now?
There was a problem hiding this comment.
oh, actually populated already; i added GEMINI_API_KEY in github action
There was a problem hiding this comment.
it is used for integration test of gemini wrapper
EugeneJinXin
force-pushed
the
xin/wrap_gemini_typescript
branch
from
bc1edd1 to
11bbdeb
Compare
js/src/wrappers/gemini.ts
Outdated
| .map((chunk) => chunk.text) | ||
| .join(""); | ||
|
|
||
| const result: any = { |
There was a problem hiding this comment.
Try to avoid any, do you even need this cast?
js/src/wrappers/gemini.ts
Outdated
| // Add input token details if available | ||
| usageMetadata.input_token_details = { | ||
| ...(usage.cachedContentTokenCount && { | ||
| cache_read_over_200k: Math.max(0, usage.promptTokenCount - 200000), |
There was a problem hiding this comment.
The math here is a bit trickier I think - cache_read_over_200k and over_200k need to be mutually exclusive for our cost tracking to work out
e.g. if you had a 400k token prompt and 100k were cached the correct usage metadata would be:
{
input_tokens: 400000,
input_token_details: {
cache_read_over_200k: 100000,
over_200k: 300000,
}
}
At least that's my read based on: https://ai.google.dev/gemini-api/docs/pricing
We could ask for confirmation there?
Adds beta TypeScript wrapper for Google Gemini API with automatic LangSmith tracing.
Features