Inference Gateway API #694
Conversation
|
| maxRetries?: number; | ||
| timeout?: number; | ||
| verbosity?: Verbosity; | ||
| extraKwargs?: LLMOptions<TModel>; |
There was a problem hiding this comment.
can we just call this extra or extraArgs and also rename to match on the python side? it's not great to leak the word "kwargs" into TypeScript since that's a Python word cc @longcw @theomonnom
| // SPDX-FileCopyrightText: 2025 LiveKit, Inc. | ||
| // | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
| export { LLM, LLMStream, type LLMModels, type LLMOptions, type OpenAIModels } from './llm.js'; |
There was a problem hiding this comment.
| export { LLM, LLMStream, type LLMModels, type LLMOptions, type OpenAIModels } from './llm.js'; | |
| import * as llm from './llm.js'; | |
| import * as stt from './stt.js'; | |
| import * as tts from './tts.js'; | |
| export { stt, tts, llm }; | |
| export { LLM, LLMStream, type LLMModels, type LLMOptions, type OpenAIModels } from './llm.js'; |
Can we also export the whole llm, stt, and tts modules so that people can access the underlying OpenAiOptions, CerebrasOptions, etc types in generated docs? (we can't just export them normally since CartesiaOptions appears in both STT and TTS)
| keyterms_prompt?: string[]; // default: not specified | ||
| } | ||
|
|
||
| export type STTModels = DeepgramModels | CartesiaModels | AssemblyaiModels; |
There was a problem hiding this comment.
I think you need | string here or at consumption sites so people can adopt new models added to the service that aren't yet added to the type enums (same for the other types)
|
I see a bunch of this error in my terminal when i run the starter agent in dev with the gateway integrated |
@toubatbrian Maybe too many Tasks? |
|
Approved by accident. Pls get @bcherry 's approval before merge |
No description provided.