Releases: BerriAI/litellm
Releases · BerriAI/litellm
v1.80.9.dev1
What's Changed
- Add Helicone as a provider and update observability documentation by @juliettech13 in #17663
- [Docs] - add sap gen ai provider on LiteLLM by @ishaan-jaff in #17667
- Move query params to create_pass_through_route call in Langfuse passthrough by @jason-nance in #17660
- Update litellm docs drop_params by @ishaan-jaff in #17658
- docs: improve Getting Started page and SDK documentation structure by @Chesars in #17614
- [Feat] New model - add nvidia nim
llama-3.2-nv-rerankqa-1b-v2by @ishaan-jaff in #17670 - Update integration docs location by @krrishdholakia in #17644
- [Feat] Dynamic Rate Limiter - allow specifying ttl for in memory cache by @ishaan-jaff in #17679
New Contributors
- @juliettech13 made their first contribution in #17663
- @jason-nance made their first contribution in #17660
Full Changelog: v1.80.9-nightly...v1.80.9.dev1
v1.80.9-nightly
What's Changed
- [Fix] Add User Writable Directory to Non Root Docker for Logo by @yuneng-jiang in #17180
- Add polling via cache feature by @xianzongxie-stripe in #16862
- Add New Bedrock OSS Models to Model List by @marty-sullivan in #17638
- docs(contributing): update clone instructions to recommend forking first by @krisxia0506 in #17637
- correct model type for
gpt-image-1-miniby @emerzon in #17635 - Native Webhook Integration Sumologic by @chetanchoudhary-sumo in #17630
- fix(anthropic): prevent duplicate tool_result blocks with same tool by @kevinmarx in #17632
- Helm chart extraResources by @expruc in #17627
- fix(proxy): async_post_call_streaming_iterator_hook now properly iterates async generators by @rcII in #17626
- add onyx guardrail hooks integration by @tamirkiviti13 in #16591
- fix(anthropic): handle partial JSON chunks in streaming responses by @Chesars in #17493
- Document missing environment variables and fix incorrect types by @AlexsanderHamir in #17649
- Add usage details in responses usage object by @Sameerlite in #17641
- Fix error for response api polling by @Sameerlite in #17654
- Use audio content for caching by @Sameerlite in #17651
- (Feat) Add fireworks rerank support by @Sameerlite in #17653
- fix: Return 403 exception when calling GET responses api by @Eric84626 in #17629
- Fix flanky tests by @AlexsanderHamir in #17665
- [New provider] Sap gen ai hub by @vasilisazayka in #16053
New Contributors
- @xianzongxie-stripe made their first contribution in #16862
- @krisxia0506 made their first contribution in #17637
- @chetanchoudhary-sumo made their first contribution in #17630
- @kevinmarx made their first contribution in #17632
- @expruc made their first contribution in #17627
- @rcII made their first contribution in #17626
- @tamirkiviti13 made their first contribution in #16591
- @Eric84626 made their first contribution in #17629
- @vasilisazayka made their first contribution in #16053
Full Changelog: v1.80.8.dev.1...v1.80.9-nightly
v1.80.8.rc.1
What's Changed
- merge main by @Sameerlite in #17408
- Guardrail API - support tool call checks on OpenAI
/chat/completions, OpenAI/responses, Anthropic/v1/messagesby @krrishdholakia in #17459 - docs: document responses and embedding api for github copilot by @codgician in #17456
- fix(openai): add clear error message for empty LLM endpoint responses by @DevajMody in #17445
- Fix: Skip empty text blocks in Anthropic system messages by @HaiyiMei in #17442
- feat: Basic Weave OTEL integration by @andrewtruong in #17439
- fix(ocr): add missing ocr and aocr to CallTypes enum by @Chesars in #17435
- fix(groq): check supports_response_schema before using json_tool_call workaround by @Chesars in #17438
- fix(pricing): add missing Moonshot turbo models and fix incorrect pricing by @Chesars in #17432
- fix: make bedrock converse messages respect modify_params as expected by @danielmklein in #17427
- Update docs to link agent hub by @krrishdholakia in #17462
- [Feature] Add Auto Redirect to SSO to New Login Page by @yuneng-jiang in #17451
- [Infra] UI - Change Credentials to use React Query by @yuneng-jiang in #17465
- Map output_tokens_details of responses API to completion_tokens_details by @Sameerlite in #17458
- Add support for bedrock qwen 2 imported model by @Sameerlite in #17461
- Add support for file content download for bedrock batches by @Sameerlite in #17470
- fix: test_default_api_base for ragfow by @Sameerlite in #17485
- feat - new model amazon.titan-image-generator-v2:0 by @ishaan-jaff in #17488
- Fix aim guardrail tests by @hxyannay in #17499
- fix: resolve ruff lint errors by @AnasAbdelR in #17490
- 🆕 feat: support routing to only websearch supported deployments by @raghav-stripe in #17500
- [Management endpoints] - Allow admin viewer to access global tag usage by @ishaan-jaff in #17501
- [Feat] Agent Access Control - Enforce Allowed agents by key, team + add agent access groups on backend by @ishaan-jaff in #17502
- [Feat] UI - Agent Gateway - set allowed agents by key, team by @ishaan-jaff in #17511
- [Docs] A2a - Permission management by @ishaan-jaff in #17515
- [Feat] Allow adding OpenAI compatible chat providers using .json + add public ai provider by @ishaan-jaff in #17448
- chore: add stack trace to mcp error message by @uc4w6c in #17269
- (fix): empty response + vllm streaming by @dominicfeliton in #17516
- fix(guardrails): mask all matching keywords in content filter by @DevajMody in #17521
- Enable detailed debugging for reference by @krrishdholakia in #17508
- fix(sql): Optimize SpendLogs queries to use timestamp filtering for index usage by @kristianmitk in #17504
- fix(gemini): handle partial JSON chunks after first valid chunk by @Chesars in #17496
- fix(unified_guardrail.py): support during_call event type for unified guardrails by @krrishdholakia in #17514
- Guardrails API - new
structured_messagesparam by @krrishdholakia in #17518 - [Feat] VertexAI - Add BGE Embeddings support by @Sameerlite in #17362
- [Fix] Select in Edit Membership Modal by @yuneng-jiang in #17524
- fix: code quality issues from ruff linter by @AlexsanderHamir in #17536
- Fix: Presidio guardrail test TypeError and license base64 decoding error by @AlexsanderHamir in #17538
- [fix] parse budget:thinking blocks for opus 4.5 by @colinlin-stripe in #17534
- Fix vector store configuration synchronization failure by @Sameerlite in #17525
- Add support for cursor BYOK with its own configuration by @Sameerlite in #17519
- Add background health checks to db by @Sameerlite in #17528
- Fix: Lack of None value checks & update publicai_chat_transformation tests by @AlexsanderHamir in #17539
- Remove hardcoded api key from test file by @krrishdholakia in #17540
- Make sure that media resolution is only for gemini 3 model by @Sameerlite in #17137
- [Bug fix] WatsonX audio transcriptions, don't force content type in request headers by @ishaan-jaff in #17546
- [Infra] Bump LiteLLM Enterprise Version by @yuneng-jiang in #17549
- Fix budget update to allow null max_budget by @ishaan-jaff in #17545
- fix failing vertex tests by @Sameerlite in #17542
- [Feature] Show all callbacks on UI by @yuneng-jiang in #16335
- fix(github_copilot): preserve encrypted_content in reasoning items for multi-turn conversations by @rgshr in #17130
- feat: Add gpt-5.1-codex-max model pricing and configuration by @Chesars in #17541
- Fix: apply_guardrail method and improve test isolation by @AlexsanderHamir in #17555
- Fix: MLflow streaming spans for Anthropic passthrough by @uc4w6c in #17288
- [Fixes] Dynamic Rate Limiter - Dynamic rate limiting token count increases/decreases by 1 instead of actual count + Redis TTL by @ishaan-jaff in #17558
- [Bug fix] Secret Managers Integration - Make email and secret manager operations independent in key management hooks by @ishaan-jaff in #17551
- feat: add Mistral Large 3 model support by @Chesars in #17547
- fix(team): use organization.members instead of deprecated organizatio… by @DevajMody in #17557
- [Feature] Customer (end user) Usage by @yuneng-jiang in #17498
- [Feat] Allow using dynamic rate limit/priority reservation on teams by @ishaan-jaff in #17061
- build(deps): bump jws from 3.2.2 to 3.2.3 in /ui/litellm-dashboard by @dependabot[bot] in #17494
- fix(presidio): handle empty content and error dict responses in guardrails by @dominicfallows in #17489
- fix(bedrock): anthropics beta headers for qwen by @irfansofyana in #17467
- fix(gemini): fix reasoning_content showing duplicate content in streaming responses by @Chesars in #17266
- Fixes #13652 - auth not working with ollama.com by @GusBricker in #17191
- fix(docs): remove
source .envby @utsumi-fj in #17466 - fix(streaming): add length validation for empty tool_calls in delta by @DevajMody in #17523
- feat(oci): Enable Oracle Cloud Infrastructure configuration via UI by @jatorre in #17159
- Fix: Gemini image_tokens treated as text tokens in cost calculation by @Chesars in #17554
- fix(responses): Add image generation support for Responses API by @Chesars in #16586
- fix: Handle global location for Vertex AI image generation endpoint by @OlivverX in #17255
- Feat/mcp preserve tool metadata calltoolresult by @uc4w6c in #17561
- [Feature] Customer Usage UI by @yuneng-jiang in #17506
- [Fix] Mock server_root_path for v2/login test by @yuneng-jiang in #17563
- [Fix] CI/CD - Adding end user and org to service types by @yuneng-jiang in #17564
- build(deps): bump mdast-util-to-hast from 13.2.0 to 13.2.1 in /ui/litellm-dashboard by @dependabot[bot] in #17444
- [Feature] Support Images in Compare UI by @yuneng-jiang in #17562
- fix: mcp test by @uc4w6c in #17570
- [Fix] Flaky UI Test by @yuneng-jiang in #17569
- [Fix] Change useAuthorized Hook to redirect to new Login Page by @yuneng-jiang in #17553
- Fix: remove merge conflict markdown by @AlexsanderHamir in https://github.c...
v1.80.8.dev.1
What's Changed
- [Feature] User Info Delete Modal by @yuneng-jiang in #17625
- [Fix] Fix links + old login page deprecation message by @yuneng-jiang in #17624
- [Feature] Logs Spend Enhancements by @yuneng-jiang in #17623
- [Refactor] Reusable Table Icon Button by @yuneng-jiang in #17573
- [Fix] Filtering for Chat UI Endpoint Selector by @yuneng-jiang in #17567
- [Fix] Race Condition Handling in SCIM v2 by @yuneng-jiang in #17513
- [Fix] Make /litellm_model_cost_map public by @yuneng-jiang in #16795
- [Fix] Custom Callback on UI by @yuneng-jiang in #17522
Full Changelog: v1.80.8-nightly...v1.80.8.dev.1
v1.80.8-nightly
What's Changed
- merge main by @Sameerlite in #17408
- Guardrail API - support tool call checks on OpenAI
/chat/completions, OpenAI/responses, Anthropic/v1/messagesby @krrishdholakia in #17459 - docs: document responses and embedding api for github copilot by @codgician in #17456
- fix(openai): add clear error message for empty LLM endpoint responses by @DevajMody in #17445
- Fix: Skip empty text blocks in Anthropic system messages by @HaiyiMei in #17442
- feat: Basic Weave OTEL integration by @andrewtruong in #17439
- fix(ocr): add missing ocr and aocr to CallTypes enum by @Chesars in #17435
- fix(groq): check supports_response_schema before using json_tool_call workaround by @Chesars in #17438
- fix(pricing): add missing Moonshot turbo models and fix incorrect pricing by @Chesars in #17432
- fix: make bedrock converse messages respect modify_params as expected by @danielmklein in #17427
- Update docs to link agent hub by @krrishdholakia in #17462
- [Feature] Add Auto Redirect to SSO to New Login Page by @yuneng-jiang in #17451
- [Infra] UI - Change Credentials to use React Query by @yuneng-jiang in #17465
- Map output_tokens_details of responses API to completion_tokens_details by @Sameerlite in #17458
- Add support for bedrock qwen 2 imported model by @Sameerlite in #17461
- Add support for file content download for bedrock batches by @Sameerlite in #17470
- fix: test_default_api_base for ragfow by @Sameerlite in #17485
- feat - new model amazon.titan-image-generator-v2:0 by @ishaan-jaff in #17488
- Fix aim guardrail tests by @hxyannay in #17499
- fix: resolve ruff lint errors by @AnasAbdelR in #17490
- 🆕 feat: support routing to only websearch supported deployments by @raghav-stripe in #17500
- [Management endpoints] - Allow admin viewer to access global tag usage by @ishaan-jaff in #17501
- [Feat] Agent Access Control - Enforce Allowed agents by key, team + add agent access groups on backend by @ishaan-jaff in #17502
- [Feat] UI - Agent Gateway - set allowed agents by key, team by @ishaan-jaff in #17511
- [Docs] A2a - Permission management by @ishaan-jaff in #17515
- [Feat] Allow adding OpenAI compatible chat providers using .json + add public ai provider by @ishaan-jaff in #17448
- chore: add stack trace to mcp error message by @uc4w6c in #17269
- (fix): empty response + vllm streaming by @dominicfeliton in #17516
- fix(guardrails): mask all matching keywords in content filter by @DevajMody in #17521
- Enable detailed debugging for reference by @krrishdholakia in #17508
- fix(sql): Optimize SpendLogs queries to use timestamp filtering for index usage by @kristianmitk in #17504
- fix(gemini): handle partial JSON chunks after first valid chunk by @Chesars in #17496
- fix(unified_guardrail.py): support during_call event type for unified guardrails by @krrishdholakia in #17514
- Guardrails API - new
structured_messagesparam by @krrishdholakia in #17518 - [Feat] VertexAI - Add BGE Embeddings support by @Sameerlite in #17362
- [Fix] Select in Edit Membership Modal by @yuneng-jiang in #17524
- fix: code quality issues from ruff linter by @AlexsanderHamir in #17536
- Fix: Presidio guardrail test TypeError and license base64 decoding error by @AlexsanderHamir in #17538
- [fix] parse budget:thinking blocks for opus 4.5 by @colinlin-stripe in #17534
- Fix vector store configuration synchronization failure by @Sameerlite in #17525
- Add support for cursor BYOK with its own configuration by @Sameerlite in #17519
- Add background health checks to db by @Sameerlite in #17528
- Fix: Lack of None value checks & update publicai_chat_transformation tests by @AlexsanderHamir in #17539
- Remove hardcoded api key from test file by @krrishdholakia in #17540
- Make sure that media resolution is only for gemini 3 model by @Sameerlite in #17137
- [Bug fix] WatsonX audio transcriptions, don't force content type in request headers by @ishaan-jaff in #17546
- [Infra] Bump LiteLLM Enterprise Version by @yuneng-jiang in #17549
- Fix budget update to allow null max_budget by @ishaan-jaff in #17545
- fix failing vertex tests by @Sameerlite in #17542
- [Feature] Show all callbacks on UI by @yuneng-jiang in #16335
- fix(github_copilot): preserve encrypted_content in reasoning items for multi-turn conversations by @rgshr in #17130
- feat: Add gpt-5.1-codex-max model pricing and configuration by @Chesars in #17541
- Fix: apply_guardrail method and improve test isolation by @AlexsanderHamir in #17555
- Fix: MLflow streaming spans for Anthropic passthrough by @uc4w6c in #17288
- [Fixes] Dynamic Rate Limiter - Dynamic rate limiting token count increases/decreases by 1 instead of actual count + Redis TTL by @ishaan-jaff in #17558
- [Bug fix] Secret Managers Integration - Make email and secret manager operations independent in key management hooks by @ishaan-jaff in #17551
- feat: add Mistral Large 3 model support by @Chesars in #17547
- fix(team): use organization.members instead of deprecated organizatio… by @DevajMody in #17557
- [Feature] Customer (end user) Usage by @yuneng-jiang in #17498
- [Feat] Allow using dynamic rate limit/priority reservation on teams by @ishaan-jaff in #17061
- build(deps): bump jws from 3.2.2 to 3.2.3 in /ui/litellm-dashboard by @dependabot[bot] in #17494
- fix(presidio): handle empty content and error dict responses in guardrails by @dominicfallows in #17489
- fix(bedrock): anthropics beta headers for qwen by @irfansofyana in #17467
- fix(gemini): fix reasoning_content showing duplicate content in streaming responses by @Chesars in #17266
- Fixes #13652 - auth not working with ollama.com by @GusBricker in #17191
- fix(docs): remove
source .envby @utsumi-fj in #17466 - fix(streaming): add length validation for empty tool_calls in delta by @DevajMody in #17523
- feat(oci): Enable Oracle Cloud Infrastructure configuration via UI by @jatorre in #17159
- Fix: Gemini image_tokens treated as text tokens in cost calculation by @Chesars in #17554
- fix(responses): Add image generation support for Responses API by @Chesars in #16586
- fix: Handle global location for Vertex AI image generation endpoint by @OlivverX in #17255
- Feat/mcp preserve tool metadata calltoolresult by @uc4w6c in #17561
- [Feature] Customer Usage UI by @yuneng-jiang in #17506
- [Fix] Mock server_root_path for v2/login test by @yuneng-jiang in #17563
- [Fix] CI/CD - Adding end user and org to service types by @yuneng-jiang in #17564
- build(deps): bump mdast-util-to-hast from 13.2.0 to 13.2.1 in /ui/litellm-dashboard by @dependabot[bot] in #17444
- [Feature] Support Images in Compare UI by @yuneng-jiang in #17562
- fix: mcp test by @uc4w6c in #17570
- [Fix] Flaky UI Test by @yuneng-jiang in #17569
- [Fix] Change useAuthorized Hook to redirect to new Login Page by @yuneng-jiang in #17553
- Fix: remove merge conflict markdown by @AlexsanderHamir in https://github.c...
v1.80.5-stable.1
Full Changelog: v1.80.5-stable...v1.80.5-stable.1
v1.80.7.dev.4
What's Changed
- merge main by @Sameerlite in #17408
- Guardrail API - support tool call checks on OpenAI
/chat/completions, OpenAI/responses, Anthropic/v1/messagesby @krrishdholakia in #17459 - docs: document responses and embedding api for github copilot by @codgician in #17456
- fix(openai): add clear error message for empty LLM endpoint responses by @DevajMody in #17445
- Fix: Skip empty text blocks in Anthropic system messages by @HaiyiMei in #17442
- feat: Basic Weave OTEL integration by @andrewtruong in #17439
- fix(ocr): add missing ocr and aocr to CallTypes enum by @Chesars in #17435
- fix(groq): check supports_response_schema before using json_tool_call workaround by @Chesars in #17438
- fix(pricing): add missing Moonshot turbo models and fix incorrect pricing by @Chesars in #17432
- fix: make bedrock converse messages respect modify_params as expected by @danielmklein in #17427
- Update docs to link agent hub by @krrishdholakia in #17462
- [Feature] Add Auto Redirect to SSO to New Login Page by @yuneng-jiang in #17451
- [Infra] UI - Change Credentials to use React Query by @yuneng-jiang in #17465
- Map output_tokens_details of responses API to completion_tokens_details by @Sameerlite in #17458
- Add support for bedrock qwen 2 imported model by @Sameerlite in #17461
- Add support for file content download for bedrock batches by @Sameerlite in #17470
- fix: test_default_api_base for ragfow by @Sameerlite in #17485
- feat - new model amazon.titan-image-generator-v2:0 by @ishaan-jaff in #17488
- Fix aim guardrail tests by @hxyannay in #17499
- fix: resolve ruff lint errors by @AnasAbdelR in #17490
- 🆕 feat: support routing to only websearch supported deployments by @raghav-stripe in #17500
- [Management endpoints] - Allow admin viewer to access global tag usage by @ishaan-jaff in #17501
- [Feat] Agent Access Control - Enforce Allowed agents by key, team + add agent access groups on backend by @ishaan-jaff in #17502
- [Feat] UI - Agent Gateway - set allowed agents by key, team by @ishaan-jaff in #17511
- [Docs] A2a - Permission management by @ishaan-jaff in #17515
- [Feat] Allow adding OpenAI compatible chat providers using .json + add public ai provider by @ishaan-jaff in #17448
- chore: add stack trace to mcp error message by @uc4w6c in #17269
- (fix): empty response + vllm streaming by @dominicfeliton in #17516
- fix(guardrails): mask all matching keywords in content filter by @DevajMody in #17521
- Enable detailed debugging for reference by @krrishdholakia in #17508
- fix(sql): Optimize SpendLogs queries to use timestamp filtering for index usage by @kristianmitk in #17504
- fix(gemini): handle partial JSON chunks after first valid chunk by @Chesars in #17496
- fix(unified_guardrail.py): support during_call event type for unified guardrails by @krrishdholakia in #17514
- Guardrails API - new
structured_messagesparam by @krrishdholakia in #17518 - [Feat] VertexAI - Add BGE Embeddings support by @Sameerlite in #17362
- [Fix] Select in Edit Membership Modal by @yuneng-jiang in #17524
- fix: code quality issues from ruff linter by @AlexsanderHamir in #17536
- Fix: Presidio guardrail test TypeError and license base64 decoding error by @AlexsanderHamir in #17538
- [fix] parse budget:thinking blocks for opus 4.5 by @colinlin-stripe in #17534
- Fix vector store configuration synchronization failure by @Sameerlite in #17525
- Add support for cursor BYOK with its own configuration by @Sameerlite in #17519
- Add background health checks to db by @Sameerlite in #17528
- Fix: Lack of None value checks & update publicai_chat_transformation tests by @AlexsanderHamir in #17539
- Remove hardcoded api key from test file by @krrishdholakia in #17540
- Make sure that media resolution is only for gemini 3 model by @Sameerlite in #17137
- [Bug fix] WatsonX audio transcriptions, don't force content type in request headers by @ishaan-jaff in #17546
- [Infra] Bump LiteLLM Enterprise Version by @yuneng-jiang in #17549
- Fix budget update to allow null max_budget by @ishaan-jaff in #17545
- fix failing vertex tests by @Sameerlite in #17542
- [Feature] Show all callbacks on UI by @yuneng-jiang in #16335
New Contributors
- @DevajMody made their first contribution in #17445
- @andrewtruong made their first contribution in #17439
- @AnasAbdelR made their first contribution in #17490
- @dominicfeliton made their first contribution in #17516
- @kristianmitk made their first contribution in #17504
Full Changelog: v1.80.7.dev.3...v1.80.7.dev.4
v1.80.7.dev.3
What's Changed
- [Infra] Building UI by @yuneng-jiang in #17212
- [Infra] Rebuilding UI by @yuneng-jiang in #17213
- docs: add OpenAI Agents SDK to projects by @Chesars in #17203
- Upgrade websockets to v15 by @hxyannay in #16734
- fix: do not include plaintext message in exception by @raghav-stripe in #17216
- [Fix] Change Add Fallback Modal to use Antd Select by @yuneng-jiang in #17223
- [Feat] Add guardrails for pass through endpoints by @ishaan-jaff in #17221
- [Feat] UI - allow adding pass through guardrails through UI by @ishaan-jaff in #17226
- [Bug fix] Vector Store List Endpoint Returns 404 by @ishaan-jaff in #17229
- [Feature]: Add Provider publicai.co by @ishaan-jaff in #17230
- [Fix] Request and Response Panel JSONViewer by @yuneng-jiang in #17233
- [Feature] Adding Button Loading States to Edit Settings by @yuneng-jiang in #17236
- [Fix] Various Text, button state, and test changes by @yuneng-jiang in #17237
- [Fix] UI - Fallbacks Immediately Deleting before API resolves by @yuneng-jiang in #17238
- [Fix] Remove Feature Flags by @yuneng-jiang in #17240
- (Bug)Migrate Anthropic provider to azure ai by @Sameerlite in #17202
- Fix/issue 16759 streaming error validation by @weichiet in #17242
- feat: add regex-based tool_name/tool_type matching for tool-permission by @uc4w6c in #17164
- SSO: fix the generic SSO provider by @saar-win in #17227
- fix(bedrock): handle cohere v4 embed response dictionary format by @AndyForest in #17220
- Fix : acompletion throws error with SambaNova models by @omkar806 in #17217
- Allow wildcard routes for nonproxy admin (SCIM) by @v0rtex20k in #17178
- Update new anthropic feats as reviewed by @Sameerlite in #17142
- Fix metadata tags and model name display in UI for Azure passthrough + Add cost tracking for responses API by @Sameerlite in #17258
- Add nova embedding support by @Sameerlite in #17253
- Better handle anonymization by @hxomer in #17207
- Added support for twelvelabs pegasus by @Sameerlite in #17193
- [Refactor] Deprecate
spend/logs& addspend/logs/v2by @AlexsanderHamir in #17167 - [Perf] Fix
/audio/speechperformance by usingshared_sessionsby @AlexsanderHamir in #16739 - Respect custom llm provider in header by @Sameerlite in #17290
- (Feat)Add passthrough cost tracking for veo by @Sameerlite in #17296
- Remove not compatible beta header from Bedrock by @Sameerlite in #17301
- Add better handling in image generation for gemini models by @Sameerlite in #17292
- fix: respect guardrail mock_response during during_call to return blo… by @uc4w6c in #17247
- Fix session consistency, move Lasso API version away from source code by @orgersh92 in #17316
- doc: add images for tool permission guardrail by @uc4w6c in #17322
- [Feat] WatsonX - allow passing zen_api_key dynamically by @ishaan-jaff in #16655
- [Feat] JWT Auth - AI Gateway, allow using regular OIDC flow with user info endpoints by @ishaan-jaff in #17324
- [Feat] Add new model
fireworks_ai /kimi-k2-instruct-0905by @ishaan-jaff in #17328 - (feat) Generic Guardrail API - allows guardrail providers to add INSTANT support for LiteLLM w/out PR to repo by @krrishdholakia in #17175
- Revert "fix: respect guardrail mock_response during during_call to return blo…" by @krrishdholakia in #17332
- Add
claude-opus-4-5alias to pricing data by @dannykopping in #17313 - Add support of audio transcription for OVHcloud by @eliasto in #17305
- [Fix] Fix Watsonx Audio Transcription API by @ishaan-jaff in #17326
- [fix] extra_headers in messages api bedrock invoke by @colinlin-stripe in #17271
- fix: GA path for azure openai realtime models by @abi-jey in #17260
- [Bug Fix] - Fix
litellm_enterpriseensure imported routes exist by @ishaan-jaff in #17337 - Fix/new org team validate against org by @rioiart in #17333
- Fix sso users not added to entra synced team by @rioiart in #17331
- Refactor Noma guardrail to use shared Responses transformation and include system instructions by @idola9 in #17315
- feat(provider): add Z.AI (Zhipu AI) as built-in provider by @Chesars in #17307
- Fix AttributeError when metadata is null in request body by @Chesars in #17306
- SSO(fix): Clear SSO integration for all users by @saar-win in #17287
- feat: add experimental latest-user filtering for Bedrock by @uc4w6c in #17282
- Add context window exception mapping for Together AI by @li-boxuan in #17284
- feat(github-copilot): Add Embedding API support by @codgician in #17278
- Update Databricks model pricing and add new models (including databricks pricing test). by @epistoteles in #17277
- fix: remove URL format validation for MCP server endpoints by @uc4w6c in #17270
- Guardrail API V2 - user api key metadata, session id, specify input type (request/response), image support by @krrishdholakia in #17338
- fix: update default database connection number by @AlexsanderHamir in #17353
- Fix: update default proxy_batch_write_at number by @AlexsanderHamir in #17355
- [Feat] Add
vllmbatch+files API support by @ishaan-jaff in #15823 - Fix 500 error for malformed request by @Sameerlite in #17291
- Fix: litellm user auth not passing issue by @Sameerlite in #17342
- Add other routes in jwt auth by @Sameerlite in #17345
- docs: add Azure AI Foundry documentation for Claude models by @Chesars in #17104
- Litellm bedrock OpenAI model support by @kothamah in #17368
- [Fix] Bedrock Guardrail Indent and Import fix by @yuneng-jiang in #17378
- [Infra] Add Google Private API Endpoint to Vertex AI fields by @yuneng-jiang in #17382
- [Docs] Add guide on how to debug gateway error vs provider error by @ishaan-jaff in #17387
- 🐛 Fix proxy caching between requests in aiohttp transport by @lcfyi in #17122
- [Feat] New model/provider - Adds support for Google Cloud Chirp3 HD on /speech by @ishaan-jaff in #17391
- [Fix] UI - Change Edit Team Models Shown to Match Create Team by @yuneng-jiang in #17394
- [New model] Add
fireworks/deepseek-v3p2by @ishaan-jaff in #17395 - feat: Add Nova lite 2 reasoning support with reasoningConfig by @flozonn in #17371
- [Fix] Show all credential values on Edit Credential Modal by @yuneng-jiang in #17397
- [Refactor] Standardize API Key vs Virtual Key in UI by @yuneng-jiang in #17325
- [Feature] Add User Alias Column to Internal User Table by @yuneng-jiang in #17321
- Fix: Datadog callback regression when ddtrace is installed by @ishaan-jaff in #17393
- Refactor add_schema_to_components to move definitions to components/s… by @richardmcsong in #17389
- Support Deepseek 3.2 with Reasoning by @matt-greathouse in #17384
- Add model price & details for Global Opus 4.5 Bedrock endpoint by @mossbanay in #17380
- docs: Update Instructions For Phoenix Integration by @ialisaleh in #17373
- update model_prices_and_context_window.json by @...
v1.80.7.dev.2
What's Changed
- [Infra] Building UI by @yuneng-jiang in #17212
- [Infra] Rebuilding UI by @yuneng-jiang in #17213
- docs: add OpenAI Agents SDK to projects by @Chesars in #17203
- Upgrade websockets to v15 by @hxyannay in #16734
- fix: do not include plaintext message in exception by @raghav-stripe in #17216
- [Fix] Change Add Fallback Modal to use Antd Select by @yuneng-jiang in #17223
- [Feat] Add guardrails for pass through endpoints by @ishaan-jaff in #17221
- [Feat] UI - allow adding pass through guardrails through UI by @ishaan-jaff in #17226
- [Bug fix] Vector Store List Endpoint Returns 404 by @ishaan-jaff in #17229
- [Feature]: Add Provider publicai.co by @ishaan-jaff in #17230
- [Fix] Request and Response Panel JSONViewer by @yuneng-jiang in #17233
- [Feature] Adding Button Loading States to Edit Settings by @yuneng-jiang in #17236
- [Fix] Various Text, button state, and test changes by @yuneng-jiang in #17237
- [Fix] UI - Fallbacks Immediately Deleting before API resolves by @yuneng-jiang in #17238
- [Fix] Remove Feature Flags by @yuneng-jiang in #17240
- (Bug)Migrate Anthropic provider to azure ai by @Sameerlite in #17202
- Fix/issue 16759 streaming error validation by @weichiet in #17242
- feat: add regex-based tool_name/tool_type matching for tool-permission by @uc4w6c in #17164
- SSO: fix the generic SSO provider by @saar-win in #17227
- fix(bedrock): handle cohere v4 embed response dictionary format by @AndyForest in #17220
- Fix : acompletion throws error with SambaNova models by @omkar806 in #17217
- Allow wildcard routes for nonproxy admin (SCIM) by @v0rtex20k in #17178
- Update new anthropic feats as reviewed by @Sameerlite in #17142
- Fix metadata tags and model name display in UI for Azure passthrough + Add cost tracking for responses API by @Sameerlite in #17258
- Add nova embedding support by @Sameerlite in #17253
- Better handle anonymization by @hxomer in #17207
- Added support for twelvelabs pegasus by @Sameerlite in #17193
- [Refactor] Deprecate
spend/logs& addspend/logs/v2by @AlexsanderHamir in #17167 - [Perf] Fix
/audio/speechperformance by usingshared_sessionsby @AlexsanderHamir in #16739 - Respect custom llm provider in header by @Sameerlite in #17290
- (Feat)Add passthrough cost tracking for veo by @Sameerlite in #17296
- Remove not compatible beta header from Bedrock by @Sameerlite in #17301
- Add better handling in image generation for gemini models by @Sameerlite in #17292
- fix: respect guardrail mock_response during during_call to return blo… by @uc4w6c in #17247
- Fix session consistency, move Lasso API version away from source code by @orgersh92 in #17316
- doc: add images for tool permission guardrail by @uc4w6c in #17322
- [Feat] WatsonX - allow passing zen_api_key dynamically by @ishaan-jaff in #16655
- [Feat] JWT Auth - AI Gateway, allow using regular OIDC flow with user info endpoints by @ishaan-jaff in #17324
- [Feat] Add new model
fireworks_ai /kimi-k2-instruct-0905by @ishaan-jaff in #17328 - (feat) Generic Guardrail API - allows guardrail providers to add INSTANT support for LiteLLM w/out PR to repo by @krrishdholakia in #17175
- Revert "fix: respect guardrail mock_response during during_call to return blo…" by @krrishdholakia in #17332
- Add
claude-opus-4-5alias to pricing data by @dannykopping in #17313 - Add support of audio transcription for OVHcloud by @eliasto in #17305
- [Fix] Fix Watsonx Audio Transcription API by @ishaan-jaff in #17326
- [fix] extra_headers in messages api bedrock invoke by @colinlin-stripe in #17271
- fix: GA path for azure openai realtime models by @abi-jey in #17260
- [Bug Fix] - Fix
litellm_enterpriseensure imported routes exist by @ishaan-jaff in #17337 - Fix/new org team validate against org by @rioiart in #17333
- Fix sso users not added to entra synced team by @rioiart in #17331
- Refactor Noma guardrail to use shared Responses transformation and include system instructions by @idola9 in #17315
- feat(provider): add Z.AI (Zhipu AI) as built-in provider by @Chesars in #17307
- Fix AttributeError when metadata is null in request body by @Chesars in #17306
- SSO(fix): Clear SSO integration for all users by @saar-win in #17287
- feat: add experimental latest-user filtering for Bedrock by @uc4w6c in #17282
- Add context window exception mapping for Together AI by @li-boxuan in #17284
- feat(github-copilot): Add Embedding API support by @codgician in #17278
- Update Databricks model pricing and add new models (including databricks pricing test). by @epistoteles in #17277
- fix: remove URL format validation for MCP server endpoints by @uc4w6c in #17270
- Guardrail API V2 - user api key metadata, session id, specify input type (request/response), image support by @krrishdholakia in #17338
- fix: update default database connection number by @AlexsanderHamir in #17353
- Fix: update default proxy_batch_write_at number by @AlexsanderHamir in #17355
- [Feat] Add
vllmbatch+files API support by @ishaan-jaff in #15823 - Fix 500 error for malformed request by @Sameerlite in #17291
- Fix: litellm user auth not passing issue by @Sameerlite in #17342
- Add other routes in jwt auth by @Sameerlite in #17345
- docs: add Azure AI Foundry documentation for Claude models by @Chesars in #17104
- Litellm bedrock OpenAI model support by @kothamah in #17368
- [Fix] Bedrock Guardrail Indent and Import fix by @yuneng-jiang in #17378
- [Infra] Add Google Private API Endpoint to Vertex AI fields by @yuneng-jiang in #17382
- [Docs] Add guide on how to debug gateway error vs provider error by @ishaan-jaff in #17387
- 🐛 Fix proxy caching between requests in aiohttp transport by @lcfyi in #17122
- [Feat] New model/provider - Adds support for Google Cloud Chirp3 HD on /speech by @ishaan-jaff in #17391
- [Fix] UI - Change Edit Team Models Shown to Match Create Team by @yuneng-jiang in #17394
- [New model] Add
fireworks/deepseek-v3p2by @ishaan-jaff in #17395 - feat: Add Nova lite 2 reasoning support with reasoningConfig by @flozonn in #17371
- [Fix] Show all credential values on Edit Credential Modal by @yuneng-jiang in #17397
- [Refactor] Standardize API Key vs Virtual Key in UI by @yuneng-jiang in #17325
- [Feature] Add User Alias Column to Internal User Table by @yuneng-jiang in #17321
- Fix: Datadog callback regression when ddtrace is installed by @ishaan-jaff in #17393
- Refactor add_schema_to_components to move definitions to components/s… by @richardmcsong in #17389
- Support Deepseek 3.2 with Reasoning by @matt-greathouse in #17384
- Add model price & details for Global Opus 4.5 Bedrock endpoint by @mossbanay in #17380
- docs: Update Instructions For Phoenix Integration by @ialisaleh in #17373
- update model_prices_and_context_window.json by @...
v1.80.7.dev.1
What's Changed
- [Infra] Building UI by @yuneng-jiang in #17212
- [Infra] Rebuilding UI by @yuneng-jiang in #17213
- docs: add OpenAI Agents SDK to projects by @Chesars in #17203
- Upgrade websockets to v15 by @hxyannay in #16734
- fix: do not include plaintext message in exception by @raghav-stripe in #17216
- [Fix] Change Add Fallback Modal to use Antd Select by @yuneng-jiang in #17223
- [Feat] Add guardrails for pass through endpoints by @ishaan-jaff in #17221
- [Feat] UI - allow adding pass through guardrails through UI by @ishaan-jaff in #17226
- [Bug fix] Vector Store List Endpoint Returns 404 by @ishaan-jaff in #17229
- [Feature]: Add Provider publicai.co by @ishaan-jaff in #17230
- [Fix] Request and Response Panel JSONViewer by @yuneng-jiang in #17233
- [Feature] Adding Button Loading States to Edit Settings by @yuneng-jiang in #17236
- [Fix] Various Text, button state, and test changes by @yuneng-jiang in #17237
- [Fix] UI - Fallbacks Immediately Deleting before API resolves by @yuneng-jiang in #17238
- [Fix] Remove Feature Flags by @yuneng-jiang in #17240
- (Bug)Migrate Anthropic provider to azure ai by @Sameerlite in #17202
- Fix/issue 16759 streaming error validation by @weichiet in #17242
- feat: add regex-based tool_name/tool_type matching for tool-permission by @uc4w6c in #17164
- SSO: fix the generic SSO provider by @saar-win in #17227
- fix(bedrock): handle cohere v4 embed response dictionary format by @AndyForest in #17220
- Fix : acompletion throws error with SambaNova models by @omkar806 in #17217
- Allow wildcard routes for nonproxy admin (SCIM) by @v0rtex20k in #17178
- Update new anthropic feats as reviewed by @Sameerlite in #17142
- Fix metadata tags and model name display in UI for Azure passthrough + Add cost tracking for responses API by @Sameerlite in #17258
- Add nova embedding support by @Sameerlite in #17253
- Better handle anonymization by @hxomer in #17207
- Added support for twelvelabs pegasus by @Sameerlite in #17193
- [Refactor] Deprecate
spend/logs& addspend/logs/v2by @AlexsanderHamir in #17167 - [Perf] Fix
/audio/speechperformance by usingshared_sessionsby @AlexsanderHamir in #16739 - Respect custom llm provider in header by @Sameerlite in #17290
- (Feat)Add passthrough cost tracking for veo by @Sameerlite in #17296
- Remove not compatible beta header from Bedrock by @Sameerlite in #17301
- Add better handling in image generation for gemini models by @Sameerlite in #17292
- fix: respect guardrail mock_response during during_call to return blo… by @uc4w6c in #17247
- Fix session consistency, move Lasso API version away from source code by @orgersh92 in #17316
- doc: add images for tool permission guardrail by @uc4w6c in #17322
- [Feat] WatsonX - allow passing zen_api_key dynamically by @ishaan-jaff in #16655
- [Feat] JWT Auth - AI Gateway, allow using regular OIDC flow with user info endpoints by @ishaan-jaff in #17324
- [Feat] Add new model
fireworks_ai /kimi-k2-instruct-0905by @ishaan-jaff in #17328 - (feat) Generic Guardrail API - allows guardrail providers to add INSTANT support for LiteLLM w/out PR to repo by @krrishdholakia in #17175
- Revert "fix: respect guardrail mock_response during during_call to return blo…" by @krrishdholakia in #17332
- Add
claude-opus-4-5alias to pricing data by @dannykopping in #17313 - Add support of audio transcription for OVHcloud by @eliasto in #17305
- [Fix] Fix Watsonx Audio Transcription API by @ishaan-jaff in #17326
- [fix] extra_headers in messages api bedrock invoke by @colinlin-stripe in #17271
- fix: GA path for azure openai realtime models by @abi-jey in #17260
- [Bug Fix] - Fix
litellm_enterpriseensure imported routes exist by @ishaan-jaff in #17337 - Fix/new org team validate against org by @rioiart in #17333
- Fix sso users not added to entra synced team by @rioiart in #17331
- Refactor Noma guardrail to use shared Responses transformation and include system instructions by @idola9 in #17315
- feat(provider): add Z.AI (Zhipu AI) as built-in provider by @Chesars in #17307
- Fix AttributeError when metadata is null in request body by @Chesars in #17306
- SSO(fix): Clear SSO integration for all users by @saar-win in #17287
- feat: add experimental latest-user filtering for Bedrock by @uc4w6c in #17282
- Add context window exception mapping for Together AI by @li-boxuan in #17284
- feat(github-copilot): Add Embedding API support by @codgician in #17278
- Update Databricks model pricing and add new models (including databricks pricing test). by @epistoteles in #17277
- fix: remove URL format validation for MCP server endpoints by @uc4w6c in #17270
- Guardrail API V2 - user api key metadata, session id, specify input type (request/response), image support by @krrishdholakia in #17338
- fix: update default database connection number by @AlexsanderHamir in #17353
- Fix: update default proxy_batch_write_at number by @AlexsanderHamir in #17355
- [Feat] Add
vllmbatch+files API support by @ishaan-jaff in #15823 - Fix 500 error for malformed request by @Sameerlite in #17291
- Fix: litellm user auth not passing issue by @Sameerlite in #17342
- Add other routes in jwt auth by @Sameerlite in #17345
- docs: add Azure AI Foundry documentation for Claude models by @Chesars in #17104
- Litellm bedrock OpenAI model support by @kothamah in #17368
- [Fix] Bedrock Guardrail Indent and Import fix by @yuneng-jiang in #17378
- [Infra] Add Google Private API Endpoint to Vertex AI fields by @yuneng-jiang in #17382
- [Docs] Add guide on how to debug gateway error vs provider error by @ishaan-jaff in #17387
- 🐛 Fix proxy caching between requests in aiohttp transport by @lcfyi in #17122
- [Feat] New model/provider - Adds support for Google Cloud Chirp3 HD on /speech by @ishaan-jaff in #17391
New Contributors
- @hxyannay made their first contribution in #16734
- @weichiet made their first contribution in #17242
- @AndyForest made their first contribution in #17220
- @omkar806 made their first contribution in #17217
- @v0rtex20k made their first contribution in #17178
- @hxomer made their first contribution in #17207
- @orgersh92 made their first contribution in #17316
- @dannykopping made their first contribution in #17313
- @codgician made their first contribution in #17278
- @epistoteles made their first contribution in #17277
- @kothamah made their first contribution in #17368
Full Changelog: v1.80.6-nightly...v1.80.7.dev.1