v1.80.9.dev1

What's Changed

• Add Helicone as a provider and update observability documentation by @juliettech13 in #17663
• [Docs] - add sap gen ai provider on LiteLLM by @ishaan-jaff in #17667
• Move query params to create_pass_through_route call in Langfuse passthrough by @jason-nance in #17660
• Update litellm docs drop_params by @ishaan-jaff in #17658
• docs: improve Getting Started page and SDK documentation structure by @Chesars in #17614
• [Feat] New model - add nvidia nim llama-3.2-nv-rerankqa-1b-v2 by @ishaan-jaff in #17670
• Update integration docs location by @krrishdholakia in #17644
• [Feat] Dynamic Rate Limiter - allow specifying ttl for in memory cache by @ishaan-jaff in #17679

New Contributors

• @juliettech13 made their first contribution in #17663
• @jason-nance made their first contribution in #17660

Full Changelog: v1.80.9-nightly...v1.80.9.dev1

v1.80.9-nightly

What's Changed

• [Fix] Add User Writable Directory to Non Root Docker for Logo by @yuneng-jiang in #17180
• Add polling via cache feature by @xianzongxie-stripe in #16862
• Add New Bedrock OSS Models to Model List by @marty-sullivan in #17638
• docs(contributing): update clone instructions to recommend forking first by @krisxia0506 in #17637
• correct model type for gpt-image-1-mini by @emerzon in #17635
• Native Webhook Integration Sumologic by @chetanchoudhary-sumo in #17630
• fix(anthropic): prevent duplicate tool_result blocks with same tool by @kevinmarx in #17632
• Helm chart extraResources by @expruc in #17627
• fix(proxy): async_post_call_streaming_iterator_hook now properly iterates async generators by @rcII in #17626
• add onyx guardrail hooks integration by @tamirkiviti13 in #16591
• fix(anthropic): handle partial JSON chunks in streaming responses by @Chesars in #17493
• Document missing environment variables and fix incorrect types by @AlexsanderHamir in #17649
• Add usage details in responses usage object by @Sameerlite in #17641
• Fix error for response api polling by @Sameerlite in #17654
• Use audio content for caching by @Sameerlite in #17651
• (Feat) Add fireworks rerank support by @Sameerlite in #17653
• fix: Return 403 exception when calling GET responses api by @Eric84626 in #17629
• Fix flanky tests by @AlexsanderHamir in #17665
• [New provider] Sap gen ai hub by @vasilisazayka in #16053

New Contributors

• @xianzongxie-stripe made their first contribution in #16862
• @krisxia0506 made their first contribution in #17637
• @chetanchoudhary-sumo made their first contribution in #17630
• @kevinmarx made their first contribution in #17632
• @expruc made their first contribution in #17627
• @rcII made their first contribution in #17626
• @tamirkiviti13 made their first contribution in #16591
• @Eric84626 made their first contribution in #17629
• @vasilisazayka made their first contribution in #16053

Full Changelog: v1.80.8.dev.1...v1.80.9-nightly

v1.80.8.rc.1

What's Changed

• merge main by @Sameerlite in #17408
• Guardrail API - support tool call checks on OpenAI /chat/completions, OpenAI /responses, Anthropic /v1/messages by @krrishdholakia in #17459
• docs: document responses and embedding api for github copilot by @codgician in #17456
• fix(openai): add clear error message for empty LLM endpoint responses by @DevajMody in #17445
• Fix: Skip empty text blocks in Anthropic system messages by @HaiyiMei in #17442
• feat: Basic Weave OTEL integration by @andrewtruong in #17439
• fix(ocr): add missing ocr and aocr to CallTypes enum by @Chesars in #17435
• fix(groq): check supports_response_schema before using json_tool_call workaround by @Chesars in #17438
• fix(pricing): add missing Moonshot turbo models and fix incorrect pricing by @Chesars in #17432
• fix: make bedrock converse messages respect modify_params as expected by @danielmklein in #17427
• Update docs to link agent hub by @krrishdholakia in #17462
• [Feature] Add Auto Redirect to SSO to New Login Page by @yuneng-jiang in #17451
• [Infra] UI - Change Credentials to use React Query by @yuneng-jiang in #17465
• Map output_tokens_details of responses API to completion_tokens_details by @Sameerlite in #17458
• Add support for bedrock qwen 2 imported model by @Sameerlite in #17461
• Add support for file content download for bedrock batches by @Sameerlite in #17470
• fix: test_default_api_base for ragfow by @Sameerlite in #17485
• feat - new model amazon.titan-image-generator-v2:0 by @ishaan-jaff in #17488
• Fix aim guardrail tests by @hxyannay in #17499
• fix: resolve ruff lint errors by @AnasAbdelR in #17490
• 🆕 feat: support routing to only websearch supported deployments by @raghav-stripe in #17500
• [Management endpoints] - Allow admin viewer to access global tag usage by @ishaan-jaff in #17501
• [Feat] Agent Access Control - Enforce Allowed agents by key, team + add agent access groups on backend by @ishaan-jaff in #17502
• [Feat] UI - Agent Gateway - set allowed agents by key, team by @ishaan-jaff in #17511
• [Docs] A2a - Permission management by @ishaan-jaff in #17515
• [Feat] Allow adding OpenAI compatible chat providers using .json + add public ai provider by @ishaan-jaff in #17448
• chore: add stack trace to mcp error message by @uc4w6c in #17269
• (fix): empty response + vllm streaming by @dominicfeliton in #17516
• fix(guardrails): mask all matching keywords in content filter by @DevajMody in #17521
• Enable detailed debugging for reference by @krrishdholakia in #17508
• fix(sql): Optimize SpendLogs queries to use timestamp filtering for index usage by @kristianmitk in #17504
• fix(gemini): handle partial JSON chunks after first valid chunk by @Chesars in #17496
• fix(unified_guardrail.py): support during_call event type for unified guardrails by @krrishdholakia in #17514
• Guardrails API - new structured_messages param by @krrishdholakia in #17518
• [Feat] VertexAI - Add BGE Embeddings support by @Sameerlite in #17362
• [Fix] Select in Edit Membership Modal by @yuneng-jiang in #17524
• fix: code quality issues from ruff linter by @AlexsanderHamir in #17536
• Fix: Presidio guardrail test TypeError and license base64 decoding error by @AlexsanderHamir in #17538
• [fix] parse budget:thinking blocks for opus 4.5 by @colinlin-stripe in #17534
• Fix vector store configuration synchronization failure by @Sameerlite in #17525
• Add support for cursor BYOK with its own configuration by @Sameerlite in #17519
• Add background health checks to db by @Sameerlite in #17528
• Fix: Lack of None value checks & update publicai_chat_transformation tests by @AlexsanderHamir in #17539
• Remove hardcoded api key from test file by @krrishdholakia in #17540
• Make sure that media resolution is only for gemini 3 model by @Sameerlite in #17137
• [Bug fix] WatsonX audio transcriptions, don't force content type in request headers by @ishaan-jaff in #17546
• [Infra] Bump LiteLLM Enterprise Version by @yuneng-jiang in #17549
• Fix budget update to allow null max_budget by @ishaan-jaff in #17545
• fix failing vertex tests by @Sameerlite in #17542
• [Feature] Show all callbacks on UI by @yuneng-jiang in #16335
• fix(github_copilot): preserve encrypted_content in reasoning items for multi-turn conversations by @rgshr in #17130
• feat: Add gpt-5.1-codex-max model pricing and configuration by @Chesars in #17541
• Fix: apply_guardrail method and improve test isolation by @AlexsanderHamir in #17555
• Fix: MLflow streaming spans for Anthropic passthrough by @uc4w6c in #17288
• [Fixes] Dynamic Rate Limiter - Dynamic rate limiting token count increases/decreases by 1 instead of actual count + Redis TTL by @ishaan-jaff in #17558
• [Bug fix] Secret Managers Integration - Make email and secret manager operations independent in key management hooks by @ishaan-jaff in #17551
• feat: add Mistral Large 3 model support by @Chesars in #17547
• fix(team): use organization.members instead of deprecated organizatio… by @DevajMody in #17557
• [Feature] Customer (end user) Usage by @yuneng-jiang in #17498
• [Feat] Allow using dynamic rate limit/priority reservation on teams by @ishaan-jaff in #17061
• build(deps): bump jws from 3.2.2 to 3.2.3 in /ui/litellm-dashboard by @dependabot[bot] in #17494
• fix(presidio): handle empty content and error dict responses in guardrails by @dominicfallows in #17489
• fix(bedrock): anthropics beta headers for qwen by @irfansofyana in #17467
• fix(gemini): fix reasoning_content showing duplicate content in streaming responses by @Chesars in #17266
• Fixes #13652 - auth not working with ollama.com by @GusBricker in #17191
• fix(docs): remove source .env by @utsumi-fj in #17466
• fix(streaming): add length validation for empty tool_calls in delta by @DevajMody in #17523
• feat(oci): Enable Oracle Cloud Infrastructure configuration via UI by @jatorre in #17159
• Fix: Gemini image_tokens treated as text tokens in cost calculation by @Chesars in #17554
• fix(responses): Add image generation support for Responses API by @Chesars in #16586
• fix: Handle global location for Vertex AI image generation endpoint by @OlivverX in #17255
• Feat/mcp preserve tool metadata calltoolresult by @uc4w6c in #17561
• [Feature] Customer Usage UI by @yuneng-jiang in #17506
• [Fix] Mock server_root_path for v2/login test by @yuneng-jiang in #17563
• [Fix] CI/CD - Adding end user and org to service types by @yuneng-jiang in #17564
• build(deps): bump mdast-util-to-hast from 13.2.0 to 13.2.1 in /ui/litellm-dashboard by @dependabot[bot] in #17444
• [Feature] Support Images in Compare UI by @yuneng-jiang in #17562
• fix: mcp test by @uc4w6c in #17570
• [Fix] Flaky UI Test by @yuneng-jiang in #17569
• [Fix] Change useAuthorized Hook to redirect to new Login Page by @yuneng-jiang in #17553
• Fix: remove merge conflict markdown by @AlexsanderHamir in https://github.c...

v1.80.8.dev.1

What's Changed

• [Feature] User Info Delete Modal by @yuneng-jiang in #17625
• [Fix] Fix links + old login page deprecation message by @yuneng-jiang in #17624
• [Feature] Logs Spend Enhancements by @yuneng-jiang in #17623
• [Refactor] Reusable Table Icon Button by @yuneng-jiang in #17573
• [Fix] Filtering for Chat UI Endpoint Selector by @yuneng-jiang in #17567
• [Fix] Race Condition Handling in SCIM v2 by @yuneng-jiang in #17513
• [Fix] Make /litellm_model_cost_map public by @yuneng-jiang in #16795
• [Fix] Custom Callback on UI by @yuneng-jiang in #17522

Full Changelog: v1.80.8-nightly...v1.80.8.dev.1

v1.80.8-nightly

What's Changed

• merge main by @Sameerlite in #17408
• Guardrail API - support tool call checks on OpenAI /chat/completions, OpenAI /responses, Anthropic /v1/messages by @krrishdholakia in #17459
• docs: document responses and embedding api for github copilot by @codgician in #17456
• fix(openai): add clear error message for empty LLM endpoint responses by @DevajMody in #17445
• Fix: Skip empty text blocks in Anthropic system messages by @HaiyiMei in #17442
• feat: Basic Weave OTEL integration by @andrewtruong in #17439
• fix(ocr): add missing ocr and aocr to CallTypes enum by @Chesars in #17435
• fix(groq): check supports_response_schema before using json_tool_call workaround by @Chesars in #17438
• fix(pricing): add missing Moonshot turbo models and fix incorrect pricing by @Chesars in #17432
• fix: make bedrock converse messages respect modify_params as expected by @danielmklein in #17427
• Update docs to link agent hub by @krrishdholakia in #17462
• [Feature] Add Auto Redirect to SSO to New Login Page by @yuneng-jiang in #17451
• [Infra] UI - Change Credentials to use React Query by @yuneng-jiang in #17465
• Map output_tokens_details of responses API to completion_tokens_details by @Sameerlite in #17458
• Add support for bedrock qwen 2 imported model by @Sameerlite in #17461
• Add support for file content download for bedrock batches by @Sameerlite in #17470
• fix: test_default_api_base for ragfow by @Sameerlite in #17485
• feat - new model amazon.titan-image-generator-v2:0 by @ishaan-jaff in #17488
• Fix aim guardrail tests by @hxyannay in #17499
• fix: resolve ruff lint errors by @AnasAbdelR in #17490
• 🆕 feat: support routing to only websearch supported deployments by @raghav-stripe in #17500
• [Management endpoints] - Allow admin viewer to access global tag usage by @ishaan-jaff in #17501
• [Feat] Agent Access Control - Enforce Allowed agents by key, team + add agent access groups on backend by @ishaan-jaff in #17502
• [Feat] UI - Agent Gateway - set allowed agents by key, team by @ishaan-jaff in #17511
• [Docs] A2a - Permission management by @ishaan-jaff in #17515
• [Feat] Allow adding OpenAI compatible chat providers using .json + add public ai provider by @ishaan-jaff in #17448
• chore: add stack trace to mcp error message by @uc4w6c in #17269
• (fix): empty response + vllm streaming by @dominicfeliton in #17516
• fix(guardrails): mask all matching keywords in content filter by @DevajMody in #17521
• Enable detailed debugging for reference by @krrishdholakia in #17508
• fix(sql): Optimize SpendLogs queries to use timestamp filtering for index usage by @kristianmitk in #17504
• fix(gemini): handle partial JSON chunks after first valid chunk by @Chesars in #17496
• fix(unified_guardrail.py): support during_call event type for unified guardrails by @krrishdholakia in #17514
• Guardrails API - new structured_messages param by @krrishdholakia in #17518
• [Feat] VertexAI - Add BGE Embeddings support by @Sameerlite in #17362
• [Fix] Select in Edit Membership Modal by @yuneng-jiang in #17524
• fix: code quality issues from ruff linter by @AlexsanderHamir in #17536
• Fix: Presidio guardrail test TypeError and license base64 decoding error by @AlexsanderHamir in #17538
• [fix] parse budget:thinking blocks for opus 4.5 by @colinlin-stripe in #17534
• Fix vector store configuration synchronization failure by @Sameerlite in #17525
• Add support for cursor BYOK with its own configuration by @Sameerlite in #17519
• Add background health checks to db by @Sameerlite in #17528
• Fix: Lack of None value checks & update publicai_chat_transformation tests by @AlexsanderHamir in #17539
• Remove hardcoded api key from test file by @krrishdholakia in #17540
• Make sure that media resolution is only for gemini 3 model by @Sameerlite in #17137
• [Bug fix] WatsonX audio transcriptions, don't force content type in request headers by @ishaan-jaff in #17546
• [Infra] Bump LiteLLM Enterprise Version by @yuneng-jiang in #17549
• Fix budget update to allow null max_budget by @ishaan-jaff in #17545
• fix failing vertex tests by @Sameerlite in #17542
• [Feature] Show all callbacks on UI by @yuneng-jiang in #16335
• fix(github_copilot): preserve encrypted_content in reasoning items for multi-turn conversations by @rgshr in #17130
• feat: Add gpt-5.1-codex-max model pricing and configuration by @Chesars in #17541
• Fix: apply_guardrail method and improve test isolation by @AlexsanderHamir in #17555
• Fix: MLflow streaming spans for Anthropic passthrough by @uc4w6c in #17288
• [Fixes] Dynamic Rate Limiter - Dynamic rate limiting token count increases/decreases by 1 instead of actual count + Redis TTL by @ishaan-jaff in #17558
• [Bug fix] Secret Managers Integration - Make email and secret manager operations independent in key management hooks by @ishaan-jaff in #17551
• feat: add Mistral Large 3 model support by @Chesars in #17547
• fix(team): use organization.members instead of deprecated organizatio… by @DevajMody in #17557
• [Feature] Customer (end user) Usage by @yuneng-jiang in #17498
• [Feat] Allow using dynamic rate limit/priority reservation on teams by @ishaan-jaff in #17061
• build(deps): bump jws from 3.2.2 to 3.2.3 in /ui/litellm-dashboard by @dependabot[bot] in #17494
• fix(presidio): handle empty content and error dict responses in guardrails by @dominicfallows in #17489
• fix(bedrock): anthropics beta headers for qwen by @irfansofyana in #17467
• fix(gemini): fix reasoning_content showing duplicate content in streaming responses by @Chesars in #17266
• Fixes #13652 - auth not working with ollama.com by @GusBricker in #17191
• fix(docs): remove source .env by @utsumi-fj in #17466
• fix(streaming): add length validation for empty tool_calls in delta by @DevajMody in #17523
• feat(oci): Enable Oracle Cloud Infrastructure configuration via UI by @jatorre in #17159
• Fix: Gemini image_tokens treated as text tokens in cost calculation by @Chesars in #17554
• fix(responses): Add image generation support for Responses API by @Chesars in #16586
• fix: Handle global location for Vertex AI image generation endpoint by @OlivverX in #17255
• Feat/mcp preserve tool metadata calltoolresult by @uc4w6c in #17561
• [Feature] Customer Usage UI by @yuneng-jiang in #17506
• [Fix] Mock server_root_path for v2/login test by @yuneng-jiang in #17563
• [Fix] CI/CD - Adding end user and org to service types by @yuneng-jiang in #17564
• build(deps): bump mdast-util-to-hast from 13.2.0 to 13.2.1 in /ui/litellm-dashboard by @dependabot[bot] in #17444
• [Feature] Support Images in Compare UI by @yuneng-jiang in #17562
• fix: mcp test by @uc4w6c in #17570
• [Fix] Flaky UI Test by @yuneng-jiang in #17569
• [Fix] Change useAuthorized Hook to redirect to new Login Page by @yuneng-jiang in #17553
• Fix: remove merge conflict markdown by @AlexsanderHamir in https://github.c...

v1.80.5-stable.1

Full Changelog: v1.80.5-stable...v1.80.5-stable.1

v1.80.7.dev.4

What's Changed

• merge main by @Sameerlite in #17408
• Guardrail API - support tool call checks on OpenAI /chat/completions, OpenAI /responses, Anthropic /v1/messages by @krrishdholakia in #17459
• docs: document responses and embedding api for github copilot by @codgician in #17456
• fix(openai): add clear error message for empty LLM endpoint responses by @DevajMody in #17445
• Fix: Skip empty text blocks in Anthropic system messages by @HaiyiMei in #17442
• feat: Basic Weave OTEL integration by @andrewtruong in #17439
• fix(ocr): add missing ocr and aocr to CallTypes enum by @Chesars in #17435
• fix(groq): check supports_response_schema before using json_tool_call workaround by @Chesars in #17438
• fix(pricing): add missing Moonshot turbo models and fix incorrect pricing by @Chesars in #17432
• fix: make bedrock converse messages respect modify_params as expected by @danielmklein in #17427
• Update docs to link agent hub by @krrishdholakia in #17462
• [Feature] Add Auto Redirect to SSO to New Login Page by @yuneng-jiang in #17451
• [Infra] UI - Change Credentials to use React Query by @yuneng-jiang in #17465
• Map output_tokens_details of responses API to completion_tokens_details by @Sameerlite in #17458
• Add support for bedrock qwen 2 imported model by @Sameerlite in #17461
• Add support for file content download for bedrock batches by @Sameerlite in #17470
• fix: test_default_api_base for ragfow by @Sameerlite in #17485
• feat - new model amazon.titan-image-generator-v2:0 by @ishaan-jaff in #17488
• Fix aim guardrail tests by @hxyannay in #17499
• fix: resolve ruff lint errors by @AnasAbdelR in #17490
• 🆕 feat: support routing to only websearch supported deployments by @raghav-stripe in #17500
• [Management endpoints] - Allow admin viewer to access global tag usage by @ishaan-jaff in #17501
• [Feat] Agent Access Control - Enforce Allowed agents by key, team + add agent access groups on backend by @ishaan-jaff in #17502
• [Feat] UI - Agent Gateway - set allowed agents by key, team by @ishaan-jaff in #17511
• [Docs] A2a - Permission management by @ishaan-jaff in #17515
• [Feat] Allow adding OpenAI compatible chat providers using .json + add public ai provider by @ishaan-jaff in #17448
• chore: add stack trace to mcp error message by @uc4w6c in #17269
• (fix): empty response + vllm streaming by @dominicfeliton in #17516
• fix(guardrails): mask all matching keywords in content filter by @DevajMody in #17521
• Enable detailed debugging for reference by @krrishdholakia in #17508
• fix(sql): Optimize SpendLogs queries to use timestamp filtering for index usage by @kristianmitk in #17504
• fix(gemini): handle partial JSON chunks after first valid chunk by @Chesars in #17496
• fix(unified_guardrail.py): support during_call event type for unified guardrails by @krrishdholakia in #17514
• Guardrails API - new structured_messages param by @krrishdholakia in #17518
• [Feat] VertexAI - Add BGE Embeddings support by @Sameerlite in #17362
• [Fix] Select in Edit Membership Modal by @yuneng-jiang in #17524
• fix: code quality issues from ruff linter by @AlexsanderHamir in #17536
• Fix: Presidio guardrail test TypeError and license base64 decoding error by @AlexsanderHamir in #17538
• [fix] parse budget:thinking blocks for opus 4.5 by @colinlin-stripe in #17534
• Fix vector store configuration synchronization failure by @Sameerlite in #17525
• Add support for cursor BYOK with its own configuration by @Sameerlite in #17519
• Add background health checks to db by @Sameerlite in #17528
• Fix: Lack of None value checks & update publicai_chat_transformation tests by @AlexsanderHamir in #17539
• Remove hardcoded api key from test file by @krrishdholakia in #17540
• Make sure that media resolution is only for gemini 3 model by @Sameerlite in #17137
• [Bug fix] WatsonX audio transcriptions, don't force content type in request headers by @ishaan-jaff in #17546
• [Infra] Bump LiteLLM Enterprise Version by @yuneng-jiang in #17549
• Fix budget update to allow null max_budget by @ishaan-jaff in #17545
• fix failing vertex tests by @Sameerlite in #17542
• [Feature] Show all callbacks on UI by @yuneng-jiang in #16335

New Contributors

• @DevajMody made their first contribution in #17445
• @andrewtruong made their first contribution in #17439
• @AnasAbdelR made their first contribution in #17490
• @dominicfeliton made their first contribution in #17516
• @kristianmitk made their first contribution in #17504

Full Changelog: v1.80.7.dev.3...v1.80.7.dev.4

v1.80.7.dev.3

What's Changed

• [Infra] Building UI by @yuneng-jiang in #17212
• [Infra] Rebuilding UI by @yuneng-jiang in #17213
• docs: add OpenAI Agents SDK to projects by @Chesars in #17203
• Upgrade websockets to v15 by @hxyannay in #16734
• fix: do not include plaintext message in exception by @raghav-stripe in #17216
• [Fix] Change Add Fallback Modal to use Antd Select by @yuneng-jiang in #17223
• [Feat] Add guardrails for pass through endpoints by @ishaan-jaff in #17221
• [Feat] UI - allow adding pass through guardrails through UI by @ishaan-jaff in #17226
• [Bug fix] Vector Store List Endpoint Returns 404 by @ishaan-jaff in #17229
• [Feature]: Add Provider publicai.co by @ishaan-jaff in #17230
• [Fix] Request and Response Panel JSONViewer by @yuneng-jiang in #17233
• [Feature] Adding Button Loading States to Edit Settings by @yuneng-jiang in #17236
• [Fix] Various Text, button state, and test changes by @yuneng-jiang in #17237
• [Fix] UI - Fallbacks Immediately Deleting before API resolves by @yuneng-jiang in #17238
• [Fix] Remove Feature Flags by @yuneng-jiang in #17240
• (Bug)Migrate Anthropic provider to azure ai by @Sameerlite in #17202
• Fix/issue 16759 streaming error validation by @weichiet in #17242
• feat: add regex-based tool_name/tool_type matching for tool-permission by @uc4w6c in #17164
• SSO: fix the generic SSO provider by @saar-win in #17227
• fix(bedrock): handle cohere v4 embed response dictionary format by @AndyForest in #17220
• Fix : acompletion throws error with SambaNova models by @omkar806 in #17217
• Allow wildcard routes for nonproxy admin (SCIM) by @v0rtex20k in #17178
• Update new anthropic feats as reviewed by @Sameerlite in #17142
• Fix metadata tags and model name display in UI for Azure passthrough + Add cost tracking for responses API by @Sameerlite in #17258
• Add nova embedding support by @Sameerlite in #17253
• Better handle anonymization by @hxomer in #17207
• Added support for twelvelabs pegasus by @Sameerlite in #17193
• [Refactor] Deprecate spend/logs & add spend/logs/v2 by @AlexsanderHamir in #17167
• [Perf] Fix /audio/speech performance by using shared_sessions by @AlexsanderHamir in #16739
• Respect custom llm provider in header by @Sameerlite in #17290
• (Feat)Add passthrough cost tracking for veo by @Sameerlite in #17296
• Remove not compatible beta header from Bedrock by @Sameerlite in #17301
• Add better handling in image generation for gemini models by @Sameerlite in #17292
• fix: respect guardrail mock_response during during_call to return blo… by @uc4w6c in #17247
• Fix session consistency, move Lasso API version away from source code by @orgersh92 in #17316
• doc: add images for tool permission guardrail by @uc4w6c in #17322
• [Feat] WatsonX - allow passing zen_api_key dynamically by @ishaan-jaff in #16655
• [Feat] JWT Auth - AI Gateway, allow using regular OIDC flow with user info endpoints by @ishaan-jaff in #17324
• [Feat] Add new modelfireworks_ai /kimi-k2-instruct-0905 by @ishaan-jaff in #17328
• (feat) Generic Guardrail API - allows guardrail providers to add INSTANT support for LiteLLM w/out PR to repo by @krrishdholakia in #17175
• Revert "fix: respect guardrail mock_response during during_call to return blo…" by @krrishdholakia in #17332
• Add claude-opus-4-5 alias to pricing data by @dannykopping in #17313
• Add support of audio transcription for OVHcloud by @eliasto in #17305
• [Fix] Fix Watsonx Audio Transcription API by @ishaan-jaff in #17326
• [fix] extra_headers in messages api bedrock invoke by @colinlin-stripe in #17271
• fix: GA path for azure openai realtime models by @abi-jey in #17260
• [Bug Fix] - Fix litellm_enterprise ensure imported routes exist by @ishaan-jaff in #17337
• Fix/new org team validate against org by @rioiart in #17333
• Fix sso users not added to entra synced team by @rioiart in #17331
• Refactor Noma guardrail to use shared Responses transformation and include system instructions by @idola9 in #17315
• feat(provider): add Z.AI (Zhipu AI) as built-in provider by @Chesars in #17307
• Fix AttributeError when metadata is null in request body by @Chesars in #17306
• SSO(fix): Clear SSO integration for all users by @saar-win in #17287
• feat: add experimental latest-user filtering for Bedrock by @uc4w6c in #17282
• Add context window exception mapping for Together AI by @li-boxuan in #17284
• feat(github-copilot): Add Embedding API support by @codgician in #17278
• Update Databricks model pricing and add new models (including databricks pricing test). by @epistoteles in #17277
• fix: remove URL format validation for MCP server endpoints by @uc4w6c in #17270
• Guardrail API V2 - user api key metadata, session id, specify input type (request/response), image support by @krrishdholakia in #17338
• fix: update default database connection number by @AlexsanderHamir in #17353
• Fix: update default proxy_batch_write_at number by @AlexsanderHamir in #17355
• [Feat] Add vllm batch+files API support by @ishaan-jaff in #15823
• Fix 500 error for malformed request by @Sameerlite in #17291
• Fix: litellm user auth not passing issue by @Sameerlite in #17342
• Add other routes in jwt auth by @Sameerlite in #17345
• docs: add Azure AI Foundry documentation for Claude models by @Chesars in #17104
• Litellm bedrock OpenAI model support by @kothamah in #17368
• [Fix] Bedrock Guardrail Indent and Import fix by @yuneng-jiang in #17378
• [Infra] Add Google Private API Endpoint to Vertex AI fields by @yuneng-jiang in #17382
• [Docs] Add guide on how to debug gateway error vs provider error by @ishaan-jaff in #17387
• 🐛 Fix proxy caching between requests in aiohttp transport by @lcfyi in #17122
• [Feat] New model/provider - Adds support for Google Cloud Chirp3 HD on /speech by @ishaan-jaff in #17391
• [Fix] UI - Change Edit Team Models Shown to Match Create Team by @yuneng-jiang in #17394
• [New model] Add fireworks/deepseek-v3p2 by @ishaan-jaff in #17395
• feat: Add Nova lite 2 reasoning support with reasoningConfig by @flozonn in #17371
• [Fix] Show all credential values on Edit Credential Modal by @yuneng-jiang in #17397
• [Refactor] Standardize API Key vs Virtual Key in UI by @yuneng-jiang in #17325
• [Feature] Add User Alias Column to Internal User Table by @yuneng-jiang in #17321
• Fix: Datadog callback regression when ddtrace is installed by @ishaan-jaff in #17393
• Refactor add_schema_to_components to move definitions to components/s… by @richardmcsong in #17389
• Support Deepseek 3.2 with Reasoning by @matt-greathouse in #17384
• Add model price & details for Global Opus 4.5 Bedrock endpoint by @mossbanay in #17380
• docs: Update Instructions For Phoenix Integration by @ialisaleh in #17373
• update model_prices_and_context_window.json by @...

v1.80.7.dev.2

What's Changed

• [Infra] Building UI by @yuneng-jiang in #17212
• [Infra] Rebuilding UI by @yuneng-jiang in #17213
• docs: add OpenAI Agents SDK to projects by @Chesars in #17203
• Upgrade websockets to v15 by @hxyannay in #16734
• fix: do not include plaintext message in exception by @raghav-stripe in #17216
• [Fix] Change Add Fallback Modal to use Antd Select by @yuneng-jiang in #17223
• [Feat] Add guardrails for pass through endpoints by @ishaan-jaff in #17221
• [Feat] UI - allow adding pass through guardrails through UI by @ishaan-jaff in #17226
• [Bug fix] Vector Store List Endpoint Returns 404 by @ishaan-jaff in #17229
• [Feature]: Add Provider publicai.co by @ishaan-jaff in #17230
• [Fix] Request and Response Panel JSONViewer by @yuneng-jiang in #17233
• [Feature] Adding Button Loading States to Edit Settings by @yuneng-jiang in #17236
• [Fix] Various Text, button state, and test changes by @yuneng-jiang in #17237
• [Fix] UI - Fallbacks Immediately Deleting before API resolves by @yuneng-jiang in #17238
• [Fix] Remove Feature Flags by @yuneng-jiang in #17240
• (Bug)Migrate Anthropic provider to azure ai by @Sameerlite in #17202
• Fix/issue 16759 streaming error validation by @weichiet in #17242
• feat: add regex-based tool_name/tool_type matching for tool-permission by @uc4w6c in #17164
• SSO: fix the generic SSO provider by @saar-win in #17227
• fix(bedrock): handle cohere v4 embed response dictionary format by @AndyForest in #17220
• Fix : acompletion throws error with SambaNova models by @omkar806 in #17217
• Allow wildcard routes for nonproxy admin (SCIM) by @v0rtex20k in #17178
• Update new anthropic feats as reviewed by @Sameerlite in #17142
• Fix metadata tags and model name display in UI for Azure passthrough + Add cost tracking for responses API by @Sameerlite in #17258
• Add nova embedding support by @Sameerlite in #17253
• Better handle anonymization by @hxomer in #17207
• Added support for twelvelabs pegasus by @Sameerlite in #17193
• [Refactor] Deprecate spend/logs & add spend/logs/v2 by @AlexsanderHamir in #17167
• [Perf] Fix /audio/speech performance by using shared_sessions by @AlexsanderHamir in #16739
• Respect custom llm provider in header by @Sameerlite in #17290
• (Feat)Add passthrough cost tracking for veo by @Sameerlite in #17296
• Remove not compatible beta header from Bedrock by @Sameerlite in #17301
• Add better handling in image generation for gemini models by @Sameerlite in #17292
• fix: respect guardrail mock_response during during_call to return blo… by @uc4w6c in #17247
• Fix session consistency, move Lasso API version away from source code by @orgersh92 in #17316
• doc: add images for tool permission guardrail by @uc4w6c in #17322
• [Feat] WatsonX - allow passing zen_api_key dynamically by @ishaan-jaff in #16655
• [Feat] JWT Auth - AI Gateway, allow using regular OIDC flow with user info endpoints by @ishaan-jaff in #17324
• [Feat] Add new modelfireworks_ai /kimi-k2-instruct-0905 by @ishaan-jaff in #17328
• (feat) Generic Guardrail API - allows guardrail providers to add INSTANT support for LiteLLM w/out PR to repo by @krrishdholakia in #17175
• Revert "fix: respect guardrail mock_response during during_call to return blo…" by @krrishdholakia in #17332
• Add claude-opus-4-5 alias to pricing data by @dannykopping in #17313
• Add support of audio transcription for OVHcloud by @eliasto in #17305
• [Fix] Fix Watsonx Audio Transcription API by @ishaan-jaff in #17326
• [fix] extra_headers in messages api bedrock invoke by @colinlin-stripe in #17271
• fix: GA path for azure openai realtime models by @abi-jey in #17260
• [Bug Fix] - Fix litellm_enterprise ensure imported routes exist by @ishaan-jaff in #17337
• Fix/new org team validate against org by @rioiart in #17333
• Fix sso users not added to entra synced team by @rioiart in #17331
• Refactor Noma guardrail to use shared Responses transformation and include system instructions by @idola9 in #17315
• feat(provider): add Z.AI (Zhipu AI) as built-in provider by @Chesars in #17307
• Fix AttributeError when metadata is null in request body by @Chesars in #17306
• SSO(fix): Clear SSO integration for all users by @saar-win in #17287
• feat: add experimental latest-user filtering for Bedrock by @uc4w6c in #17282
• Add context window exception mapping for Together AI by @li-boxuan in #17284
• feat(github-copilot): Add Embedding API support by @codgician in #17278
• Update Databricks model pricing and add new models (including databricks pricing test). by @epistoteles in #17277
• fix: remove URL format validation for MCP server endpoints by @uc4w6c in #17270
• Guardrail API V2 - user api key metadata, session id, specify input type (request/response), image support by @krrishdholakia in #17338
• fix: update default database connection number by @AlexsanderHamir in #17353
• Fix: update default proxy_batch_write_at number by @AlexsanderHamir in #17355
• [Feat] Add vllm batch+files API support by @ishaan-jaff in #15823
• Fix 500 error for malformed request by @Sameerlite in #17291
• Fix: litellm user auth not passing issue by @Sameerlite in #17342
• Add other routes in jwt auth by @Sameerlite in #17345
• docs: add Azure AI Foundry documentation for Claude models by @Chesars in #17104
• Litellm bedrock OpenAI model support by @kothamah in #17368
• [Fix] Bedrock Guardrail Indent and Import fix by @yuneng-jiang in #17378
• [Infra] Add Google Private API Endpoint to Vertex AI fields by @yuneng-jiang in #17382
• [Docs] Add guide on how to debug gateway error vs provider error by @ishaan-jaff in #17387
• 🐛 Fix proxy caching between requests in aiohttp transport by @lcfyi in #17122
• [Feat] New model/provider - Adds support for Google Cloud Chirp3 HD on /speech by @ishaan-jaff in #17391
• [Fix] UI - Change Edit Team Models Shown to Match Create Team by @yuneng-jiang in #17394
• [New model] Add fireworks/deepseek-v3p2 by @ishaan-jaff in #17395
• feat: Add Nova lite 2 reasoning support with reasoningConfig by @flozonn in #17371
• [Fix] Show all credential values on Edit Credential Modal by @yuneng-jiang in #17397
• [Refactor] Standardize API Key vs Virtual Key in UI by @yuneng-jiang in #17325
• [Feature] Add User Alias Column to Internal User Table by @yuneng-jiang in #17321
• Fix: Datadog callback regression when ddtrace is installed by @ishaan-jaff in #17393
• Refactor add_schema_to_components to move definitions to components/s… by @richardmcsong in #17389
• Support Deepseek 3.2 with Reasoning by @matt-greathouse in #17384
• Add model price & details for Global Opus 4.5 Bedrock endpoint by @mossbanay in #17380
• docs: Update Instructions For Phoenix Integration by @ialisaleh in #17373
• update model_prices_and_context_window.json by @...

v1.80.7.dev.1

What's Changed

• [Infra] Building UI by @yuneng-jiang in #17212
• [Infra] Rebuilding UI by @yuneng-jiang in #17213
• docs: add OpenAI Agents SDK to projects by @Chesars in #17203
• Upgrade websockets to v15 by @hxyannay in #16734
• fix: do not include plaintext message in exception by @raghav-stripe in #17216
• [Fix] Change Add Fallback Modal to use Antd Select by @yuneng-jiang in #17223
• [Feat] Add guardrails for pass through endpoints by @ishaan-jaff in #17221
• [Feat] UI - allow adding pass through guardrails through UI by @ishaan-jaff in #17226
• [Bug fix] Vector Store List Endpoint Returns 404 by @ishaan-jaff in #17229
• [Feature]: Add Provider publicai.co by @ishaan-jaff in #17230
• [Fix] Request and Response Panel JSONViewer by @yuneng-jiang in #17233
• [Feature] Adding Button Loading States to Edit Settings by @yuneng-jiang in #17236
• [Fix] Various Text, button state, and test changes by @yuneng-jiang in #17237
• [Fix] UI - Fallbacks Immediately Deleting before API resolves by @yuneng-jiang in #17238
• [Fix] Remove Feature Flags by @yuneng-jiang in #17240
• (Bug)Migrate Anthropic provider to azure ai by @Sameerlite in #17202
• Fix/issue 16759 streaming error validation by @weichiet in #17242
• feat: add regex-based tool_name/tool_type matching for tool-permission by @uc4w6c in #17164
• SSO: fix the generic SSO provider by @saar-win in #17227
• fix(bedrock): handle cohere v4 embed response dictionary format by @AndyForest in #17220
• Fix : acompletion throws error with SambaNova models by @omkar806 in #17217
• Allow wildcard routes for nonproxy admin (SCIM) by @v0rtex20k in #17178
• Update new anthropic feats as reviewed by @Sameerlite in #17142
• Fix metadata tags and model name display in UI for Azure passthrough + Add cost tracking for responses API by @Sameerlite in #17258
• Add nova embedding support by @Sameerlite in #17253
• Better handle anonymization by @hxomer in #17207
• Added support for twelvelabs pegasus by @Sameerlite in #17193
• [Refactor] Deprecate spend/logs & add spend/logs/v2 by @AlexsanderHamir in #17167
• [Perf] Fix /audio/speech performance by using shared_sessions by @AlexsanderHamir in #16739
• Respect custom llm provider in header by @Sameerlite in #17290
• (Feat)Add passthrough cost tracking for veo by @Sameerlite in #17296
• Remove not compatible beta header from Bedrock by @Sameerlite in #17301
• Add better handling in image generation for gemini models by @Sameerlite in #17292
• fix: respect guardrail mock_response during during_call to return blo… by @uc4w6c in #17247
• Fix session consistency, move Lasso API version away from source code by @orgersh92 in #17316
• doc: add images for tool permission guardrail by @uc4w6c in #17322
• [Feat] WatsonX - allow passing zen_api_key dynamically by @ishaan-jaff in #16655
• [Feat] JWT Auth - AI Gateway, allow using regular OIDC flow with user info endpoints by @ishaan-jaff in #17324
• [Feat] Add new modelfireworks_ai /kimi-k2-instruct-0905 by @ishaan-jaff in #17328
• (feat) Generic Guardrail API - allows guardrail providers to add INSTANT support for LiteLLM w/out PR to repo by @krrishdholakia in #17175
• Revert "fix: respect guardrail mock_response during during_call to return blo…" by @krrishdholakia in #17332
• Add claude-opus-4-5 alias to pricing data by @dannykopping in #17313
• Add support of audio transcription for OVHcloud by @eliasto in #17305
• [Fix] Fix Watsonx Audio Transcription API by @ishaan-jaff in #17326
• [fix] extra_headers in messages api bedrock invoke by @colinlin-stripe in #17271
• fix: GA path for azure openai realtime models by @abi-jey in #17260
• [Bug Fix] - Fix litellm_enterprise ensure imported routes exist by @ishaan-jaff in #17337
• Fix/new org team validate against org by @rioiart in #17333
• Fix sso users not added to entra synced team by @rioiart in #17331
• Refactor Noma guardrail to use shared Responses transformation and include system instructions by @idola9 in #17315
• feat(provider): add Z.AI (Zhipu AI) as built-in provider by @Chesars in #17307
• Fix AttributeError when metadata is null in request body by @Chesars in #17306
• SSO(fix): Clear SSO integration for all users by @saar-win in #17287
• feat: add experimental latest-user filtering for Bedrock by @uc4w6c in #17282
• Add context window exception mapping for Together AI by @li-boxuan in #17284
• feat(github-copilot): Add Embedding API support by @codgician in #17278
• Update Databricks model pricing and add new models (including databricks pricing test). by @epistoteles in #17277
• fix: remove URL format validation for MCP server endpoints by @uc4w6c in #17270
• Guardrail API V2 - user api key metadata, session id, specify input type (request/response), image support by @krrishdholakia in #17338
• fix: update default database connection number by @AlexsanderHamir in #17353
• Fix: update default proxy_batch_write_at number by @AlexsanderHamir in #17355
• [Feat] Add vllm batch+files API support by @ishaan-jaff in #15823
• Fix 500 error for malformed request by @Sameerlite in #17291
• Fix: litellm user auth not passing issue by @Sameerlite in #17342
• Add other routes in jwt auth by @Sameerlite in #17345
• docs: add Azure AI Foundry documentation for Claude models by @Chesars in #17104
• Litellm bedrock OpenAI model support by @kothamah in #17368
• [Fix] Bedrock Guardrail Indent and Import fix by @yuneng-jiang in #17378
• [Infra] Add Google Private API Endpoint to Vertex AI fields by @yuneng-jiang in #17382
• [Docs] Add guide on how to debug gateway error vs provider error by @ishaan-jaff in #17387
• 🐛 Fix proxy caching between requests in aiohttp transport by @lcfyi in #17122
• [Feat] New model/provider - Adds support for Google Cloud Chirp3 HD on /speech by @ishaan-jaff in #17391

New Contributors

• @hxyannay made their first contribution in #16734
• @weichiet made their first contribution in #17242
• @AndyForest made their first contribution in #17220
• @omkar806 made their first contribution in #17217
• @v0rtex20k made their first contribution in #17178
• @hxomer made their first contribution in #17207
• @orgersh92 made their first contribution in #17316
• @dannykopping made their first contribution in #17313
• @codgician made their first contribution in #17278
• @epistoteles made their first contribution in #17277
• @kothamah made their first contribution in #17368

Full Changelog: v1.80.6-nightly...v1.80.7.dev.1