v1.70.1-stable · BerriAI litellm · Discussion #10916 · GitHub

krrishdholakia
May 17, 2025
Maintainer

What's Changed

Handle gemini audio input by @krrishdholakia in Handle gemini audio input #10739
Fixed Ollama Structured Response not working [Bug]: Structured Output not working with Ollama #10616 by @imdigitalashish in Fixed Ollama Structured Response not working #10616 #10617
fix(factory.py): Add reasoning content handling for missing assistant… by @LouisShark in fix(factory.py): Add reasoning content handling for missing assistant… #10688
[Feat] Add tools support for Nvidia NIM by @ishaan-jaff in [Feat] Add tools support for Nvidia NIM #10763
[Fix]: /messages - allow using dynamic AWS params by @ishaan-jaff in [Fix]: /messages - allow using dynamic AWS params #10769
fix: pass application/json for GenericAPILogger by @ishaan-jaff in fix: pass application/json for GenericAPILogger #10772
[Docs] Using litellm with Google ADK by @ishaan-jaff in [Docs] Using litellm with Google ADK #10777
Update Nscale model providers to point to website by @OscarSavNS in Update Nscale model providers to point to website #10764
[Fix] Allow using dynamic aws_region with /messages on Bedrock by @ishaan-jaff in [Fix] Allow using dynamic aws_region with /messages on Bedrock #10779
[Feat] Option to force/always use the litellm proxy ([Feature]: Flag to always send queries via the lite_proxy #10559) ([Feat] Option to force/always use the litellm proxy (#10559) #10633) by @ishaan-jaff in [Feat] Option to force/always use the litellm proxy (#10559) (#10633) #10773
feat: Addded EU Anthropic Inference profile for Claude 3.7 by @wagnerjt in feat: Addded EU Anthropic Inference profile for Claude 3.7 #10767
Add new model provider Novita AI (Add new model provider Novita AI #7582) by @krrishdholakia in Add new model provider Novita AI (#7582) #9527
Support Anthropic web search tool + Add more google finish reason mapping by @krrishdholakia in Support Anthropic web search tool + Add more google finish reason mapping #10785
Fix azure dall e 3 call with custom model name + Handle Bearer $LITELLM_API_KEY in x-litellm-api-key custom header by @krrishdholakia in Fix azure dall e 3 call with custom model name + Handle Bearer $LITELLM_API_KEY in x-litellm-api-key custom header #10776
[Refactor] Move LLM Guard, Secret Detection to Enterprise Pip packagea by @ishaan-jaff in [Refactor] Move LLM Guard, Secret Detection to Enterprise Pip packagea #10782
fix: URL encode OTEL_EXPORTER_OTLP_TRACES_HEADERS for Phoenix Integration by @arizedatngo in fix: URL encode OTEL_EXPORTER_OTLP_TRACES_HEADERS for Phoenix Integration #10654
[README] Add Llama API in supported providers table by @seyeong-han in [README] Add Llama API in supported providers table #10802
[Fix] Maximum Vertex Recursion Depth Reached when transforming response_format by @ishaan-jaff in [Fix] Maximum Vertex Recursion Depth Reached when transforming response_format #10798
add ollama api base defaults by @jugaldb in add ollama api base defaults #10805
Add Azure gpt-4o-mini-tts by @emerzon in Add Azure gpt-4o-mini-tts #10807
Add Azure Cohere Embed v4 by @emerzon in Add Azure Cohere Embed v4 #10806
[Fix] Allow overriding all constants using a .env variable by @ishaan-jaff in [Fix] Allow overriding all constants using a .env variable #10803
[Bug Fix] Presidio integration failing and making inference return 500 by @ishaan-jaff in [Bug Fix] Presidio integration failing and making inference return 500 #10197
[Feat] Allow specifying PII Entities Config when using Presidio Guardrails by @ishaan-jaff in [Feat] Allow specifying PII Entities Config when using Presidio Guardrails #10810
[Bedrock] Fixes to only allow accepted fields for tool json schema by @ishaan-jaff in [Bedrock] Fixes to only allow accepted fields for tool json schema #10062
Fix: Separate Sync/Async Azure Client Cache to Prevent Type Mismatches by @daikeren in Fix: Separate Sync/Async Azure Client Cache to Prevent Type Mismatches #10781
fix(litellm_logging.py): log custom headers in requester metadata by @krrishdholakia in fix(litellm_logging.py): log custom headers in requester metadata #10818
Migrate cohere to /v2/embed - adds support for output_dimensions param by @krrishdholakia in Migrate cohere to /v2/embed - adds support for output_dimensions param #10809
add support to parse metadata by @jugaldb in add support to parse metadata #10832
Team id + Status filter on logs by @NANDINI-star in Team id + Status filter on logs #10831
Update Groq models on model_prices_and_context_window.json by @naliotopier in Update Groq models on model_prices_and_context_window.json #10077
[Feat] Add endpoints for adding, deleting, editing guardrails in DB by @ishaan-jaff in [Feat] Add endpoints for adding, deleting, editing guardrails in DB #10833
[Feat UI] Add Guardrails on LiteLLM UI by @ishaan-jaff in [Feat UI] Add Guardrails on LiteLLM UI #10834
[UI] Fixes for bulk invite users - add validation for bad inputs by @ishaan-jaff in [UI] Fixes for bulk invite users - add validation for bad inputs #10836
Support OTLP standard environment variables for configuration by @damienpontifex in Support OTLP standard environment variables for configuration #10813
[Feat] Presidio Improvements - Allow adding presidio api base on UI, Test presidio on Test Key, fixes for running presidio hook by @ishaan-jaff in [Feat] Presidio Improvements - Allow adding presidio api base on UI, Test presidio on Test Key, fixes for running presidio hook #10840
patch: validate_team_org_change by @ishaan-jaff in patch: validate_team_org_change #10835
Feat add retention config by @jugaldb in Feat add retention config #10815
fix failing test typo by @jugaldb in fix failing test typo #10843
Capture Gemini reasoning tokens usage in streaming mode by @Dima-Mediator in Capture Gemini reasoning tokens usage in streaming mode #10789
Anthropic unified web search + tool cost tracking support by @krrishdholakia in Anthropic unified web search + tool cost tracking support #10846
avoid error AttributeError: 'Message' object has no attribute 'audio' by @igtm in avoid error AttributeError: 'Message' object has no attribute 'audio' #10814
fix: bedrock-sonnet-caching-cost by @shibaboy in fix: bedrock-sonnet-caching-cost #10752
Fix/issue 10113 embeddings use non default tokenizer by @camfarineau in Fix/issue 10113 embeddings use non default tokenizer #10629
Add models import command by @msabramo in Add models import command #10581
Add documentation on in-memory and disk caching by @user202729 in Add documentation on in-memory and disk caching #10522
Bedrock pixtral fix by @ajac-zero in Bedrock pixtral fix #10439
added fix to resolve delete API by @jugaldb in added fix to resolve delete API #10845
build(deps): bump undici from 6.21.1 to 6.21.3 in /docs/my-website by @dependabot in build(deps): bump undici from 6.21.1 to 6.21.3 in /docs/my-website #10855
[Refactor] Make Pagerduty a free feature by @ishaan-jaff in [Refactor] Make Pagerduty a free feature #10857
[Refactor] Move enterprise_routes within litellm_enterprise by @ishaan-jaff in [Refactor] Move enterprise_routes within litellm_enterprise #10860
refactor: make send_management_endpoint_alert a free feature by @ishaan-jaff in refactor: make send_management_endpoint_alert a free feature #10863
[Feat - Guardrails] Expose /apply_guardrail endpoint for directly calling guardrail by @ishaan-jaff in [Feat - Guardrails] Expose /apply_guardrail endpoint for directly calling guardrail #10867
[CI/CD] Test: fix mapped tests by @ishaan-jaff in [CI/CD] Test: fix mapped tests #10870
[Feat] UI: Add support for Deleting Guardrail on UI by @ishaan-jaff in [Feat] UI: Add support for Deleting Guardrail on UI #10869
[UI] Allow adding Bedrock, Presidio, Lakera, AIM guardrails on UI by @ishaan-jaff in [UI] Allow adding Bedrock, Presidio, Lakera, AIM guardrails on UI #10874
fix(proxy/_types.py): add missing comma for /v2/rerank by @krrishdholakia in fix(proxy/_types.py): add missing comma for /v2/rerank #10861
fix(streaming_handler.py): patch vertex ai meta llama streaming error by @krrishdholakia in fix(streaming_handler.py): patch vertex ai meta llama streaming error #10878
build(dependencies)📦: Add numpydoc to dependencies in pyproject.toml by @ericmjl in build(dependencies)📦: Add numpydoc to dependencies in pyproject.toml #10736
[Feat]: Configure LiteLLM to Parse User Headers from Open Web UI by @damgem in [Feat]: Configure LiteLLM to Parse User Headers from Open Web UI #9802
Feat/support anonymize in aim guardrail by @hxdror in Feat/support anonymize in aim guardrail #10757
Gemini Multimodal Live API support by @krrishdholakia in Gemini Multimodal Live API support #10841
Feat log retention cleanup by @jugaldb in Feat log retention cleanup #10872
Model filter added on logs tab. (Model filter added on logs tab. #10877) by @ishaan-jaff in Model filter added on logs tab. (#10877) #10891
fix: fix novita ai docs by @jasonhp in fix: fix novita ai docs #10882
[Feat] Add Tracing for guardrails in StandardLoggingPayload, Langfuse by @ishaan-jaff in [Feat] Add Tracing for guardrails in StandardLoggingPayload, Langfuse #10890
[UI] - Add Guardrail Tracing to LiteLLM SpendLogs by @ishaan-jaff in [UI] - Add Guardrail Tracing to LiteLLM SpendLogs #10893
Fix example commands in reliability.md by @wwwillchen in Fix example commands in reliability.md #10894
Feat: add guardrail tracing to OTEL, Arize phoenix by @ishaan-jaff in Feat: add guardrail tracing to OTEL, Arize phoenix #10896
Key hash filter added by @NANDINI-star in Key hash filter added #10903
[Feat] UI: Allow clicking into a guardrail by @ishaan-jaff in [Feat] UI: Allow clicking into a guardrail #10904
[Feat] Add Lakera v2 Guardrail Support by @ishaan-jaff in [Feat] Add Lakera v2 Guardrail Support #10880
Key alias filter on logs by @NANDINI-star in Key alias filter on logs #10905
[Feat] handle multi-pod deployment for SpendLogs Retention by @jugaldb in [Feat] handle multi-pod deployment for SpendLogs Retention #10895
[UI] Fixes for using Lakera Guardrail on LiteLLM UI by @ishaan-jaff in [UI] Fixes for using Lakera Guardrail on LiteLLM UI #10906
add constant to config settings.md by @jugaldb in add constant to config settings.md #10908
Remove extra test file by @jugaldb in Remove extra test file #10889
[UI] Allow editing guardrails by @ishaan-jaff in [UI] Allow editing guardrails #10907
Handle openai gpt file data + add openai 'supports_pdf_input' to all vision models + Support bedrock tool cache pointing by @krrishdholakia in Handle openai gpt file data + add openai 'supports_pdf_input' to all vision models + Support bedrock tool cache pointing #10897

New Contributors

@imdigitalashish made their first contribution in Fixed Ollama Structured Response not working #10616 #10617
@LouisShark made their first contribution in fix(factory.py): Add reasoning content handling for missing assistant… #10688
@OscarSavNS made their first contribution in Update Nscale model providers to point to website #10764
@arizedatngo made their first contribution in fix: URL encode OTEL_EXPORTER_OTLP_TRACES_HEADERS for Phoenix Integration #10654
@jugaldb made their first contribution in add ollama api base defaults #10805
@daikeren made their first contribution in Fix: Separate Sync/Async Azure Client Cache to Prevent Type Mismatches #10781
@naliotopier made their first contribution in Update Groq models on model_prices_and_context_window.json #10077
@damienpontifex made their first contribution in Support OTLP standard environment variables for configuration #10813
@Dima-Mediator made their first contribution in Capture Gemini reasoning tokens usage in streaming mode #10789
@igtm made their first contribution in avoid error AttributeError: 'Message' object has no attribute 'audio' #10814
@shibaboy made their first contribution in fix: bedrock-sonnet-caching-cost #10752
@camfarineau made their first contribution in Fix/issue 10113 embeddings use non default tokenizer #10629
@ajac-zero made their first contribution in Bedrock pixtral fix #10439
@damgem made their first contribution in [Feat]: Configure LiteLLM to Parse User Headers from Open Web UI #9802
@hxdror made their first contribution in Feat/support anonymize in aim guardrail #10757
@wwwillchen made their first contribution in Fix example commands in reliability.md #10894

Full Changelog: v1.69.0-stable...v1.70.1-stable

This discussion was created from the release v1.70.1-stable.

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment