[Draft] April 2025 Roadmap

[krrishdholakia]

Putting together a list of items for April 2025

Key Focus:

1. LLM Translation
2. Cost Tracking

LLM Translation

OpenAI

• [Bug]: Audio Transcriptions Hang #9961

Bedrock

• [Bug]: eu.anthropic.claude-3-7-sonnet-20250219-v1:0 is not mapped #9974
• [Bug]: Gemini tool call "default" errors #9793

Anthropic

• [Bug]: Prompt caching documentation is incomplete #9969
• [Feature]: Support injecting cache control flags on anthropic requests #9949
• [Bug]: anthropic-beta header not being forwarded on litellm-proxy #9016 (comment)
• [Feature]: Anthropic native Responses API Support #10160

Vertex AI

• [Bug]: drop_params not working for (image only?) vertex_ai generation #9936
• [Bug]: Inconsistent VertexAI calls due to intermittent auth issues #8771
• [Bug]: supports_vision function returns false for gemini/gemini-2.5-pro-exp-03-25 #9842
• [Feature]: Custom TTL for Gemini Context Caching #9810
• [Bug]: gemini 2.0 Flash GA is erroring due to context caching not being supported #8296

Others

• [Bug]: IBM Watsonx - invalid tool choice parameter name #9979
• [Bug]: sagemaker_chat provider does not correctly pass the model_id for the SageMaker Inference Component #9909
• [Feature]: VoyageAI reranking #9823
• [Bug]: TogetherAi tool call support issues #9797

Logging / Guardrail Improvements

• [Feature]: Team-based logging not respected when using passthrough endpoints (e.g., Google AI Studio) #9967
• [Bug]: Bedrock guardrails analysing (and blocking) old+system messages #9882
• [Feature]: Support passing 'environment' variable to langfuse on trace/generations per key #10020
• [Bug]: connects to langfuse even it is not configured #7732
• Langfuse Prompt Management - Ability to send label #9003 (comment)

Reliability / Loadbalancing / Performance

• [Bug]: caching does not account for thinking or reasoning_effort config #9844
• [Bug]: Failure in fallback to gpt-4o for gemini requests with image url #9816

General Proxy Improvements

• [Bug]: Cannot disable cost calculation verbose with custom llms & local cost map #9815