create_agent support fallback

[tornadotang]

Checked other resources

• This is a feature request, not a bug report or usage question.
• I added a clear and descriptive title that summarizes the feature request.
• I used the GitHub search to find a similar feature request and didn't find it.
• I checked the LangChain documentation and API reference to see if this feature already exists.
• This is not related to the langchain-community package.

Feature Description

To better support structured output, create_agent no longer supports pre-bound models with tools or configuration:

BUT w/o pre-bound models, we cannot pass model.with_fallback to create_agent, that means we lost the capability of failover when openai outage

Use Case

I want to build agent support failover when openai outage, w/ create_react_agent it can support pre-bound model.with_fallback().

but langchain.agents create_agent disabled it.

Proposed Solution

I request create_agent support pre-bound model.with_fallback()

Alternatives Considered

No response

Additional Context

No response

[keenborder786]

from langchain.agents.middleware_agent import create_agent
from langchain_openai import ChatOpenAI
model = ChatOpenAI(model="gpt-5").with_fallbacks([ChatOpenAI(model="gpt-4o")])
agent = create_agent(model=model, tools=[], system_prompt="You are a helpful assistant").compile()
print(agent.invoke({"messages": [{"role": "user", "content": "What is 3^3?"}]}))

You can still pass the fallback

[aaron-seq]

Great point @keenborder786! You're absolutely right that create_agent does indeed support fallback models.

To further clarify this for @tornadotang and others who might encounter this issue:

✅ create_agent DOES Support Fallbacks

The create_agent function accepts both string model names and BaseChatModel instances. This means you can pass a pre-configured model with fallbacks:

from langchain.agents.middleware_agent import create_agent
from langchain_openai import ChatOpenAI

# Create model with fallback chain
model_with_fallback = ChatOpenAI(model="gpt-4o").with_fallbacks([
    ChatOpenAI(model="gpt-4o-mini"),
    ChatOpenAI(model="gpt-3.5-turbo")
])

# This works perfectly fine
agent = create_agent(
    model=model_with_fallback,
    tools=[],
    system_prompt="You are a helpful assistant with failover capability"
).compile()

# Test the agent
response = agent.invoke({"messages": [{"role": "user", "content": "Hello!"}]})
print(response)

📚 Complete Example with Error Handling

import asyncio
from langchain.agents.middleware_agent import create_agent
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool

@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return f"The weather in {location} is sunny and 75°F"

def create_resilient_agent():
    # Set up fallback chain: GPT-4 -> GPT-4-mini -> GPT-3.5
    primary_model = ChatOpenAI(model="gpt-4o", temperature=0)
    fallback_model_1 = ChatOpenAI(model="gpt-4o-mini", temperature=0)
    fallback_model_2 = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)
    
    resilient_model = primary_model.with_fallbacks([
        fallback_model_1,
        fallback_model_2
    ])
    
    return create_agent(
        model=resilient_model,
        tools=[get_weather],
        system_prompt="You are a helpful assistant with automatic failover capabilities."
    ).compile()

# Usage
agent = create_resilient_agent()
result = agent.invoke({
    "messages": [{"role": "user", "content": "What's the weather like in San Francisco?"}]
})
print(result)

🔄 How Fallbacks Work in create_agent

1. Model Initialization: The function checks if the model is a string or BaseChatModel instance
2. Tool Binding: The model (with its fallbacks) gets bound to tools
3. Execution: If the primary model fails, LangChain automatically tries the fallback models
4. Transparency: The agent works normally, with fallbacks handled at the model level

So the capability you're looking for is already there! The key is to set up your fallback chain before passing it to create_agent, not within the agent creation itself.

Hope this helps clarify the functionality! 🚀