A2A Roadmap

[krrishdholakia]

Starting a discussion for how LiteLLM can support A2A.

How would you expect A2A to work with LiteLLM?

Common Asks:

• Allow adding agents onto LiteLLM
• Allow discovering agents
• Allow discovering agents by skills and capabilities
• Allow executing agents on LiteLLM (?)
• team/key permission management for agents

Relevant docs: https://google-a2a.github.io/A2A/latest/specification

[wagnerjt]

How would you expect A2A to work with LiteLLM?

LiteLLM's proxy would operate purely as a registry/catalog and the SDK would include an A2A client.

The proxy would include the same core functionality the MCP servers was implemented with:

1. CRUD operations to add, edit, and remove A2A servers
   Note: Only need to store the friendly name, description, and server uri
2. Team/Key permission controlled management

Reasoning: It is the A2A clients responsibility (external ones and litellm Sdk) to identify the associated agent it needs to interact with via the agent card which is required per A2A implementation server. This means Litellm proxy only needs to care about this small info and allow clients to discover the agents from some a2a endpoint. Ref agent card struct

From other discovery options, the UI can be a bit friendlier and include the catalog view that shows:

1. The A2A servers associated on the proxy
2. All of the agent cards for each proxy
3. Ways to filter or group by agent capability (fetched from the agent card)
4. Some way to make a call from the proxy to an A2A server
5. Allow two views for users, one strict to the A2A servers for that user and a "public" registry view to be visible to all proxy users (similar to the public model catalog).

Happy to help in this area as well!

[krrishdholakia]

Hey @wagnerjt so the goal would be to onboard this AgentCard onto the proxy, and make it discoverable?

# This will be the public-facing agent card
public_agent_card = AgentCard(
    name='Hello World Agent',
    description='Just a hello world agent',
    url='http://0.0.0.0:9999/',
    version='1.0.0',
    defaultInputModes=['text'],
    defaultOutputModes=['text'],
    capabilities=AgentCapabilities(streaming=True),
    skills=[skill],  # Only the basic skill for the public card
    supportsAuthenticatedExtendedCard=True,
)

[krrishdholakia]

Would you expect the agent url to be exposed or would this call be through litellm?

[wagnerjt]

I don't think litellm even needs to store the agent card, but rather where the agent would only store what is referred to the Service Endpoint: The url where the A2A service can be reached. Once clients know the various locations, the A2A client can look into the agent cards as needed

Refer to how agent discovery works and how litellm proxy could become the catalog in this discovery flow.

My understanding is the underlying A2A framework should cover identifying the agent to use and litellm plays the registry/agent discovery aspects only

[alecswan]

I agree with @wagnerjt proposal.

It is expected that the content of the agent card may change over time. So, litellm would persistently store the url of the agent card and load additional properties from the agent card at runtime in order to faciliate filtering of agents in the UI / API.
Invoking the agent from the litellm UI / API is not required for the first release. It would be a nice-to-have for later releases.

Here are some ideas for the API endpoints which somewhat mirrors the MCP API endpoints in litellm:

1. POST /v1/a2a/agent - accepts the url of the agent card and returns the uuid (could be just a hash of the url) generated for that agent as well as the content of the agent card (to ensure that litellm has permissions to read the card); the API key who initiated the request should be granted permissions to access and modify this agent's record.
2. PUT /v1/a2a/agent/{agent_uuid} - allows updating agent's url as well as permissions of API keys and/or teams who are allowed to read/update this agent record.
3. GET /v1/a2a/agent - iterates over agent records (uuid and url) stored in litellm database to which the invoking API key / team has been granted access; fetches metadata, such as name, description, capabilities, skills, etc. by loading the agent card from the url when the API is invoked; this API will be used from litellm UI to load agent properties needed for filtering / searching agents in the UI. This API call would be made when user visits the a2a tab in litellm UI. Refreshing the UI would make another call to this endpoint to load agent properties and cache them in the session to enable snappy filtering / search UI experience.
4. GET /v1/a2a/agent/{agent_uuid} - if the API key / team has permissions to access this agent's record, returns the entire content of the agent's card

[wagnerjt]

Recently a2a got donated to the linux foundation and there is also a discussion around the agent registry spec

[ishaan-jaff]

"I don't want to store keys on with my agents. We can't centralize agents until LiteLLM supports it. LiteLLM can offer a central gateway for everything AI by supporting A2A, MCP, LLMs"

[krrishdholakia]

A2A Protocol Support - v0 scope

1. Register agent card on LiteLLM
2. Use Agents -> allow them to call agents (ideally to same base url)
3. Agents can see MCPs on LiteLLM

---

1. Register agent card on LiteLLM

1. Add agent via config.yaml

agent_list:
  - agent_name: my_custom_agent
    agent_card_params:
      name: 'Hello World Agent',
      description: Just a hello world agent
      url: http://localhost:9999/
      version: 1.0.0
      default_input_modes: ['text']
      default_output_modes: ['text']
      capabilities:
        streaming: true
      skills:
        - skill:
            name: Hello World Skill
            description: Just a hello world skill
            url: http://localhost:9999/
            version: 1.0.0
            input_modes: ['text']
            output_modes: ['text']
      supports_authenticated_extended_card: True

1. Create key with agent access

curl -L -X POST 'http://0.0.0.0:4000/key/generate' \
-H 'Authorization: Bearer sk-1234' \
-H 'Content-Type: application/json' \
-d '{
    "agents": ["my_custom_agent"] 
}'

1. Get available agents

User can call /v1/agents to retrieve all agents they have access to

curl http://0.0.0.0:4000/v1/agents \
  -H "Authorization: Bearer $LITELLM_API_KEY"

{
  "object": "list",
  "data": [
    {
      "id": "agent-id-1",
      "object": "agent",
      "created": 1686935002,
      "owned_by": "organization-owner",
      "metadata": {} # includes non-sensitive agent card params - don't reveal underlying api base
    },
    {
      "id": "agent-id-1",
      "object": "agent",
      "created": 1686935002,
      "owned_by": "organization-owner",
    },
    {
      "id": "agent-id-2",
      "object": "agent",
      "created": 1686935002,
      "owned_by": "openai"
    },
  ],
  "object": "list"
}

2. Call Specific Agent

1. List available agents

User can call /v1/agents to retrieve all agents they have access to

curl http://0.0.0.0:4000/v1/agents \
  -H "Authorization: Bearer $LITELLM_API_KEY"

1. Retrieve agent card

base_url = 'http://0.0.0.0:4000/agents/{agent_id}'

resolver = A2ACardResolver(
    httpx_client=httpx_client,
    base_url=base_url,
    # agent_card_path uses default, extended_agent_card_path also uses default
)

# Fetch Public Agent Card and Initialize Client
final_agent_card_to_use: AgentCard | None = None

_public_card = (
    await resolver.get_agent_card()
)  # Fetches from default public path - `/agents/{agent_id}/`

final_agent_card_to_use = _public_card

if _public_card.supports_authenticated_extended_card:
    try:
        auth_headers_dict = {
            'Authorization': 'Bearer dummy-token-for-extended-card'
        }
        _extended_card = await resolver.get_agent_card(
            relative_card_path=EXTENDED_AGENT_CARD_PATH,
            http_kwargs={'headers': auth_headers_dict},
        )
        final_agent_card_to_use = (
            _extended_card  # Update to use the extended card
        )
    except Exception as e_extended:
        logger.warning(
            f'Failed to fetch extended agent card: {e_extended}. Will proceed with public card.',
            exc_info=True,
        )

1. Call agent

client = A2AClient(
    httpx_client=httpx_client, agent_card=final_agent_card_to_use
)

send_message_payload: dict[str, Any] = {
    'message': {
        'role': 'user',
        'parts': [
            {'kind': 'text', 'text': 'how much is 10 USD in INR?'}
        ],
        'messageId': uuid4().hex,
    },
}
request = SendMessageRequest(
    id=str(uuid4()), params=MessageSendParams(**send_message_payload)
)

response = await client.send_message(request)
print(response.model_dump(mode='json', exclude_none=True))

3. Agents can see MCPs on LiteLLM

Just register the MCP as a tool on A2A for it to work

Note: set the base url of the MCP server to the litellm proxy base - {LITELLM_PROXY_API_BASE}/

async def init_agent(self):
        logger.info(f'Initializing {self.agent_name} metadata')
        config = get_mcp_server_config()
        logger.info(f'MCP Server url={config.url}')
        tools = await MCPToolset(
            connection_params=SseServerParams(url=config.url)
        ).get_tools()

        for tool in tools:
            logger.info(f'Loaded tools {tool.name}')
        generate_content_config = genai_types.GenerateContentConfig(
            temperature=0.0
        )
        LITELLM_MODEL = os.getenv('LITELLM_MODEL', 'gemini/gemini-2.0-flash')
        self.agent = Agent(
            name=self.agent_name,
            instruction=self.instructions,
            model=LiteLlm(model=LITELLM_MODEL),
            disallow_transfer_to_parent=True,
            disallow_transfer_to_peers=True,
            generate_content_config=generate_content_config,
            tools=tools,
        )
        self.runner = AgentRunner()

Open Questions

1. Is there a well known api spec we can adopt for allowing client apps to discover multiple agents?