feat: support multiple A2AStarletteApplication on a single port #323
Conversation
…nstances or AgentExecutor implementations on a single port.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Summary of Changes
Hello @Chenxan, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request introduces a new example that demonstrates how to effectively manage and serve multiple A2A (Agent-to-Agent) applications or agent executors from a single network port. The core change involves a custom Starlette application that dynamically routes requests to different agent instances, providing a streamlined deployment model for complex multi-agent systems. It also includes a client-side agent capable of intelligently selecting and interacting with these diverse agents, showcasing a more robust and scalable approach to agent communication.
Highlights
- Multi-Agent Support on Single Port: Introduces a new example demonstrating how to run multiple A2AStarletteApplication instances or AgentExecutor implementations on a single network port, enhancing deployment flexibility.
- Dynamic A2A Application Routing: A new
DatabaseA2AStarletteApplicationis implemented to dynamically handle agent card requests and route incoming A2A messages to the appropriate agent based on the URL path. - Intelligent Client-Side Agent Orchestration: Includes a sophisticated client-side
Agentthat uses a Large Language Model (LLM) and Jinja2 templates to intelligently decide which A2A agent to interact with, enabling more complex multi-agent workflows. - Integrated Tool Calling with fastmcp: The server-side
CoreAgentExecutorintegrates with thefastmcplibrary to facilitate tool calling within the agent's logic, allowing agents to leverage external functionalities.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
This pull request adds a new Python sample demonstrating how to support multiple A2A agents on a single port. The implementation is quite comprehensive, covering both client and server logic. However, there are several critical issues related to mixed synchronous and asynchronous code that will cause runtime errors. I've also found some misleading naming, unused code, and configuration issues that should be addressed to improve clarity and correctness. My review includes suggestions to fix these problems.
| async def decide( | ||
| self, | ||
| question: str, | ||
| agents_prompt: str, | ||
| called_agents: list[dict] | None = None, | ||
| ) -> Generator[str, None]: |
There was a problem hiding this comment.
The decide function is defined as a coroutine (async def), but it doesn't use await and returns a non-awaitable value (a string or a generator from call_llm). This will cause a TypeError at runtime when you await it in the stream method on line 201.
To fix this, you should remove the async keyword from the decide function definition, as it performs synchronous operations.
| async def decide( | ||
| self, question: str, called_tools: list[dict] | None = None | ||
| ) -> Generator[str, None]: |
There was a problem hiding this comment.
The decide function is defined as a coroutine (async def), but it doesn't perform any awaitable operations. It calls the synchronous function call_llm and returns its result (a generator). When this function is awaited in the stream method (line 139), it will raise a TypeError because a generator is not awaitable.
To fix this, you should remove the async keyword from the decide function definition.
| self.agent = Agent( | ||
| mode='stream', | ||
| token_stream_callback=print, | ||
| mcp_url=[mcp_url[str(agent_index)]], | ||
| ) |
There was a problem hiding this comment.
There is a type mismatch when initializing the Agent. The mcp_url parameter in Agent.__init__ expects a string (str | None), but you are passing a list containing a single string: [mcp_url[str(agent_index)]]. This will cause an error when mcp_url is used inside the Agent class, as it expects a string for the URL.
You should pass the URL as a string directly.
| self.agent = Agent( | |
| mode='stream', | |
| token_stream_callback=print, | |
| mcp_url=[mcp_url[str(agent_index)]], | |
| ) | |
| self.agent = Agent( | |
| mode='stream', | |
| token_stream_callback=print, | |
| mcp_url=mcp_url[str(agent_index)], | |
| ) |
| @click.option('--mode', 'mode', default='streaming') | ||
| @click.option('--question', 'question', required=True) | ||
| async def a_main( | ||
| host: str, | ||
| port: int, | ||
| mode: Literal['completion', 'streaming'], |
There was a problem hiding this comment.
The mode command-line option is defined but its value is never used; the Agent is always initialized with mode='stream' on line 31. Additionally, the choices for the mode option ('completion', 'streaming') do not match the values expected by the Agent class ('complete', 'stream').
To fix this, you should pass the mode to the Agent and align the option choices.
| def call_llm(self, prompt: str) -> str: | ||
| """Call the LLM with the given prompt and return the response as a string or generator. | ||
|
|
||
| Args: | ||
| prompt (str): The prompt to send to the LLM. | ||
|
|
||
| Returns: | ||
| str or Generator[str]: The LLM response as a string or generator, depending on mode. | ||
| """ |
There was a problem hiding this comment.
The logic in call_llm seems to be inverted. When mode is 'complete', it returns a generator, and when it's 'stream', it consumes the generator and returns a single string. This is counter-intuitive and the opposite of what the mode names imply.
Additionally, the return type hint is str, but for mode == 'complete', it returns a Generator[str], which will cause a type error.
| name = "no-llm-framework" | ||
| version = "0.1.0" | ||
| description = "Use A2A without any agent framework" |
There was a problem hiding this comment.
The project name and description in this file are misleading. They seem to be from a different example. The name is "no-llm-framework" and description is "Use A2A without any agent framework", while this example is about supporting multiple agents on a single port. This can cause confusion. Please update them to be more descriptive of this specific example.
name = "a2a-multi-agent-on-single-port"
version = "0.1.0"
description = "An example of how to support multiple A2A agents on a single port"
| The Agent2Agent (A2A) Protocol is an open standard that enables seamless communication and collaboration between AI agents built on diverse frameworks and by different vendors. It provides a common language, breaking down silos and fostering interoperability. A2A facilitates dynamic, multimodal communication between agents as peers, allowing them to delegate sub-tasks, exchange information, and coordinate actions to solve complex problems. The protocol ensures security and preserves intellectual property by enabling agents to interact without sharing internal memory, tools, or proprietary logic. To get started, users can read the introduction, dive into the specification, follow tutorials, or explore code samples. | ||
|
|
||
| A code example is not explicitly provided in the document; however, the document references a Python quickstart tutorial: | ||
| ```markdow |
There was a problem hiding this comment.
There's a typo in the markdown block. It should be markdown instead of markdow.
```markdown
| mode: Literal['complete', 'stream'] = 'stream', | ||
| token_stream_callback: Callable[[str], None] | None = None, | ||
| agent_urls: list[str] | None = None, | ||
| agent_prompt: str | None = None, |
There was a problem hiding this comment.
The agent_prompt parameter in the __init__ method is not used within the method. It should either be used or removed to avoid confusion and improve code clarity.
| Previous agents have been called. {% for agent in called_agents %} | ||
| - Agent: {{ agent.name }} | ||
| - Prompt: {{ agent.prompt }} | ||
| - Answer: {{ agent.answer }} | ||
| -------------------------------- | ||
| {% endfor %} No newline at end of file |
There was a problem hiding this comment.
This Jinja template file seems to be unused in the codebase. The client/agent.py file uses agent_answer.jinja to render the history of called agents. If this file is not needed, it should be removed to keep the codebase clean.
| return context | ||
|
|
||
|
|
||
| class DatabaseA2AStarletteApplication(A2AStarletteApplication): |
There was a problem hiding this comment.
The class name DatabaseA2AStarletteApplication is a bit misleading, as this implementation uses InMemoryTaskStore and does not seem to interact with a database. A more descriptive name like MultiAgentA2AStarletteApplication would better reflect its purpose of handling multiple agents.
1 participant
Description
Shows how to support multiple A2AStarletteApplication instances or AgentExecutor implementations on a single port.
CONTRIBUTINGGuide.Fixes #272 🦕