Skip to content

Conversation

constantinius
Copy link
Contributor

@constantinius constantinius commented Sep 30, 2025

Description

We cannot directly intercept MCP Tool calls, as they are done remotely by the LLM and not in the Agent itself. However, we see when such a tool call took place, so we can emit a zero-length span with the tool call specifics. It will start at the same time as the parent span.

Closes https://linear.app/getsentry/issue/TET-1192/openai-agents-hosted-mcp-calls-cannot-be-wrapped-in-an-execute-tool


Note

Emit execute_tool spans for MCP tool calls detected in agent results, with tool metadata, input/output (PII-gated), and error status.

  • Tracing/Spans (openai_agents):
    • Add utils._create_mcp_execute_tool_spans to emit OP.GEN_AI_EXECUTE_TOOL spans for MCP tool calls (McpCall) found in result.output.
      • Sets GEN_AI_TOOL_TYPE=mcp, GEN_AI_TOOL_NAME, propagates input/output when PII allowed, and marks SPANSTATUS.ERROR on error.
      • Spans start at the parent span's start time (zero-length representation of remote call).
    • Wire into spans/ai_client.update_ai_client_span to create these tool spans after setting usage/input/output data.
    • Update imports to include SPANSTATUS and OP.

Written by Cursor Bugbot for commit 96df8c1. This will update automatically on new commits. Configure here.

@constantinius constantinius requested a review from a team as a code owner September 30, 2025 09:20
Copy link

linear bot commented Sep 30, 2025

Copy link

codecov bot commented Sep 30, 2025

❌ 4 Tests Failed:

Tests completed Failed Passed Skipped
23695 4 23691 2305
View the top 3 failed test(s) by shortest run time
tests.integrations.openai_agents.test_openai_agents::test_tool_execution_span
Stack Traces | 0.08s run time
.../integrations/openai_agents/test_openai_agents.py:469: in test_tool_execution_span
    assert agent_span["data"]["gen_ai.request.available_tools"] == available_tools
E   assert '[{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true}]' == '[{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true, "tool_input_guardrails": null, "tool_output_guardrails": null}]'
E     
E     - [{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true, "tool_input_guardrails": null, "tool_output_guardrails": null}]
E     ?                                                                                                                                                                                                                                                                                                                                                                                                                                      ---------------------------------------------------------------
E     + [{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true}]
tests.integrations.openai_agents.test_openai_agents::test_tool_execution_span
Stack Traces | 0.097s run time
.../integrations/openai_agents/test_openai_agents.py:469: in test_tool_execution_span
    assert agent_span["data"]["gen_ai.request.available_tools"] == available_tools
E   assert '[{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true}]' == '[{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true, "tool_input_guardrails": null, "tool_output_guardrails": null}]'
E     
E     - [{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true, "tool_input_guardrails": null, "tool_output_guardrails": null}]
E     ?                                                                                                                                                                                                                                                                                                                                                                                                                                      ---------------------------------------------------------------
E     + [{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true}]
tests.integrations.openai_agents.test_openai_agents::test_tool_execution_span
Stack Traces | 0.132s run time
.../integrations/openai_agents/test_openai_agents.py:469: in test_tool_execution_span
    assert agent_span["data"]["gen_ai.request.available_tools"] == available_tools
E   assert '[{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true}]' == '[{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true, "tool_input_guardrails": null, "tool_output_guardrails": null}]'
E     
E     - [{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true, "tool_input_guardrails": null, "tool_output_guardrails": null}]
E     ?                                                                                                                                                                                                                                                                                                                                                                                                                                      ---------------------------------------------------------------
E     + [{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true}]
tests.integrations.openai_agents.test_openai_agents::test_tool_execution_span
Stack Traces | 0.157s run time
.../integrations/openai_agents/test_openai_agents.py:469: in test_tool_execution_span
    assert agent_span["data"]["gen_ai.request.available_tools"] == available_tools
E   assert '[{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true}]' == '[{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true, "tool_input_guardrails": null, "tool_output_guardrails": null}]'
E     
E     - [{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true, "tool_input_guardrails": null, "tool_output_guardrails": null}]
E     ?                                                                                                                                                                                                                                                                                                                                                                                                                                      ---------------------------------------------------------------
E     + [{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true}]

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

cursor[bot]

This comment was marked as outdated.

cursor[bot]

This comment was marked as outdated.

Copy link
Contributor

@sentrivana sentrivana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check the cursor comment -- might have a point.

Looks good to me otherwise, but please also add a test.

cursor[bot]

This comment was marked as outdated.

cursor[bot]

This comment was marked as outdated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants