-
Notifications
You must be signed in to change notification settings - Fork 19.6k
fix(langchain): ensure HITL middleware edit decisions persist in agent state #33789
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
fix(langchain): ensure HITL middleware edit decisions persist in agent state #33789
Conversation
1c13e07 to
5bd39be
Compare
5bd39be to
5a2c343
Compare
Fix issues langchain-ai#33787 and langchain-ai#33784 where Human-in-the-Loop middleware edits were not persisting correctly in the agent's message history. The problem occurred because the middleware was directly mutating the AIMessage.tool_calls attribute, but LangGraph's state management doesn't properly persist direct object mutations. This caused the agent to see the original (unedited) tool calls in subsequent model invocations, leading to duplicate or incorrect tool executions. Changes: - Create new AIMessage instance instead of mutating the original - Ensure message has an ID (generate UUID if needed) so add_messages reducer properly replaces instead of appending - Add comprehensive test case that reproduces and verifies the fix
5a2c343 to
69d4f40
Compare
|
@sydney-runkle Hi! I've investigated the CI failure and found: The failing test is unrelated to my PR:
The test only fails on Python 3.12:
All my HITL tests pass:
This appears to be a Python 3.12-specific issue in |
e50a698 to
69d4f40
Compare
|
Hi @sydney-runkle, I've reverted the master merge that was causing CI failures. Here's what happened: Timeline:
Analysis:
Resolution:
The PR is ready for review. I can merge master again after the core test issue is resolved. |
Enhances the fix for issues langchain-ai#33787 and langchain-ai#33784 by adding a HumanMessage that informs the AI when a tool call has been edited by a human operator. This ensures that the AI's subsequent responses reference the edited parameters rather than the original request parameters. Changes: - Modified _process_decision to create a HumanMessage on edit - The message informs the AI about the edited tool call arguments - Uses HumanMessage instead of ToolMessage to avoid interfering with actual tool execution - Updated all affected tests to expect the context message - All 70 middleware agent tests pass This complements the previous fix that ensured tool calls persist correctly in state by also providing context to the AI about the edit.
- Updated _process_decision return type to allow HumanMessage - Updated artificial_tool_messages list type annotation - Removed unused BaseMessage import
This commit adds a before_model hook to inject a reminder message after tool execution for edited tool calls. This ensures the AI's final response references the edited parameters rather than the original user request. The fix addresses issue langchain-ai#33787 where the AI would generate a final response referencing the original parameters despite the tool being executed with edited parameters. Now a [System Reminder] message is injected after tool execution to provide context about the edited parameters. Changes: - Added _pending_edit_contexts dict to track edited tool calls - Added before_model hook to inject post-execution reminder messages - Updated test to expect two context messages (pre and post execution) - Added type guard for tool_call_id to satisfy mypy Fixes langchain-ai#33787
CI Failure AnalysisThe failing test in is unrelated to this PR's changes. Details:
Root Cause:This is a flaky timing-sensitive test: The test expects producer/consumer to run in parallel with minimal delay, but CI machine load caused 90ms delay, exceeding the 20ms tolerance. Evidence This is Unrelated:
Request: Could a maintainer please re-run the failed CI job? This appears to be a transient infrastructure issue. |
Description
Fixes a critical bug in the Human-in-the-Loop middleware where the agent's final response referenced the original tool call parameters instead of the edited ones.
The Problem
When a user edited a tool call via HITL middleware:
AIMessage.tool_callswould be updated in stateExample:
Root Causes (Fixed in 3 iterations)
This PR contains three progressive fixes addressing different layers of the problem:
Fix 1: Data Persistence (Commit 69d4f40)
Problem: Direct mutations to
AIMessage.tool_callsweren't persisting in LangGraph's state.Solution: Create a new
AIMessageinstance instead of mutating:Fix 2: Pre-execution Context (Commit ce9892f)
Problem: Even with persisted edits, the AI didn't know the tool call had been edited.
Solution: Add a
[System Note]message before tool execution:Fix 3: Post-execution Reminder (Commit 4d4039f) ← Critical Fix
Problem: The pre-execution context was too far from the AI's final response generation. The AI would "forget" the edit and summarize based on the user's original request.
Solution: Add a
before_model()hook that injects a reminder immediately before the AI generates its final response:Message Flow After All Fixes
Changes
_pending_edit_contexts: Dictionary to track edited tool calls across middleware hooksbefore_model()hook: Injects post-execution reminder messagesafter_model(): Records edit information for later use inbefore_model()if tool_call_id:to satisfy mypyTesting
✅ All 16 HITL middleware tests pass
✅ New test
test_human_in_the_loop_middleware_edit_actually_executes_with_edited_argsvalidates:✅ Linting passes
✅ Type checking passes
Issue
Fixes #33787
Fixes #33784
Dependencies
No new dependencies added.