fix(voice): include conversation items added during tool execution in reply context#6084
fix(voice): include conversation items added during tool execution in reply context#6084ShayneP wants to merge 1 commit into
Conversation
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
| # Conversational analog of the update_instructions() refresh below. | ||
| known_ids = {item.id for item in chat_ctx.items} | ||
| known_ids.update(item.id for item in tool_messages) | ||
| chat_ctx.items.extend( |
There was a problem hiding this comment.
Maybe
agents/livekit-agents/livekit/agents/voice/agent.py
Lines 966 to 972 in e87d3ee
… reply context When a function tool awaits an inline AgentTask, the sub-conversation it runs is merged into the agent's chat_ctx at handoff-return. The reply generated after tool execution used a chat_ctx snapshot taken before the tool ran, so it was blind to everything said inside the task and would re-ask for fields the task had already collected. Merge the agent's chat_ctx back into the snapshot (after adding the tool messages, so merge() dedups them) before generating the tool reply, mirroring the existing update_instructions() refresh.
ShayneP
force-pushed
the
ShayneP/agent-task-chat-ctx-refresh
branch
from
8746591 to
04ac3be
Compare
theomonnom
left a comment
theomonnom
left a comment
There was a problem hiding this comment.
lgtm! discussed in a meeting
| # Conversational analog of the update_instructions() refresh below. | ||
| # tool_messages must be added first so merge() dedups them by id. | ||
| chat_ctx.items.extend(tool_messages) | ||
| chat_ctx.merge(self._agent._chat_ctx, exclude_instructions=True) |
There was a problem hiding this comment.
🚩 exclude_instructions=True excludes all system/developer messages, not just the instruction message
The merge call at line 3056 passes exclude_instructions=True, which in chat_context.py:583-588 skips ALL items where item.type == 'message' and item.role in ['system', 'developer']. This means if the sub-conversation (run inside an inline AgentTask) produced any system or developer messages beyond instructions — e.g., system-level context injected by the sub-agent — those would be excluded from the merge into the tool-response generation context. This appears intentional since the update_instructions call at lines 3061-3065 handles instruction refresh separately, but it's worth noting that non-instruction system messages from sub-conversations will be silently dropped.
Was this helpful? React with 👍 or 👎 to provide feedback.
2 participants
When a function tool awaits an inline
AgentTask, the sub-conversation it runs is merged into the agent'schat_ctxat handoff-return. But the reply generated after tool execution uses achat_ctxsnapshot taken before the tool ran, so it's blind to everything said inside the task — in practice the agent re-asks for fields the task already collected.This refreshes the snapshot with any chat items added during tool execution before generating the tool reply, mirroring the existing
update_instructions()refresh just below it.Found while benchmarking the hotel receptionist example, where the card-collection
AgentTaskwould complete and the agent would immediately ask for the card number again.