Skip to content

Cache append-only Trace turn counts #1820

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
xeophon wants to merge 1 commit into
base: feat/nano-as-v1
Choose a base branch
from
+15 −1
Open

Cache append-only Trace turn counts #1820

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion verifiers/v1/trace.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,8 @@ class Trace(StrictBaseModel, Generic[TaskT, StateT]):
_head_index: dict = PrivateAttr(default_factory=dict)
"""`(parent, msg_hash) -> node_id` for the graph builder (`graph.prepare_turn` / `commit`);
rebuilt lazily from `nodes` after deserialization."""
_num_turns_cache: tuple[int, int] = PrivateAttr(default=(0, 0))
"""`(counted nodes, sampled turns)` for the append-only message graph."""

@property
def reward(self) -> float:
Expand Down Expand Up @@ -300,7 +302,19 @@ def num_branches(self) -> int:
def num_turns(self) -> int:
"""Total model turns (sampled responses) across all branches — prompt-supplied
assistant messages don't count."""
return sum(1 for n in self.nodes if n.sampled)
counted_nodes, num_turns = self._num_turns_cache
node_count = len(self.nodes)
# Graph commits append nodes, so count only the unseen suffix between reads.
# If a caller shrinks the list, rebuild the count from the remaining nodes.
if node_count == counted_nodes:
return num_turns
Comment on lines +309 to +310

@chatgpt-codex-connector chatgpt-codex-connector Bot Jun 21, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Invalidate turn cache for same-length node changes

When trace.num_turns has been read once, this early return makes the cache depend only on len(trace.nodes). Because nodes is a public mutable trace field, any same-length node replacement or sampled-flag correction leaves num_turns stale; for example, after changing one node from sampled=False to True, turn-limit checks and metrics continue to see the old count until the list length changes. Either invalidate the cache on node assignment/mutation or avoid the cached fast path when append-only mutation cannot be guaranteed.

Useful? React with 👍 / 👎.

if node_count < counted_nodes:
counted_nodes, num_turns = 0, 0
while counted_nodes < node_count:
num_turns += self.nodes[counted_nodes].sampled
counted_nodes += 1
self._num_turns_cache = (node_count, num_turns)
return num_turns

@property
def is_truncated(self) -> bool:
Expand Down
Loading