Cache append-only Trace turn counts

[xeophon]

Overview

Cache Trace.num_turns incrementally so turn-limit and interception reads scale with newly appended graph nodes rather than the full rollout history.

Design

The message graph is owned by Trace.nodes, and normal graph commits append nodes. The trace now keeps a private (counted_nodes, sampled_turns) cache:

• unchanged node length returns the cached count immediately
• growth counts only the unseen suffix
• observed shrinkage resets and rebuilds the count from the remaining nodes

The cache is a Pydantic private attribute, so wire and disk serialization remain unchanged and deserialized traces begin with a cold cache. Turn semantics are also unchanged: only nodes marked sampled count, independent of branch topology. This does not restructure the graph or alter interception control flow.

For a progressive rollout with a count read after every append, the prior expression visits N(N + 1) / 2 nodes. The incremental path visits each appended node once, reducing that work from quadratic to linear.

Performance

Measured on 20,000 progressive sampled turns with separate old/new processes and three uninstrumented timing repetitions:

Metric	Previous	Incremental	Saved
Median wall time	3.439785792 s	0.081755542 s	3.358030250 s (97.62%)
Median CPU time	3.437374 s	0.081643 s	3.355731 s (97.62%)
p99 synchronous count latency	456.542 µs	1.000 µs	455.542 µs (99.78%)
Node visits	200,010,000	20,000	199,990,000 (99.99%)

Resource measurements stayed effectively flat:

Resource	Previous	Incremental	Difference
tracemalloc peak	35,529,111 B	35,529,554 B	+443 B
tracemalloc retained after trace deletion	155 B	155 B	0 B
Live RSS increase over baseline	68,632,576 B	68,567,040 B	-65,536 B
Process peak RSS	178,782,208 B	178,487,296 B	-294,912 B
Async tasks / network connections	0 / 0	0 / 0	unchanged

The incremental implementation performs no suffix-list copy; it walks newly appended nodes directly by index.

---

Note

Low Risk
Localized performance optimization to a read-only property with explicit cache invalidation on list shrinkage; no wire format or counting semantics change.

Overview
Trace.num_turns no longer rescans every node on each read. A private _num_turns_cache stores (len(nodes), sampled turn count) and only walks newly appended suffix nodes when the graph grows; if nodes shrinks, the cache resets and rebuilds.

Turn semantics are unchanged (only sampled nodes count). Serialization is unchanged because the cache is a Pydantic PrivateAttr. This targets hot paths such as turn-limit checks during progressive rollouts, cutting repeated work from quadratic to linear in node count.

^{Reviewed by Cursor Bugbot for commit f964fcd. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Cache Trace.num_turns incrementally for append-only message graphs

The num_turns property in trace.py previously summed over all nodes on every call. It now uses a _num_turns_cache tuple (counted_nodes, sampled_turns) to track progress incrementally, only iterating over newly appended nodes. If the node list shrinks, the cache is rebuilt from scratch.

^{Macroscope summarized f964fcd.}

[macroscopeapp]

Approvability

Verdict: Approved

This is a straightforward performance optimization that caches turn counts to avoid redundant iteration over nodes. The return value semantics are unchanged, and edge cases (list growth/shrink) are handled correctly.

^{You can customize Macroscope's approvability policy. Learn more.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f964fcdb1a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Invalidate turn cache for same-length node changes

When trace.num_turns has been read once, this early return makes the cache depend only on len(trace.nodes). Because nodes is a public mutable trace field, any same-length node replacement or sampled-flag correction leaves num_turns stale; for example, after changing one node from sampled=False to True, turn-limit checks and metrics continue to see the old count until the list length changes. Either invalidate the cache on node assignment/mutation or avoid the cached fast path when append-only mutation cannot be guaranteed.

Useful? React with 👍 / 👎.