feat(recall): prefer_observations — dedupe raw facts superseded by observations

[nicoloboschi]

What

Adds a prefer_observations flag to recall. When you recall raw facts (world/experience) together with observation, any raw fact that a returned observation was consolidated from is dropped, so the observation supersedes it — no duplicate content.

Enabled by default. Set prefer_observations: false to keep raw facts even when an observation already covers them.

Why

Today, recalling all types returns the same information twice — once as a raw fact and once folded into an observation built from it. Users had to choose between:

• raw facts only (no consolidation benefit), or
• observations only (which lag behind the latest retains while consolidation catches up).

This flag gives the best of both: recall everything, but prefer the observation whenever it already covers a raw fact.

How

• Observations already store their provenance in memory_units.source_memory_ids. Dedup is an exact id-membership filter over that set — not semantic guessing. A fact that is semantically similar to an observation but not in its source list survives.
• Runs before the recall truncation, so the freed slots backfill with the next-best results and the result count stays at the requested budget.
• The default is flipped on only at the user-facing boundary (HTTP RecallRequest + MCP recall tool). The engine method recall_async defaults it to False so internal callers — notably consolidation, which needs the raw facts it folds into observations — are never silently deduped.

Scope of changes

• api/http.py — RecallRequest.prefer_observations (default true), threaded to engine
• engine/memory_engine.py — Step 4.8 dedup in the recall pipeline
• mcp_tools.py — flag on both recall tool variants
• Regenerated OpenAPI + Python/TS/Go/Rust SDKs
• Control plane proxy route + lib/api.ts types
• docs/developer/api/recall.mdx
• tests/test_recall_prefer_observations.py — provenance-based dedup (deterministic, no LLM), plus the user-facing-on / engine-off default split

Tests

uv run pytest tests/test_recall_prefer_observations.py — 5 passing:

• disabled → sources + observation all returned
• enabled → source facts dropped, observation kept
• enabled → non-source fact survives (provenance, not semantics)
• no-op when observation not in types
• default split: user-facing default True, engine default False

Behavior-change note for reviewers

Because the default is true and a default recall already includes observation, raw facts covered by an observation will now drop out of default recalls unless prefer_observations: false is passed. This is intentional. Internal recall paths (reflect, consolidation, mental-model triggers) are unaffected — they use the engine default (False).