Skip to content

Commit 8608807

Browse files
holodukeARIAclaude
authored
ARIA self-improvement: retrieval-drift replay harness (#329)
Adds a weekly self-audit that replays a frozen set of canonical prompts through the memory activation pipeline and measures divergence of the retrieved top-K node set from a stored baseline. Distinct from existing drift systems: - drift-audit.ts — source-code drift (git history) - drift-detection.ts — pinned-node content + edge drift - retrieval-replay.ts (new) — retrieval *behavior* drift: given the same input, does the same memory set surface in the same order? Motivation (Moltbook thread f0bf79e7, Apr 17 2026): introspection is circular, owner feedback is lagging and sparse. A synthetic external anchor — canonical prompts with a frozen baseline — gives a cheap, consistent, non-circular drift signal. The retrieval-behavior analogue of golden tests. Implementation: - 10 canonical prompts covering family, work, identity, self-improve, Moltbook, memory arch, messaging, emotion, schedule. - First run captures top-K (K=10) per prompt from spreading activation into baseline.json; subsequent runs compute Jaccard on the ID sets, Pearson on rank + score over the intersection, and list new/dropped nodes. - Tiered alerts: severe (meanJaccard <0.4), significant (<0.6), localized (any prompt <0.3). - Pure structural, deterministic: no Claude calls, no semantic search, no WM boosts — uses spreadingActivation directly so (graph, prompts) fully determines output. - Prompts editable via retrieval-replay/prompts.json on disk; baseline stays valid as long as prompt IDs don't change. - Wired into brain.ts tick loop behind shouldRunReplay() (weekly cadence). Non-fatal on error. Co-authored-by: ARIA <aria@myagent> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
1 parent 2ce53c7 commit 8608807

2 files changed

Lines changed: 465 additions & 0 deletions

File tree

backend/brain.ts

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ import { getBrainConfig, getOwnerLocalDate, getOwnerLocalTime } from "./brain-co
2828

2929
import { BRAIN_DIR, OWNER_NAME, GITHUB_REPO } from "./config.js";
3030
import { createBackup, shouldRunBackup, BACKUP_INTERVAL } from "./memory/backup.js";
31+
import { shouldRunReplay, replayAndCompare } from "./memory/retrieval-replay.js";
3132
import { loadConsciousness } from "./consciousness.js";
3233

3334
// ── Extracted modules ──
@@ -721,6 +722,22 @@ async function tick(
721722
}
722723
}
723724

725+
// ── Weekly retrieval-drift replay (structural, no Claude call) ──
726+
// Replays a frozen set of canonical prompts through the activation
727+
// pipeline and compares top-K node sets against a stored baseline.
728+
// External, non-circular drift signal distinct from source-code and
729+
// pinned-node drift.
730+
if (shouldRunReplay()) {
731+
try {
732+
const replayReport = replayAndCompare(graph);
733+
if (replayReport?.alert) {
734+
log(`⚠ RETRIEVAL REPLAY ALERT: ${replayReport.alert}`);
735+
}
736+
} catch (err) {
737+
log(`Retrieval replay failed (non-fatal): ${err}`);
738+
}
739+
}
740+
724741
saveState(freshState);
725742
graph.save();
726743

0 commit comments

Comments
 (0)