Skip to content

feat(agent-memory): index bootstrap — MemoryStore.ensureIndex()#261

Merged
jamby77 merged 1 commit into
masterfrom
feature/agent-memory-ensure-index
Jun 22, 2026
Merged

feat(agent-memory): index bootstrap — MemoryStore.ensureIndex()#261
jamby77 merged 1 commit into
masterfrom
feature/agent-memory-ensure-index

Conversation

@jamby77

@jamby77 jamby77 commented Jun 18, 2026

Copy link
Copy Markdown
Collaborator

Agent Memory — index bootstrap (ensureIndex)

Stacked on #259 (Phase 11).

Why

MemoryStore queries {name}:mem:idx in five paths (recall, forgetByScope, consolidate, capacity count + scan) but the package never creates that index — it was implicitly the caller's job, with the integration suite hand-rolling an FT.CREATE. This is the gap KIvanow flagged in review on #249/#250, and spec-agent-memory.md (data model + "reuse the valkey-search helper for index create/version-skew handling") calls for the package to own it. No phase had picked it up.

What's new

  • buildMemoryIndexbuildMemoryIndexArgs(name, dims) / memoryIndexName(name): the FT.CREATE arg list for the memory schema — vector (HNSW, FLOAT32, COSINE), threadId/agentId/namespace/tags(comma-sep)/source as TAG, importance/created_at/last_accessed_at/access_count as NUMERIC, content as TEXT. Matches exactly the fields buildMemoryRecord writes and buildRecallQuery/buildScopeFilter query.
  • MemoryStore.ensureIndex() — idempotent: FT.INFO → on isIndexNotFoundError (from @betterdb/valkey-search-kit) → FT.CREATE. Resolves the vector dimension from embedFn (probe) when no write has been observed yet, and reuses a dimension already learned from a write. Non-"not found" errors propagate. Mirrors Retriever.createIndex() in the retrieval SDK.
  • AgentMemory.initialize() now calls memory.ensureIndex() first, so a freshly constructed facade is immediately usable for remember/recall without the caller hand-rolling the index. A create failure surfaces (the tier is unusable without it).

Tests

  • 13 new hermetic unit tests: buildMemoryIndex arg shape + dim validation; ensureIndex create-when-missing / idempotent / dim-probe / dim-reuse / non-not-found-rethrow; AgentMemory.initialize() creates the index.
  • The integration suite now dogfoods store.ensureIndex() instead of a hand-rolled FT.CREATE. (Skip-guarded; runs against a live valkey-search in CI / with the bundle up — it was not run locally here, as the bundle wasn't running. The hermetic tests verify the arg construction and idempotency.)
  • tsc --noEmit + prettier clean; 115 tests pass in the package.

Notes

  • The index is HNSW (per spec, production-appropriate). The integration suite previously hand-rolled a FLAT index for test simplicity; switching it to ensureIndex aligns the test on the real production path.
  • Optional follow-up (spec's literal intent): lift a generic FT.CREATE builder into @betterdb/valkey-search-kit so retrieval and agent-memory share one. Not required here — the memory schema is fixed.

Note

Medium Risk
Index creation runs on initialize and locks vector dimension to the embedder; failures block startup, but behavior is idempotent and covered by new unit/integration tests.

Overview
agent-memory now owns creation of the {name}:mem:idx vector search index instead of leaving that to callers or integration tests.

A new buildMemoryIndex helper builds the FT.CREATE schema (HNSW cosine vector plus TAG/NUMERIC/TEXT fields aligned with writes and recall queries). MemoryStore.ensureIndex() checks FT.INFO, creates the index only when missing (using isIndexNotFoundError), and picks vector DIM from a prior write or an embedFn('probe') call. AgentMemory.initialize() awaits ensureIndex() before discovery so the facade is ready for remember/recall without manual index setup.

Hermetic tests cover index args, idempotency, dim resolution, and error propagation; the live integration suite drops its hand-rolled FT.CREATE (previously FLAT) in favor of ensureIndex() (HNSW).

Reviewed by Cursor Bugbot for commit 4728320. Bugbot is set up for automated code reviews on this repo. Configure here.

@jamby77 jamby77 force-pushed the feature/agent-memory-phase11-integration branch from e517bec to 8447b8e Compare June 19, 2026 09:51
@jamby77 jamby77 force-pushed the feature/agent-memory-ensure-index branch from cbef0e3 to 2d2dc3e Compare June 19, 2026 09:51
@jamby77 jamby77 force-pushed the feature/agent-memory-phase11-integration branch from 8447b8e to 602bdc4 Compare June 19, 2026 10:01
@jamby77 jamby77 force-pushed the feature/agent-memory-ensure-index branch from 2d2dc3e to d1c1837 Compare June 19, 2026 10:01
@jamby77 jamby77 force-pushed the feature/agent-memory-phase11-integration branch from 602bdc4 to ce94197 Compare June 19, 2026 11:32
@jamby77 jamby77 force-pushed the feature/agent-memory-ensure-index branch from d1c1837 to 7fd146a Compare June 19, 2026 11:32
@jamby77 jamby77 force-pushed the feature/agent-memory-phase11-integration branch from ce94197 to b258743 Compare June 19, 2026 11:47
@jamby77 jamby77 force-pushed the feature/agent-memory-ensure-index branch from 7fd146a to 9de1897 Compare June 19, 2026 11:47
@jamby77 jamby77 force-pushed the feature/agent-memory-phase11-integration branch from b258743 to 71793fa Compare June 19, 2026 13:10
@jamby77 jamby77 force-pushed the feature/agent-memory-ensure-index branch from 9de1897 to da84258 Compare June 19, 2026 13:10
@jamby77 jamby77 force-pushed the feature/agent-memory-phase11-integration branch from 71793fa to 7c2413d Compare June 19, 2026 13:34
@jamby77 jamby77 force-pushed the feature/agent-memory-ensure-index branch from da84258 to ea7ef4a Compare June 19, 2026 13:34
@jamby77 jamby77 force-pushed the feature/agent-memory-phase11-integration branch 2 times, most recently from c971f49 to df18e14 Compare June 22, 2026 07:02
Base automatically changed from feature/agent-memory-phase11-integration to master June 22, 2026 07:03
… index

The package queried {name}:mem:idx for recall/forget/consolidate/capacity
but never created it, leaving the index as an undocumented caller
responsibility (flagged in review on #249/#250). Spec calls for the package
to own index create/version-skew handling.

- buildMemoryIndex: HNSW + memory schema (vector, scope/tag TAGs, numeric
  tunables, content TEXT) as reusable FT.CREATE args
- MemoryStore.ensureIndex(): idempotent FT.INFO -> FT.CREATE, resolving the
  vector dimension from embedFn when not yet observed
- AgentMemory.initialize() creates the index so the facade is usable without
  a hand-rolled FT.CREATE
- Integration suite dogfoods ensureIndex() instead of hand-rolling the index
@jamby77 jamby77 force-pushed the feature/agent-memory-ensure-index branch from ea7ef4a to 4728320 Compare June 22, 2026 07:20
@jamby77 jamby77 merged commit ddcbf6c into master Jun 22, 2026
2 checks passed
@jamby77 jamby77 deleted the feature/agent-memory-ensure-index branch June 22, 2026 07:20
@github-actions github-actions Bot locked and limited conversation to collaborators Jun 22, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants