RFC: tapes memory layer

[bdougie]

The current search pipeline is unstructured recall, embed nodes, query by similarity, return branches. It answers "what past conversations are similar to X?" but doesn't synthesize or prioritize.

What a memory layer could add

A memory config, modeled after the search config pattern, could introduce structured recall:

Short-term memory: branch-scoped context

• Recent nodes within the current proxy branch
• Sliding window or token budget (e.g., last N nodes)
• Cheap, fast, ephemeral: lives in the worker pool or in-memory storage driver

Long-term memory: persistent, cross-branch knowledge

• Extracting durable facts/patterns from conversations into a knowledge graph
• Goes beyond vector similarity
• Could use a graph store (Cognee-style) or a curated vector collection with metadata

How it could map to tapes' config pattern

[memory]
provider = "cognee"       # or "local", "graph"
target = "http://..."

[memory.short_term]
enabled = true
window = 10               # last N nodes in current branch

[memory.long_term]
enabled = true
provider = "cognee"       # knowledge graph extraction

This follows the same optional-feature pattern as search:

• Zero config — proxy works without it
• Graceful degradation — returns "memory not configured" if missing
• Driver interface — memory.Driver with Store(), Recall(), Forget()
• Pluggable backends — local graph, Cognee, or custom

Key difference from search

Search is query-driven (user asks, system retrieves). Memory would be context-driven: the proxy automatically injects relevant recalled context into LLM requests before forwarding them upstream.

[linear]

PCC-102

[bdougie]

I asked claude to draw a proposed diagrram.

tapes memory layer — architecture diagram

How the proposed memory.Driver integrates with the existing storage, vector, and proxy pipeline.

                         ┌─────────────────────────────────────┐
                         │            LLM Client               │
                         └──────────────┬──────────────────────┘
                                        │
                                   req  │  ▲ resp
                                        ▼  │
                         ┌─────────────────────────────────────┐
                         │          tapes proxy                │
                         │                                     │
                         │  ┌───────────────────────────────┐  │
                         │  │      memory.Driver (NEW)      │  │
                         │  │                               │  │
                         │  │  Recall() ──► inject context  │  │
                         │  │  into req before forwarding   │  │
                         │  └───────────────────────────────┘  │
                         │                                     │
                         └──────────────┬──────────────────────┘
                                        │
                                   req  │  ▲ resp
                                        ▼  │
                         ┌─────────────────────────────────────┐
                         │          LLM Upstream               │
                         │    (OpenAI / Anthropic / Ollama)    │
                         └─────────────────────────────────────┘


  ═══════════════════════════════════════════════════════════════
   After response, worker pool processes the job asynchronously
  ═══════════════════════════════════════════════════════════════


         ┌──────────────────────────────────────────┐
         │          Worker Pool (async)              │
         │                                          │
         │  processJob(job)                         │
         │    │                                     │
         │    ├──► storeConversationTurn()           │
         │    │         │                           │
         │    │         ▼                           │
         │    │   ┌──────────┐                      │
         │    │   │ storage  │  SQLite (ent)        │
         │    │   │ .Driver  │  Merkle DAG nodes    │
         │    │   └──────────┘                      │
         │    │                                     │
         │    ├──► storeEmbeddings()   (EXISTING)   │
         │    │         │                           │
         │    │         ▼                           │
         │    │   ┌──────────┐    ┌────────────┐   │
         │    │   │ Embedder │───►│ vector     │   │
         │    │   │ (Ollama) │    │ .Driver    │   │
         │    │   └──────────┘    │ (sqlitevec)│   │
         │    │                   └────────────┘   │
         │    │                                     │
         │    └──► storeMemory()       (NEW)        │
         │              │                           │
         │              ▼                           │
         │        ┌──────────────┐                  │
         │        │ memory.Driver│                  │
         │        └──────┬───────┘                  │
         │               │                          │
         └───────────────┼──────────────────────────┘
                         │
            ┌────────────┴────────────┐
            ▼                         ▼
   ┌─────────────────┐    ┌────────────────────┐
   │  Short-Term     │    │  Long-Term         │
   │                 │    │                    │
   │  in-memory      │    │  Cognee / graph    │
   │  recent nodes   │    │  knowledge store   │
   │  from current   │    │                    │
   │  branch via     │    │  Store() extracts  │
   │  ParentHash     │    │  durable facts     │
   │  traversal      │    │  across branches   │
   └─────────────────┘    └────────────────────┘
            │                         │
            └────────────┬────────────┘
                         │
                    Recall()
                         │
                         ▼
              ┌─────────────────────┐
              │  Merged context     │
              │  injected into      │
              │  proxy request      │
              │  before upstream    │
              └─────────────────────┘

No changes to existing storage:

• storage.Driver (SQLite/ent) keeps storing Merkle DAG nodes exactly as-is
• vector.Driver (sqlitevec/chroma) keeps storing embeddings for search
• Search API stays untouched

Memory hooks into two existing points:

1. Worker pool processJob() — after storeConversationTurn() and storeEmbeddings(), a new storeMemory() step feeds nodes to the memory driver
2. Proxy request path — before forwarding to upstream, memory.Driver.Recall() retrieves relevant context and injects it into the request

Short-term just walks ParentHash links in the existing DAG — no new storage needed. Long-term is the only new storage dependency (Cognee or a local graph).

[jpmcb]

RFC looks very solid to me! Great work! I love the idea of extending the storage.Driver and vector.Driver paradigms into memory.Driver: I think that package would be a good approach.

Something to think through is the memory.Driver interface and how it'll work with node hashes, how "facts" are de-dupped back to the storage layer:

// Driver handles storage, recall, and lifecycle of conversation memory.
// Implementers can provide short-term (branch-scoped) and long-term (cross-branch)
// recall capabilities.
type Driver interface {
	// Store persists one or more nodes into memory.
	Store(ctx context.Context, nodes []*merkle.Node) error

	// Recall retrieves relevant memories for the given query context.
	// branchHash scopes recall to short-term memory on a specific branch (pass ""
	// to skip short-term recall). Limit caps the total results returned.
	Recall(ctx context.Context, query string, branchHash string, limit int) ([]RecallRes, error)

	// Forget removes memories associated with the given node hashes.
	Forget(ctx context.Context, hashes []string) error

	// Close releases driver resources.
	Close() error
}

Something else we might also consider doing in the future that'd be good to keep in mind is how overloaded the worker pool might get: these all should be relatively simple operations (POST, GET, put to tables, etc.) but keeping them in a sole thread could be painful / slow. We could fan out the worker pool instead of making it sequential:

processJob 
  --> fan out to sepearte go routines
     --> handle storage 
     --> handle embedding / vec 
     --> handle memory
  --> collect errors, results