llms.txt

# memex — llms.txt

> Map of where things live, for AI agents that just cloned the repo.
> Read CLAUDE.md first (it's loaded automatically), this file second.

## What this repo is

Self-hostable personal knowledge brain, reached over MCP only:
- **memex** — knowledge brain (Bun + Postgres + pgvector). Hybrid
  search + graph-signals ranking, entity + code call graph
  (callers/callees), push-context, advisor, 61 MCP tools. Retrieval-first
  — Bedrock Titan v2 embeddings + Claude Haiku intent/expansion; answer synthesis
  is the MCP client's job. An opt-in, off-by-default LLM step distils notes
  into a separate `synth_*` store (source notes never touched). PGLite is a
  dev-only fallback.
- **cloudflared** — public HTTPS ingress for the MCP server
  (`brain.<domain>/mcp`) so MCP clients (Claude Code, Cursor, …) connect

Both run as Docker Compose services on a single `t4g.medium` EC2.

## Where to read first

| Want to know... | Read |
|---|---|
| Project conventions, gotchas, never-do-X rules | `CLAUDE.md` |
| Topology + diagrams + resource inventory | `ARCHITECTURE.md` |
| What changed when, with reasoning | `CHANGELOG.md` |
| Explicitly deferred work + when to pick it up | `TODO.md` |
| How to behave inside this repo (build / test / commit) | `AGENTS.md` |
| memex subsystem deep-dive | `deploy/memex/docs/{ARCHITECTURE,API,OPERATIONS,README}.md` |
| Onboarding (`make init` flow) | `README.md` |

## Source code map

```
deploy/
├── docker-compose.yml           orchestrator (memex + cloudflared + internal net)
├── memex/                       knowledge brain
│   ├── Dockerfile               oven/bun:1.3.13-alpine
│   ├── package.json             pinned deps incl. yaml + chokidar + AWS SDK Bedrock + postgres
│   ├── memex.yml.example        declarative runtime config overlay
│   ├── templates/               SOUL/USER/ACCESS_POLICY/HEARTBEAT.md.template (seeded by init)
│   ├── src/
│   │   ├── cli.ts               argparse + subcommand dispatch
│   │   ├── commands/            init, serve, index, search, reindex, doctor,
│   │   │                        integrity, eval, backlinks, extract,
│   │   │                        reconcile-links, orphans, pages, lint, reports,
│   │   │                        skillpack
│   │   ├── http/                two-route HTTP surface (/health + /mcp;
│   │   │                        legacy REST routes removed in Phase A.7)
│   │   ├── mcp/                 JSON-RPC transport, tool dispatch, rate limiter
│   │   └── core/
│   │       └── engine/          adapter pattern (interface, pglite, postgres, factory)
│   └── docs/                    subsystem deep-dive (README, ARCHITECTURE, API, OPERATIONS)
├── cloudflared/                 public ingress sidecar (upstream image, no Dockerfile)
│   └── docs/
├── secrets/                     AWS Secrets Manager fetch
│   ├── fetch-secrets.sh         reads <secrets_prefix>/* into deploy/.secrets/
│   └── README.md
└── systemd/                     timer + service for the bearer-rotation job

scripts/
├── audit.sh                     PII gate
├── init.sh                      interactive bootstrap (.env, tfvars, backend.hcl)
├── lib/
│   ├── pii-patterns.txt         audit input
│   └── pii-allowlist.txt        files exempt from audit
├── bootstrap.sh                 EC2 first-boot — runs from cloud-init user_data
└── rotate-memex-public-bearer.sh   daily bearer rotation (restarts memex)

terraform/
├── main.tf                      providers + partial backend
├── variables.tf                 all input vars
├── *.tf                         per-resource files
├── terraform.tfvars.example     schema doc (real tfvars is gitignored)
├── backend.hcl.example          schema doc (real backend.hcl is gitignored)
└── user_data.sh.tftpl           cloud-init user_data template

tests/
├── audit.test.sh, init.test.sh  bash unit tests
└── test_*.py                    pytest suites (memex skeleton, dockerfiles, etc.)

Makefile                         orchestration entry points
```

## Conventions you'll want to know

- Every secret is namespaced by `var.secrets_prefix` (default
  `memex`, override via `scripts/init.sh` for fresh installs).
- The `.env` at repo root is the runtime contract — it's rendered by
  `scripts/init.sh` (local) or `scripts/bootstrap.sh` (on EC2).
- `make audit` reads `scripts/lib/pii-patterns.txt` and fails on any
  maintainer-private identifier — used as a pre-publish gate.
- AI agents working on this repo MUST honor `CLAUDE.md` — no commits
  without explicit ask, no destructive terraform without showing plan
  first, no unrequested infrastructure additions.

## Where to start when implementing a change

1. Skim `ARCHITECTURE.md` to place the change in the topology.
2. Find the closest existing pattern via grep — `secrets_prefix`,
   `dynamic "egress"`, `${EFS_MOUNT}`, etc. — match it.
3. Add tests before code: bash units in `tests/*.test.sh`, pytest in
   `tests/test_*.py`.
4. Update `CHANGELOG.md`, `TODO.md`, and `ARCHITECTURE.md` in the same
   commit if behavior or topology changed.
5. `make audit && make test && terraform validate` before opening a PR.