Skip to content

Commit cedcbc2

Browse files
feat(memory): add local embedding provider via transformers.js (#95)
Wire EMBEDDING_PROVIDER=local to an in-process embedding adapter so PR-memory works without an external embedding API. LocalEmbeddingAdapter runs a sentence-transformers model (default Xenova/all-MiniLM-L6-v2, 384-dim) through @huggingface/transformers, forcing the WASM backend and a single thread because the Alpine runner image cannot load the native onnxruntime-node binary. The memory module now selects the embedding adapter from EMBEDDING_PROVIDER instead of hard-wiring Voyage, and EMBEDDING_MODEL becomes optional with a per-adapter default. A migration aligns pr_memory.embedding with EMBEDDING_DIMENSIONS, acting only when the dimension actually changes and truncating the (model-specific, now-incompatible) vectors since the column is NOT NULL. A persistent model-cache volume keeps the downloaded model across restarts.
1 parent 0db8c80 commit cedcbc2

10 files changed

Lines changed: 1037 additions & 10 deletions

File tree

.env.example

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,9 +29,17 @@ LLM_API_KEY=
2929
# Provider: voyage | local
3030
EMBEDDING_PROVIDER=voyage
3131
VOYAGE_API_KEY=
32-
# EMBEDDING_MODEL=voyage-3-lite
33-
# Must match the embedding model (voyage-3-lite=512, voyage-3=1024).
32+
# EMBEDDING_MODEL defaults per provider (voyage: voyage-3-lite, local: Xenova/all-MiniLM-L6-v2)
33+
# EMBEDDING_MODEL=
34+
# Must match the embedding model (voyage-3-lite=512, voyage-3=1024, all-MiniLM-L6-v2=384).
3435
# EMBEDDING_DIMENSIONS=512
36+
#
37+
# Local embeddings: runs a sentence-transformers model in-process via
38+
# transformers.js (WASM), no API key. Downloads the model once to the cache dir.
39+
# EMBEDDING_PROVIDER=local
40+
# EMBEDDING_MODEL=Xenova/all-MiniLM-L6-v2
41+
# EMBEDDING_DIMENSIONS=384
42+
# EMBEDDING_CACHE_DIR=/app/models
3543

3644
# ===========================================================================
3745
# Database & Redis (required)

docker-compose.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@ services:
1010
- NODE_ENV=production
1111
extra_hosts:
1212
- 'host.docker.internal:host-gateway'
13+
volumes:
14+
- modelcache:/app/models
1315
depends_on:
1416
db:
1517
condition: service_healthy
@@ -55,3 +57,4 @@ services:
5557
volumes:
5658
pgdata:
5759
redisdata:
60+
modelcache:

0 commit comments

Comments
 (0)