refactor(knowledge): restructure into domain services and adopt core/job primitives

docs/references/data/database-construction.md

@@ -100,7 +100,7 @@ A table rebuild (drizzle's `INSERT…SELECT` drops the implicit rowid) **and `VA
 
 ### Knowledge `search_text_fts` follows the same rule
 
-`src/main/features/knowledge/vectorstore/indexStore/schema.ts` keys `search_text_fts` on a stable `fts_rowid` column too (assigned by the `search_text_ai` trigger; `content_rowid='fts_rowid'`). It is a **separate per-base `index.sqlite`** (not the main DB, not drizzle-managed, not in `CUSTOM_SQL_STATEMENTS`), but the same hazard applies: its `reclaim()` path runs `VACUUM` to return freed pages to the OS after a large delete, which renumbers the implicit rowid — keying on `fts_rowid` keeps the external-content index aligned by construction. The regression guard is `KnowledgeIndexStore.test.ts` → "keeps search_text_fts aligned after a rowid-reshuffling rebuild".
+`src/main/features/knowledge/pipeline/vectorstore/indexStore/schema.ts` keys `search_text_fts` on a stable `fts_rowid` column too (assigned by the `search_text_ai` trigger; `content_rowid='fts_rowid'`). It is a **separate per-base `index.sqlite`** (not the main DB, not drizzle-managed, not in `CUSTOM_SQL_STATEMENTS`), but the same hazard applies: its `reclaim()` path runs `VACUUM` to return freed pages to the OS after a large delete, which renumbers the implicit rowid — keying on `fts_rowid` keeps the external-content index aligned by construction. The regression guard is `KnowledgeIndexStore.test.ts` → "keeps search_text_fts aligned after a rowid-reshuffling rebuild".
 
 ## 5. Testing the build
 

docs/references/knowledge/experiment/knowledge-technical-design.md

@@ -101,7 +101,7 @@ Unique on `(target_type, target_id, kind)`; both FTS and vectors enter through `
 
 ### 4.7 search_text_fts
 
-External-content FTS5 (trigram). **FTS hits must join back through `search_text.rowid = search_text_fts.rowid`** — `search_text_id` is a TEXT business key, not the FTS rowid.
+External-content FTS5 (trigram). **FTS hits must join back through `search_text.fts_rowid = search_text_fts.rowid`** — `fts_rowid` is a stable integer surrogate column (not SQLite's implicit rowid), because VACUUM and any `INSERT...SELECT` rebuild renumber the implicit rowid and would silently desync an external-content index keyed on it. `search_text_id` is a separate TEXT business key, not the FTS rowid.
 
 ## 5. Index interface and implementation notes
 
@@ -149,7 +149,7 @@ A vector base's `knowledge_base.embeddingModelId` / `dimensions` must be valid t
 1. Relational tables use generic SQLite DDL only; FTS5 is built into both engines; CJK handling lives in the application layer.
 2. **Decision A1**: the canonical vector storage is a plain `BLOB` column holding little-endian float32 bytes (a generic SQLite BLOB, not any engine-proprietary vector type); it is the source of truth and both engines read the same bytes.
 3. First-version vector retrieval is a brute-force scan over the canonical BLOBs (sqlite-vec's `vec_distance_cosine(col, ?)`, binding the query vector as raw little-endian float32 BLOB bytes — no `vector32()` wrapper), exposed through the `VectorIndex` adapter; **no** vec0 / ANN derived index (left as a purely additive change after performance evaluation).
-4. A thin `SqliteDriver` port (execute / transaction / close) so the store is written once; better-sqlite3 keeps one synchronous, persistent connection — PRAGMAs (e.g. WAL) are set once and persist — and per-base writes are serialized by `KnowledgeLockManager.withBaseMutationLock(baseId)`, so the driver carries no write mutex or busy-retry of its own.
+4. A thin `SqliteDriver` port (execute / transaction / close) so the store is written once; better-sqlite3 keeps one synchronous, persistent connection — PRAGMAs (e.g. WAL) are set once and persist — and per-base writes are serialized by the per-base mutation lock (`KeyedMutex.runExclusive`), so the driver carries no write mutex or busy-retry of its own.
 
 ## 6. Retrieval
 
@@ -184,6 +184,6 @@ A vector base's `knowledge_base.embeddingModelId` / `dimensions` must be valid t
   - An explicit `maxParallelCalls` (plus token-aware batching) for `AiService.embedMany`, so one large document cannot fan out unbounded batches, exceed provider per-request token limits, and discard embeddings already paid for in a failed attempt.
   - Startup-recovery cross-cancellation: a crash-recovered delete-subtree job and the `recoverDeletingItems` re-enqueue get different idempotency keys and cancel each other via roots-intersection (`jobTouchesSubtree`); cancel only jobs whose roots are fully covered by the current job's roots.
   - Hybrid search runs its two lanes as independent read snapshots; a rebuild committing between them can transiently return both copies of a chunk — close with a shared read transaction or a second dedupe by material id + unit index.
-  - The per-base index driver's `close()` no longer needs to take a write mutex: better-sqlite3 uses one synchronous, persistent connection and per-base writes are serialized by `KnowledgeLockManager.withBaseMutationLock(baseId)`, so there is no driver-level async write mutex to race — shutdown safety rests on JobManager draining before the store service stops.
+  - The per-base index driver's `close()` no longer needs to take a write mutex: better-sqlite3 uses one synchronous, persistent connection and per-base writes are serialized by the per-base mutation lock (`KeyedMutex.runExclusive`), so there is no driver-level async write mutex to race — shutdown safety rests on JobManager draining before the store service stops.
   - Retrieval-surface follow-ups (PR C): the `searchMode` `default`→`vector` rename is externally visible through the gateway's pass-through base entity, and a permanent open failure (legacy layout) currently maps to a retryable 503.
 - PR A's full test matrix and risk notes live in this repo's test suites (`src/main/features/knowledge/**/__tests__`) and the PR #15973 description.

docs/references/knowledge/knowledge-service.md

@@ -2,7 +2,7 @@
 
 This document records the current v2 knowledge backend shape in the main process.
 
-It covers the `src/main/services/knowledge` workflow path and the SQLite-backed data services. It does not describe the legacy `src/main/knowledge` service or the old `knowledge-base:*` IPC channels.
+It covers the `src/main/features/knowledge` workflow path and the SQLite-backed data services. It does not describe the legacy `src/main/knowledge` service or the old `knowledge-base:*` IPC channels.
 
 For workflow guard details, see [Knowledge Operation Guards](./operation-guards.md). For the workflow architecture overview, see [Knowledge Workflow Architecture](./workflow-architecture.md).
 
@@ -21,15 +21,14 @@ The current implementation is split into four responsibility areas:
    - Expose database-backed list/get operations and base metadata/config patch.
    - Do not perform vector-store mutations.
 3. `KnowledgeService`
-   - Owns caller-facing runtime IPC workflow.
-   - Creates/deletes/restores bases through data services and vector store services.
-   - Registers Knowledge JobManager handlers.
-   - Holds the `KnowledgeWorkflowService` and `KnowledgeLockManager`.
-   - Collapses delete/reindex item inputs to top-level roots and enforces runtime guards.
+   - Thin lifecycle facade: registers Knowledge JobManager handlers, runs boot recovery, and delegates every public method to `base/`, `ingestion/`, and `query/`.
+   - `base/` (`KnowledgeBaseAdminService`) creates/deletes/restores bases through data services and vector store services. The per-base mutation lock (a core `KeyedMutex`) is created by the `KnowledgeService` facade and shared with `base/`, `ingestion/`, and the job handlers.
+   - `ingestion/` (`KnowledgeIngestionService`) collapses delete/reindex item inputs to top-level roots, enforces runtime guards, and schedules the next workflow step.
+   - `query/` (`KnowledgeQueryService` / `KnowledgeConceptService`) serves search and the Concept ID tool surface.
 4. Knowledge job handlers
    - Execute durable workflow stages through JobManager.
-   - Use `KnowledgeWorkflowService` for next-step scheduling.
-   - Use `KnowledgeLockManager` for same-base mutations and vector cleanup.
+   - Use `KnowledgeIngestionService` for next-step scheduling.
+   - Use the per-base mutation lock (`KeyedMutex.runExclusive`) for same-base mutations and vector cleanup.
 
 ```text
 caller
@@ -39,13 +38,13 @@ caller
 caller
   -> preload knowledge IPC
      -> KnowledgeService
-        -> KnowledgeWorkflowService
+        -> KnowledgeIngestionService
         -> JobManager
            -> knowledge.prepare-root / knowledge.index-documents
            -> knowledge.delete-subtree / knowledge.reindex-subtree
-              -> KnowledgeLockManager
+              -> KeyedMutex.runExclusive
                  -> KnowledgeBaseService / KnowledgeItemService
-                 -> KnowledgeVectorStoreService / FileManager
+                 -> KnowledgeVectorStoreService
 ```
 
 There is no current `KnowledgeRuntimeService` and no in-memory Knowledge queue. Durable work is owned by `JobManager`.
@@ -104,21 +103,19 @@ reindex-items(baseId, itemIds)
 
 ## IPC Surface
 
-`KnowledgeService` currently owns these public IPC entrypoints:
+`KnowledgeService` currently owns these public IpcApi routes, defined in `src/shared/ipc/schemas/knowledge.ts` and handled in `src/main/ipc/handlers/knowledge.ts`:
 
-- `knowledge:create-base`
-- `knowledge:restore-base`
-- `knowledge:delete-base`
-- `knowledge:add-items`
-- `knowledge:delete-items`
-- `knowledge:reindex-items`
-- `knowledge:search`
-- `knowledge:list-item-chunks`
+- `knowledge.create_base`
+- `knowledge.restore_base`
+- `knowledge.delete_base`
+- `knowledge.add_items`
+- `knowledge.delete_items`
+- `knowledge.reindex_items`
+- `knowledge.search`
+- `knowledge.list_item_chunks`
 
 These IPC handlers are workflow-oriented. They validate payloads, call data services, and enqueue or execute runtime work internally. (The former `knowledge:delete-item-chunk` entrypoint was removed with the per-base index store cutover — chunks are derived index rows, replaced wholesale by reindexing.)
 
-`KnowledgeService` also owns one orphaned v1 bridge entrypoint, `knowledge-base:delete`. Its only caller was the legacy Redux `store/knowledge` slice, which has now been removed, so this entrypoint is dead and pending cleanup. It routes to the same `delete-base` path.
-
 The chunk IPC entrypoint is a runtime inspection helper:
 
 - `list-item-chunks` rejects failed bases.
@@ -134,11 +131,10 @@ Knowledge runtime work is persisted in JobManager. `KnowledgeService.onInit` reg
 - `knowledge.delete-subtree`
 - `knowledge.reindex-subtree`
 
-Each base uses queue `base.${baseId}`. JobManager owns queue persistence, dispatch, retry, cancellation, timeout, and startup recovery. Knowledge code uses `KnowledgeLockManager` to serialize same-base vector and item mutations inside the current process.
+Each base uses queue `base.${baseId}`. JobManager owns queue persistence, dispatch, retry, cancellation, timeout, and startup recovery. Knowledge code uses the per-base mutation lock (`KeyedMutex.runExclusive`) to serialize same-base vector and item mutations inside the current process.
 
 Current item statuses are:
 
-- `idle`
 - `preparing`
 - `processing`
 - `reading`

docs/references/knowledge/operation-guards.md

@@ -59,7 +59,7 @@ Used only by `reindexItems`.
 - Runs after selected item ids have been collapsed to top-level roots.
 - Loads each selected root subtree with roots included.
 - Allows reindex only when every item in every selected subtree is terminal: `completed` or `failed`.
-- Rejects active or deleting subtree state: `idle`, `preparing`, `processing`, `reading`, `embedding`, or `deleting`.
+- Rejects active or deleting subtree state: `preparing`, `processing`, `reading`, `embedding`, or `deleting`.
 
 This is the backend authority for user-triggered reindex. UI may hide the reindex action for non-terminal rows, but the service guard must still reject stale or direct calls.
 
@@ -262,7 +262,7 @@ The child scheduling compensation mirrors `addItems`: once a child job was accep
 
 ## Shutdown
 
-`KnowledgeService` does not cancel knowledge jobs during service shutdown. Knowledge job handlers use JobManager `recovery: 'retry'`, so unfinished pending, delayed, or running rows are left for JobManager startup recovery instead of being terminal-cancelled while their knowledge items still show active statuses.
+`KnowledgeService` does not cancel knowledge jobs during service shutdown. The indexing handlers (`prepare-root`, `index-documents`, `check-file-processing-result`) and `reindex-subtree` use JobManager `recovery: 'abandon'` — an app restart never silently resumes them, which would otherwise re-spend the paid embedding API. `KnowledgeIngestionService.recoverInterruptedItems()` runs on startup and parks any item left in an active status by an interrupted job as `failed`. Only `delete-subtree` uses `recovery: 'retry'`, so unfinished pending, delayed, or running delete jobs are left for JobManager startup recovery instead of being terminal-cancelled.
 
 ## Review Checklist
 

docs/references/knowledge/workflow-architecture.md

@@ -8,20 +8,20 @@ Knowledge operations are modelled as a lightweight workflow rather than a single
 
 ```text
 API / user action
-  -> KnowledgeWorkflowService
+  -> KnowledgeIngestionService
      -> JobManager
         -> Knowledge job handlers
-           -> KnowledgeLockManager
-              -> SQLite / vector store / FileManager
+           -> KeyedMutex.runExclusive
+              -> SQLite / index store / knowledge-owned files (raw/)
 ```
 
 The design keeps three owners:
 
-- `KnowledgeWorkflowService` decides the next workflow step.
-- `KnowledgeLockManager` serializes same-base mutations and cleanup.
+- `KnowledgeIngestionService` decides the next workflow step.
+- The per-base mutation lock (`KeyedMutex.runExclusive`) serializes same-base mutations and cleanup.
 - Knowledge job handlers execute one durable stage and call the workflow service for the next step.
 
-Helpers may own source planning, lifecycle writes, artifact refs, and FileProcessing adaptation. They should stay as modules until they need lifecycle-managed resources, IPC, timers, or long-lived state.
+Helpers may own source planning, lifecycle writes, knowledge-owned raw files, and FileProcessing adaptation. They should stay as modules until they need lifecycle-managed resources, IPC, timers, or long-lived state.
 
 ## Workflow Entry Points
 
@@ -78,10 +78,10 @@ Round 1 job types:
 
 ## Mutation And Crash Semantics
 
-Same-base Knowledge mutations must go through `KnowledgeLockManager`. Main SQLite writes still use a synchronous write transaction (`DbService.withWriteTx`, or an equivalent `db.transaction()`); the lock manager is not a replacement for that write transaction.
+Same-base Knowledge mutations must go through the per-base mutation lock (`KeyedMutex.runExclusive`). Main SQLite writes still use a synchronous write transaction (`DbService.withWriteTx`, or an equivalent `db.transaction()`); the mutation lock is not a replacement for that write transaction.
 
 Crash safety comes from durable jobs, durable item states, JobManager recovery, and idempotent cleanup. The in-memory mutation lock only serializes concurrent work in the current process.
 
 Delete and reindex span two stores: the main SQLite database and the per-base vector store. They cannot be one cross-store transaction. Consistency relies on durable re-entry and idempotent vector/artifact/row cleanup.
 
-User-triggered reindex is not a cancellation primitive. The service admits reindex only when the entire selected subtree is already `completed` or `failed`. Active states (`idle`, `preparing`, `processing`, `reading`, `embedding`) and `deleting` are rejected; delete remains the operation that can be requested at any time.
+User-triggered reindex is not a cancellation primitive. The service admits reindex only when the entire selected subtree is already `completed` or `failed`. Active states (`preparing`, `processing`, `reading`, `embedding`) and `deleting` are rejected; delete remains the operation that can be requested at any time.

migrations/sqlite-drizzle/0018_easy_gertrude_yorkes.sql

@@ -0,0 +1,39 @@
+PRAGMA foreign_keys=OFF;--> statement-breakpoint
+CREATE TABLE `__new_knowledge_item` (
+	`id` text PRIMARY KEY NOT NULL,
+	`base_id` text NOT NULL,
+	`group_id` text,
+	`type` text NOT NULL,
+	`data` text NOT NULL,
+	`status` text NOT NULL,
+	`error` text,
+	`created_at` integer NOT NULL,
+	`updated_at` integer NOT NULL,
+	FOREIGN KEY (`base_id`) REFERENCES `knowledge_base`(`id`) ON UPDATE no action ON DELETE cascade,
+	FOREIGN KEY (`base_id`,`group_id`) REFERENCES `knowledge_item`(`base_id`,`id`) ON UPDATE no action ON DELETE cascade,
+	CONSTRAINT "knowledge_item_type_check" CHECK("__new_knowledge_item"."type" IN ('file', 'url', 'note', 'directory')),
+	CONSTRAINT "knowledge_item_status_check" CHECK("__new_knowledge_item"."status" IN ('preparing', 'processing', 'reading', 'embedding', 'completed', 'failed', 'deleting')),
+	CONSTRAINT "knowledge_item_type_status_check" CHECK(
+        ("__new_knowledge_item"."type" IN ('file', 'url', 'note') AND "__new_knowledge_item"."status" IN ('processing', 'reading', 'embedding', 'completed', 'failed', 'deleting'))
+        OR ("__new_knowledge_item"."type" = 'directory' AND "__new_knowledge_item"."status" IN ('preparing', 'processing', 'completed', 'failed', 'deleting'))
+      ),
+	CONSTRAINT "knowledge_item_status_error_check" CHECK(
+        (
+          "__new_knowledge_item"."status" IN ('preparing', 'processing', 'reading', 'embedding', 'completed', 'deleting')
+          AND "__new_knowledge_item"."error" IS NULL
+        )
+        OR (
+          "__new_knowledge_item"."status" = 'failed'
+          AND "__new_knowledge_item"."error" IS NOT NULL
+          AND length(trim("__new_knowledge_item"."error")) > 0
+        )
+      )
+);
+--> statement-breakpoint
+INSERT INTO `__new_knowledge_item`("id", "base_id", "group_id", "type", "data", "status", "error", "created_at", "updated_at") SELECT "id", "base_id", "group_id", "type", "data", "status", "error", "created_at", "updated_at" FROM `knowledge_item`;--> statement-breakpoint
+DROP TABLE `knowledge_item`;--> statement-breakpoint
+ALTER TABLE `__new_knowledge_item` RENAME TO `knowledge_item`;--> statement-breakpoint
+PRAGMA foreign_keys=ON;--> statement-breakpoint
+CREATE INDEX `knowledge_item_base_type_created_idx` ON `knowledge_item` (`base_id`,`type`,`created_at`);--> statement-breakpoint
+CREATE INDEX `knowledge_item_base_group_created_idx` ON `knowledge_item` (`base_id`,`group_id`,`created_at`);--> statement-breakpoint
+CREATE UNIQUE INDEX `knowledge_item_baseId_id_unique` ON `knowledge_item` (`base_id`,`id`);