Skip to content

Latest commit

 

History

History
660 lines (362 loc) · 248 KB

File metadata and controls

660 lines (362 loc) · 248 KB

Changelog

All notable changes to this project are documented in this file.

The format is based on Keep a Changelog 1.1.0, and EDEN versioning follows the protocol-level lineage (currently eden-protocol/v0) rather than semver — each entry is a chunk (a self-contained PR or group of waves), dated by its merge into main.

Per-chunk entries preserve the full implementation record: contract amendments, impl details, deferred items, and codex-review provenance. The roadmap at docs/roadmap.md carries one-line status flips; this file is the source of truth for per-chunk completion details.

[Unreleased]

Fix: align spec prose + integrator manifest to the evaluation field name (issue #273)

Resolves the pre-existing drift the #122 entry surfaced (below, "Pre-existing drift surfaced"): the variant's evaluation-payload field is evaluation in variant.schema.json + the eden_contracts.Variant model + the storage/wire impl, but the spec prose spelled it metrics in several chapters and the integrator-manifest table. Two names for one field is a readability + onboarding hazard and a latent parity trap (the schema-parity job only checks schema↔model — both already say evaluation — so the prose metrics naming was unguarded). Option 1 from the issue was selected: align prose + manifest to the schema/model evaluation naming, keeping the wire + on-tree manifest key stable (the reference integrator already emitted evaluation in .eden/variants/<id>/evaluation.json — see _manifest.py — so this is a prose-and-docstring correction, not a manifest-shape change).

Scope was wider than the issue's two named locations. The issue body named 02-data-model.md §9.1 and 06-integrator.md §4.2, but a full audit (per the AGENTS.md "spec inter-chapter restatement is a conflict surface" pitfall — grep chapters 03/04/05/07/08 for the same field) found the same metrics field-name spelling restated across eight chapters. Renaming only the two named spots would have relocated the drift rather than removing it, so the fix renames every backtick'd field reference in lockstep:

  • Spec prose. 02-data-model.md §9.1 variant field table + §9.2 + the evaluated_by description; 03-roles.md §4.2 (evaluator output) + §4.4 (the submission field, the variant-side write rule, the retry-exhausted no-graft rule, and the resubmission-equivalence formula); 04-task-protocol.md §4.2 content-equivalence formula; 05-event-protocol.md §6 (the read-the-entity boundary example); 06-integrator.md §4.1 (validation prose) + §4.2 manifest-shape table; 07-wire-protocol.md §11 reference-helper endpoint (/_reference/.../validate/metricsvalidate/evaluation, which the impl already exposed); 08-storage.md §4.1 + §4.3; 10-checkpoints.md §13 (variant round-trip field list).
  • JSON Schema. variant.schema.json evaluated_by description prose (the field key was already evaluation).
  • Impl docstrings + one error string. submissions.py (EvaluationSubmission docstring), variants.py (declare_variant_evaluation_error docstring), and the validate_acceptance reason string ("success submission requires metrics""… requires evaluation").

Latent conformance bug fixed. conformance/scenarios/test_evaluator_submission.py asserted variant.get("metrics") is None in two evaluation_error scenarios — but the wire variant uses evaluation, so those reads were always None and the assertions passed trivially without testing the no-graft guarantee they claimed. Corrected to variant.get("evaluation") so they actually verify the variant carries no evaluation payload; the surrounding docstrings (which quote the renamed §4.4 prose) were aligned too. A stale checkpoint round-trip fixture (test_checkpoint_roundtrip.py) and a hardening-test reason assertion (test_store_hardening.py) were updated to match.

Deliberately left as-is. The baseline.metrics config block (02-data-model.md §2.7, experiment-config.schema.json, eden_contracts.config.BaselineConfig.metrics) keeps the metrics name — it is a distinct config field that writes into variant.evaluation, and renaming it would touch the config surface (out of scope for Option 1, which keeps the wire/config stable). Plain-English / concept uses of "metric" (metric values §1.3, metric names in the evaluation schema §8, "objective over metrics") are left untouched — a metric is a real domain concept; the field that holds the metrics is evaluation. The generated docs/conformance-coverage.md (a non-CI-enforced snapshot last regenerated at #112) is not regenerated in this PR: it has drifted ~40 keyword lines since #122, so regenerating it now would dump unrelated churn into this focused rename. It will pick up the renamed prose on its next routine regeneration.

Control-plane as a first-class Compose service + lease-handoff smoke (issue #147; re-scoped)

Backfills the Phase 12c CHANGELOG-narrated deferral of a compose-smoke-multi-experiment CI job. Re-scoped during impl (operator-authorized): the draft plan's headline — two experiments end-to-end with cross-experiment isolation asserted via wire reads — is not buildable on the reference impl, because it hosts exactly one experiment per deployment. Three sites enforce this: the task-store-server's Store binds a single experiment_id and the wire layer rejects any other (ExperimentIdMismatch at _dependencies.py:73); the orchestrator multi-experiment loop targets one task-store URL for all experiments (multi_loop.py make_runtime_factory); and the integrator is one shared bare repo deployment-wide (cli.py _build_runtime_factory). 12c's multi-experiment surface was validated only against fake stores + the single-IUT conformance binding. True multi-experiment hosting + the cross-experiment-isolation smoke are deferred to #254 (filed at re-scope time). This chunk ships the genuinely-new, genuinely-shippable substrate piece instead: the control plane as a first-class Compose service, plus a lease-lifecycle + lease-handoff chaos smoke.

Control-plane Compose service. compose.yaml gains an always-on control-plane service (Postgres-backed; chapter 11 §3.4 Option A — a separate eden_control_plane database in the same instance, created by the new init-control-plane-db.sh postgres init hook). A /healthz endpoint was added to the control-plane server (app.py; unauthenticated, outside /v0/control) for the container healthcheck. The service is always-on but opt-in: the orchestrator and web-ui only talk to it when EDEN_CONTROL_PLANE_URL is non-empty, so the existing six Compose smokes are unchanged in posture.

Env-fallback instead of entrypoint wrappers. Rather than the draft plan's bash wrapper scripts, the orchestrator + web-ui CLIs gained an EDEN_CONTROL_PLANE_URL env fallback for --control-plane-url (empty treated as unset), mirroring the existing EDEN_CONTROL_PLANE_ADMIN_TOKEN fallback. The orchestrator selects mode solely on --control-plane-url being set, and --experiment-id is harmless in lease mode (a logging label), so a single compose service definition flips between single- and multi-experiment mode purely via the env var — no wrapper scripts, no Dockerfile change. compose.control-plane.yaml (whose only content was web-ui flag-passing) is deleted; docs/observability.md §3.4 is rewritten to the first-class-service + env-toggle flow.

Lease-handoff smoke. New compose.multi-experiment.yaml overlay (a second orchestrator-2 replica in lease mode) + smoke-multi-experiment.sh + the compose-smoke-multi-experiment CI job (unrequired initially; bump to required-status after ~2 weeks clean on main). The smoke brings up the control plane + two lease-contending replicas against one registered experiment, asserts the lease-singleton invariant, kills the lease holder and asserts clean hand-off to the standby, has the surviving replica drive the full pipeline to ≥2 variant.integrated, then issues an operator-driven terminate_experiment and asserts the control-plane last_known_state converges to terminated. Validated locally end-to-end (PASS).

Two pre-existing gaps surfaced by the smoke (filed, not fixed here). (1) In lease-driven mode the orchestrator joins only the control-plane orchestrators group, not the task-store one (single-experiment mode self-joins via _ensure_orchestrators_membership; the multi-experiment path does not), so without seeding, the lease holder's §3.7-gated dispatch/integrate calls 403 — the smoke seeds the task-store group as a workaround; folded into #254. (2) The orchestrator's auto-termination decision (dispatch_mode.termination = "auto") 403s under wire auth because terminate_experiment is admins-gated while the orchestrator is in orchestrators (a spec inter-chapter drift between 03 §6.2 and 07 §2.9 / 04 §8.2, never caught because existing smokes use never_terminate + quiescence-exit and dispatch tests run auth-disabled) — filed as #256. The smoke uses the supported operator-driven termination path instead.

setup-experiment. Emits the control-plane store DSN (EDEN_CONTROL_PLANE_STORE_URL, POSTGRES_DB_CONTROL_PLANE) + EDEN_CONTROL_PLANE_URL= (empty) and creates the logs/control-plane substrate dir. No --register-additional-experiment flag (that was the two-experiment path; deferred to #254). Closes #147.

Gate terminate_experiment on (admins, orchestrators) (issue #256)

Fixes a spec inter-chapter drift that made the orchestrator's decision-type 0 (auto-termination) unexecutable under the normative wire-auth model. 03-roles.md §6.2 makes termination an orchestrator decision (the orchestrator commits the running → terminated transition), but the wire op terminate_experiment was gated on the admins group only (07-wire-protocol.md §2.9, 04-task-protocol.md §8.2) — and the orchestrator is a member of orchestrators, not admins. An auto-terminating deployment (dispatch_mode.termination == "auto" with a policy returning Terminate(...)) therefore got a 403 the moment the policy fired. Not multi-experiment-specific — a single-experiment deployment hits it too (single-experiment mode joins orchestrators). Surfaced first by the #147 control-plane + lease smoke against the auth-enabled stack; never caught earlier because every Compose smoke/e2e uses never_terminate + quiescence-exit, and the dispatch driver unit tests run against auth-disabled stores where the group gate short-circuits.

Option 1 of the issue selected: gate on admins OR orchestrators, mirroring how accept / reject / emit_policy_error are already orchestrators-gated for the same 03-roles.md §6 rationale. This keeps the orchestrator out of admins (which would over-grant it reassign_task / update_dispatch_mode authority). The operator-driven path still uses an admins bearer; the two paths race-resolve to a single observable event per §6.4.1, unchanged.

Spec. 07-wire-protocol.md §2.9 (the §13.3 auth-column table row, the group-gating bullet, and the Authority prose) and 04-task-protocol.md §8.2 now state the gate is admins OR orchestrators with the two-paths rationale; 03-roles.md §6.5 manual-termination bullet corrected from "admin-gated" to the OR-gate; 09-conformance.md §5 "Experiment lifecycle" scope documents the now-asserted authority gate. No schema / Pydantic change (authority is a wire-gate concern, not a payload shape).

Impl. reference/packages/eden-wire/src/eden_wire/routers/experiment_lifecycle.py _terminate_experiment now calls enforce_in_any_group(deps, request, ("admins", "orchestrators")). No client / dispatch-driver change — the orchestrator's StoreClient.terminate_experiment already authenticates as its own (orchestrators-group) bearer.

Tests. Wire-package test_lifecycle_wire.py: the prior test_orchestrators_member_not_sufficient (asserted 403) is replaced by test_orchestrators_member_can_terminate (asserts 200 + terminated + terminated_by stamping); the negative test renamed to test_non_admin_non_orchestrator_rejected_with_forbidden and the literal-admin-principal rejection (test_admin_bearer_rejected) is unchanged. Two new conformance scenarios in test_experiment_lifecycle.py (group "Experiment lifecycle", citing 04-task-protocol.md §8.2): test_orchestrator_group_bearer_can_terminate (orchestrators-only bearer → 200 + terminated) and test_worker_outside_both_groups_cannot_terminate (neither-group bearer → 403 forbidden, state stays running). The pre-existing test_forbidden_vocabulary.py terminate probe (which uses a neither-group test-worker) was renamed test_terminate_by_neither_group_returns_403_forbidden and its docstring corrected from "gated on the admins group" to the OR-gate.

Doc-consistency sweep. Updated the now-stale "admin-gated" restatements that the spec change exposed: docs/glossary.md's terminate entry (canonical vocabulary), and the StoreClient.terminate_experiment / _seed.terminate_experiment helper docstrings.

Evaluatable baseline variant (issue #122)

Elevates the experiment seed — the single commit on main at experiment start — to a first-class kind == "baseline" variant so operators have a "what did the seed score?" comparison anchor and the lineage tree has a colored root. Default-on (suppressed by baseline.enabled: false). Spans spec + schemas + contracts + storage + wire + dispatch + orchestrator (both modes) + web-ui.

Spec. New Variant.kind field (02-data-model.md §9.1) + a normative §9.4 "Baseline variants": at-most-one per experiment, idea_id MAY be absent, parent_commits == [base_commit_sha] (the seed framed as its own parent — the no-op case, permitted for baselines), never integrated, reaches terminal success by ordinary evaluation (default) or a config-supplied-metrics override. New §2.7 baseline config block. Carves: §2.4 integration-decision predicate and §2.5 termination-drain rule both exclude kind != "baseline" (the load-bearing carve — without it a default-on baseline blocks termination forever); §10 invariant 2 (a baseline references no idea); §3.3 (03-roles.md) + §4.2 (04-task-protocol.md) exempt baselines from the no-op prohibition (scoped to executor submissions); §1.7 (08-storage.md) relaxes the create_variant precondition (a baseline MAY be created directly in success with validated metrics) and pins orchestrators-group create authority; §3.3 (05-event-protocol.md) makes variant.started's idea_id optional + kind required-for-baselines and documents the override-path started+succeeded composite; §2 (06-integrator.md) MUST-NOT-integrate-baseline; §4/§5 (07-wire-protocol.md) per-kind create authority + integrate_variant baseline rejection. Also adds experiment.base_commit_sha to the runtime object (§2.5), surfaced on read_experiment (§14.3) and round-tripped through portable checkpoints (10-checkpoints.md §5). New chapter-9 §5 conformance group "Baseline variant" (v1+roles+integrator). All three schemas updated (variantkind enum + conditional-required idea_id via allOf-if-then; experiment-configbaseline block with the enabled:false+metrics conflict gate; experimentbase_commit_sha).

Contracts + parity. Variant.kind + conditional idea_id (model_validator); BaselineConfig + ExperimentConfig.baseline; Experiment.base_commit_sha; variant.started event payload idea_id-optional + kind. The experiment runtime object joins the schema-parity corpus (MODEL_NAMES / _MODEL_VALIDATORS / round-trip dumper) so base_commit_sha drift is caught. Conditional-required idea_id got the densest fixture coverage (the riskiest JSON-Schema if/then vs Pydantic-validator parity surface).

Storage / wire / dispatch / orchestrator. create_variant accepts a baseline directly in success (validates evaluation against evaluation_schema at create time, emits variant.started+variant.succeeded atomically); integrate_variant rejects a baseline. base_commit_sha persisted via a v7 column migration on both sqlite + postgres backends (and the in-memory store), threaded build_store → backend constructors from a new task-store-server --base-commit-sha flag (defaulting to $EDEN_BASE_COMMIT_SHA, wired in compose.yaml), and round-tripped on checkpoint import via a new _Tx.base_commit_sha_update staging field. read_experiment surfaces it (omitted-when-absent; imported_from stays explicit-null). _integrate_successful_variants skips baselines; state_view excludes baselines from running_variant_count + attempted_variant_count (so baselines are invisible to the parallel_variants and max_variants policies — they remain invisible to target/convergence policies too, which read only integration-gated counts); baseline evaluation tasks are untargeted (no idea_id lookup). New ensure_baseline_variant helper (idempotent by verified read-back — kind + seed-SHA check, else fail loudly) runs at single-mode startup (cli.py, after orchestrators-group join so the bearer satisfies the per-kind create authority) and per-experiment in multi_loop.py; _experiment_is_drained_terminated excludes baselines. Multi-experiment mode applies the bootstrap config's baseline block to every driven experiment (per-experiment config resolution remains deferred to #214). Per-kind wire authority enforced in routers/variants.py.

Web-UI null-safety. Every variant.idea_id consumer made baseline-safe: _lineage.py renders a baseline as a root node; admin/observability.py (variant detail + per-idea counts) and the manual-UI evaluator.py draft path tolerate a None idea (template panels show a "baseline (seed)" note); the integrator carries a defensive kind == "baseline" reject behind the dispatch skip.

Tests. New test_baseline_variant.py (storage create/integrate/metrics-validation), test_baseline_counts.py (dispatch count carves), test_baseline.py (orchestrator ensure_baseline_variant + the §8.1 drain-with-baseline deadlock guard), a checkpoint base_commit_sha round-trip test, contract/event/experiment-config fixtures, and 5 wire-observable conformance scenarios (kind round-trip, not-integratable, override-metrics validation, conditional idea_id, per-kind authority). The exact-count orchestrator e2e tests (test_e2e.py / test_subprocess_e2e.py) deliberately run their task-store-server without --base-commit-sha (no baseline created — the legacy/skip path); the default-on baseline path end-to-end is covered by the Compose smokes (which set --base-commit-sha) and the deterministic test_baseline.py.

Pre-existing drift surfaced (tracked). The variant evaluation-payload field is evaluation in the schema + Pydantic model but metrics in the 02-data-model.md §9.1 prose + the integrator manifest. This predates issue #122 and was left untouched (out of scope); filed as #273.

Per-route store swapping for the experiment switcher (issue #145)

Closes the Phase 12c §3.6 deferral: the cross-experiment switcher shipped in 12c (the /admin/experiments/ dashboard, the Session.selected_experiment_id cookie field, and POST /admin/experiments/{E}/select) recorded the operator's selection but every per-experiment route still read the startup-bound app.state.store / experiment_id / experiment_config, so "select experiment Y" only relabelled the page. This chunk makes the selection load-bearing: every per-experiment web-ui route now resolves the active experiment per-request and operates against its store / config / repo. Reference-impl web-ui only — no spec / wire / JSON-schema / Pydantic / conformance change (Decision 10/11; per-route swapping is web-ui behavior with no observable signal at the chapter-9 §6 IUT contract).

Active-experiment resolution. A new resolve_active_context(request) helper (routes/_helpers.py) is the single entry point each handler calls after its session guard: it reads Session.selected_experiment_id, falls back to the deployment default (--experiment-id), and returns either a ready ActiveContext (resolved experiment_id + per-experiment store / admin_store / config) or a Response the handler returns verbatim. With no control plane configured it always returns the deployment default with zero validation overhead — single-experiment deployments are observably unchanged. For a non-default selection in control-plane mode it (1) validates existence in the control plane (StaleSelection → dashboard redirect + cleared session field otherwise) and (2) classifies seeded vs registered-but-unseeded against the task-store-server (Decision 8's three-state model: an experiment registered on the dashboard but not yet bootstrapped by setup-experiment / checkpoint-import renders an "initialize me" page rather than being mis-classified as stale, Risk 11). experiment_id moved from a render-time Jinja global to a per-request template context processor so every page reflects the active experiment.

Per-experiment store vending. store_factory.py: the live StoreFactory vends per-(experiment_id, role) StoreClient views against the one deployment-wide task-store URL (12c Decision 11 — no service discovery; only the experiment_id path segment varies), sharing one httpx.Client so connection-pooling is preserved. Worker-role views are JIT-credentialed on first access by BearerCache, which reuses eden_service_common.auth.bootstrap_worker_credential verbatim (the per-worker_id lock + idempotent-register-then-reissue + persisted-token /whoami verify) — never reimplementing those disciplines — under a per-experiment credential subtree <credential-dir>/<experiment_id>/<worker_id>.token. The auth-disabled posture (no admin token, no persisted credential) returns a None bearer, mirroring the prior resolve_worker_bearer posture 3. StaticStoreFactory vends one pre-built store for the single-experiment / test path. make_app now takes store_factory as its sole store dependency — the legacy store= / admin_store= kwargs and the app.state.store / admin_store attributes are gone (tests construct via the conftest._one_experiment_factory helper).

Credential plumbing (Posture B/C/D, §3.2). credentials.py adds a deployment-scoped control-plane worker credential (bootstrap_control_plane_credential, persisted at <credential-dir>/control-plane/<worker-id>.token) so the switcher's control-plane reads (list_experiments / read_experiment_metadata, which accept any authenticated principal) keep working after the operator rotates the admin token out of the runtime env (Posture C). resolve_credential_dir resolves --credential-dir / $EDEN_CREDENTIAL_DIR → the common --credentials-dir / $EDEN_WORKER_CREDENTIALS_DIR → an XDG default, so the web-ui (itself a worker host) shares the established credentials volume by default. New CLI flags: --credential-dir, --experiment-config-dir, --control-plane-worker-id.

Per-experiment config + repo. Each experiment's ExperimentConfig is loaded lazily from <--experiment-config-dir>/<experiment_id>.yaml (Decision 6; the deployment default still uses --experiment-config); setup-experiment.sh drops each experiment's YAML there. The executor module's local integrator clone is per-experiment via repo_factory.py's RepoMaterializer (clones <repo-path-parent>/<experiment_id>.git from the substituted --forgejo-url org base, fetch-on-access); repo_for returns the startup clone for the default experiment.

Switcher + safety. base.html gains a no-JS top-nav switcher dropdown (CSRF-protected select POSTs; a 5s in-process list_experiments cache, §3.7; hidden without a control plane). Every worker submit form carries a hidden form_experiment_id; form_experiment_guard discards a submission whose form was rendered against a different experiment than the now-active one and redirects with a clear banner rather than writing to the wrong experiment (§3.6). The dashboard renders the resolution-failure banners (stale-selection / control-plane-unreachable / cannot-bootstrap-credential / task-store-unreachable / config-missing / config-invalid / switched-mid-form). The AdminGateMiddleware admins-group check now follows the active experiment (deployment-scoped /admin/experiments + /admin/control pages gate against the default and are exempt from per-experiment resolution — they are the redirect target, so resolving them per-experiment would loop).

Compose. The web-ui service gains --experiment-config-dir /var/lib/eden/web-ui-configs (+ bind-mount) and an explicit --credentials-dir /var/lib/eden/credentials (the new resolver otherwise falls back to a non-persisted in-container XDG path); setup-experiment.sh creates the web-ui-configs/ dir and copies each experiment's config in.

Tests. New test_store_factory.py, test_resolve_active.py, test_per_experiment_repo.py; test_admin_experiments_routes.py extended (switcher render/highlight, resolution-failure banners, no-control-plane switcher absence). Full web-ui suite green (667), including the real-subprocess e2e tests; ruff + pyright + complexity-gate clean.

Deferred (tracked):

  • GET /v0/experiments/{E}/config wire endpoint (Decision 6 alternative) — the cleaner long-term shape that removes the on-disk config-dir and its drift risk (Risk 12); a normative chapter-7 amendment, out of scope here. Filed as #259.
  • Per-request active-experiment resolution cache (Decision 8 5s TTL) — only the switcher's list_experiments cache shipped; the per-request seeded/unseeded classification is uncached (correct, but a latency cost on non-default admin pages). Filed as #260.
  • Tab-scoped ?exp= permalink override + draft-survives-switch (§7.5 / §7.2, v1 affordances). Filed as #261.
  • form_experiment_id guard on admin mutating forms — shipped on the worker submit forms; admin forms fail safe (cross-experiment id → NotFound) but lack the explicit guard banner. Filed as #262.
  • Multi-experiment Compose smoke remains the existing #147; the single-experiment smoke is the golden path and stays green through this chunk. The #128 / #140 / #141 retrofits are unchanged (§4.2).

Move deployment CLI flags into experiment-config fields (issue #157)

Audits the orchestrator / worker-host CLI surface (issue #157) and moves five flags whose values two experiments sharing one deployment plausibly want different values for, from deployment-wide CLI flags / env vars into typed experiment-config.yaml fields — validated on both the JSON Schema and the Pydantic eden-contracts side per the repo's schema↔model parity discipline. Mirrors the #133 ideation_policy template (discriminated-union YAML block + build_* factory + flag removal).

Five fields moved into the config (5 of the 8 audited):

  • termination_policy — declarative discriminated-union block (kind: never_terminate / max_variants / max_wall_time / convergence_window / target_condition + per-kind params), cousin of ideation_policy. Replaces the orchestrator's --termination-policy <module:callable> flag + EDEN_TERMINATION_POLICY env var for the single-experiment path. Required when dispatch_mode.termination == "auto" (cross-field rule enforced on BOTH schema (allOf-if-then) and Pydantic (@model_validator(mode="after")) sides; the four parity cases — auto×{present,absent}, manual×{present,absent} — gate the lockstep).
  • max_quiescent_iterations (integer ≥ 2) — the orchestrator quiescent-exit budget. Replaces --max-quiescent-iterations / EDEN_MAX_QUIESCENT_ITERATIONS for the single-experiment path.
  • ideation_task_deadline / execution_task_deadline / evaluation_task_deadline (number > 0; defaults 120 / 600 / 300 s) — per-task worker-host SLA on a single *_command invocation. Replace the three --*-task-deadline worker-host flags. These travel with the *_command they bound (already in the experiment config under extra="allow").

Per-flag disposition. --ideas-per-ideation integration is deferred (#246) — the default Compose ideator runs in scripted mode, which doesn't load an experiment-config, so integration needs a deeper refactor. --lease-duration-seconds and --claim-ttl-seconds stay as deployment flags (the lease is the orchestrator-replica↔control-plane contract per chapter 11 §4.3; claim-ttl is a web-UI manual-claim UX preference) — neither is experiment-shaping.

Schema + Pydantic. Five new optional top-level fields in experiment-config.schema.json (termination_policy.duration uses format: "duration"). New DurationStr validated-string alias in _common.py (modeled on DateTimeStr/UriStr: a string validated by TypeAdapter(timedelta).validate_python + a positivity check — a timedelta field on a ConfigDict(strict=True) model rejects ISO-8601 strings, so the string is the storage shape and the factory re-parses at the boundary). Matching duration FormatChecker handler in tests/conftest.py reuses the same _check_duration validator so both sides accept/reject identical strings (test_format_coverage gates the handler's existence). TerminationPolicyConfig discriminated union + four scalar fields + cross-field validator added to config.py; 30 new accept/reject fixtures in cases.py (per-kind required-params, min/max, duration format, the four cross-field cases).

Factory. New build_termination_policy(config) in eden-dispatch/termination.py maps the declarative block to a TerminationPolicy callable (cousin of build_policy). The deprecated env_max_variants_policy() factory + the EDEN_TERMINATION_MAX_VARIANTS env-var bridge are removed — their purpose was bridging pre-12a-3 field-removal to the callable shape; this chunk restores the config-field shape under a new name. A downstream fork that used env_max_variants_policy migrates to termination_policy: {kind: max_variants, target: N}.

Service wiring. The single-experiment orchestrator branch reads config.termination_policy (→ build_termination_policy) and config.max_quiescent_iterations; these supersede the CLI flags. The flags stay registered on the argparser because multi-experiment mode still consults them (per-experiment config resolution through the chapter-11 registry is deferred to #214); a non-default CLI value in single-experiment mode triggers a orchestrator_cli_flag_ignored startup WARN (not a silent no-op). The three worker hosts drop their --*-task-deadline flags (clean break — argparse errors on the retired flag) and read the deadline from config in subprocess mode.

Compose + smokes. compose.yaml + compose.multi-orchestrator.yaml drop the single-experiment orchestrator's --termination-policy / --max-quiescent-iterations lines + the EDEN_TERMINATION_MAX_VARIANTS env; .env.example drops EDEN_TERMINATION_POLICY / EDEN_TERMINATION_MAX_VARIANTS / EDEN_MAX_QUIESCENT_ITERATIONS. Each smoke script's experiment-config-YAML append loop gains max_quiescent_iterations: 30 (reproducing the retired compose-level default). The orchestrator/web-ui subprocess e2e tests move their max_quiescent_iterations from the CLI invocation into the config YAML.

Spec + docs. 02-data-model.md §2.4 gains prose for termination_policy (incl. the cross-field MUST) + the four scalars; §2.5 + 03-roles.md §6.2 decision-type 0 reword "deployment-supplied termination policy callable" → "experiment-config-supplied termination_policy". docs/user-guide.md §2 table + worked example; docs/operations/experiment-lifecycle.md rewrites the policy-driven recipes from module:callable to the YAML block; docs/glossary.md §8 adds termination_policy / task deadline entries + updates quiescence.

Conformance. Unchanged — all five fields are implementation-defined per chapter 03 §6 (orchestrator decision-type contracts are normative; the policies that drive them are not; worker-host SLA deadlines are not wire surface). No new §5 group; no new scenario citations.

Deferred (tracked): --ideas-per-ideation integration (#246); symmetric build_policy()build_ideation_policy() rename (#247); per-experiment config resolution in multi-experiment mode (#214). Plan: docs/plans/issue-157-cli-flags-to-config.md. Closes #157.

Hierarchical artifacts substrate — by entity, not flat (issue #168)

Replaces the flat artifact substrate (every idea / variant artifact dumped as artifacts/<entity_id>.<ext> with no grouping) with an entity-hierarchical layout grouped by the durable entity that owns the artifact and the role that produced it. Discovered during the 2026-05-22 manual demo; the operator's intuition was "subdirectories would group naturally." Plan: docs/plans/issue-168-hierarchical-artifacts-substrate.md.

artifacts/
  ideas/<idea_id>/                    # ideator-produced
    content.md | <upload> | bundle.tar.gz
  variants/<variant_id>/
    executor/  exec-<uuid>.{md,<ext>,tar.gz}
    evaluator/ eval-<uuid>.{md,<ext>,tar.gz}

Single shared path-builder + bundle writer. The artifact path-builder and bundle writer moved from eden_web_ui.artifacts into eden_service_common.artifacts — the right dependency direction, since the ideator subprocess host (a worker-host package that must not depend on the web-UI) also needs it. eden_web_ui.artifacts is now a thin re-export so existing from ..artifacts import … call sites are unchanged. The interface was reworked from "(artifacts_dir, artifact_id) and the helper picks the basename" to (target_dir, ArtifactNaming) — separating where (entity_artifact_dir(producer, entity_id)) from what name (the per-role filename policy). predict_artifact_uri and write_artifact_bundle share one name-derivation path so they cannot drift.

Filename policy. The ideator's ideas/<idea_id>/ is write-once, so its leaf files take clean fixed names (content.md, the upload's own name, bundle.tar.gz). The executor/ / evaluator/ dirs are keyed only by the stable variant_id and accumulate across resubmissions, so each submission mints a fresh exec-<uuid> / eval-<uuid> stem — no two submissions for one variant target the same path. As belt-and-suspenders the writer materializes via an exclusive os.link that raises FileExistsError rather than clobbering, making the chapter-8 §5.4 no-overwrite guarantee independent of caller id-discipline.

Text-filename unification (content.md). The ideator text entry — both the single-.md on-disk name and the bundle-internal headline — is unified on content.md (the subprocess host + binding §2.3 value), resolving a pre-existing three-way drift (idea.md in the web-UI write path, content.md in predict_artifact_uri's collision check, IDEA_BUNDLE_HEADLINE = "idea.md" in the inline-render reader). The predict/write/read lockstep is now consistent. The evaluator (evaluation.md) and executor (variant.md) bundle headlines are unchanged (role-coherent).

Writers routed through the shared helper: the web-UI ideator + evaluator + executor routes, the ideator subprocess host's _write_content, and the standalone eden-manual CLI (which hand-mirrors the helper because it ships as a system-python3 script and can't import workspace packages). The CLI's write-side URI stamp was the subtle fix — it stamped basename-only (file://…/artifacts/<name>), correct only for the flat layout; it now stamps the full nested relative path (file://…/artifacts/ideas/<idea_id>/content.md). The read-side _translate_artifacts_uri_to_host was already nested-safe.

Scope deviation from the plan (flagged). The plan (written before #212 merged) assumed no executor artifact-byte writer existed and treated the executor/ subdir as forward-looking only. But #212 since landed _maybe_bundle_executor_artifact, a real web-UI executor byte-writer that wrote flat. Leaving it flat would have split variant artifacts (evaluator hierarchical, executor flat) and defeated the chunk's consistency goal, so the executor writer was routed through the shared helper into variants/<variant_id>/executor/ with the exec-<uuid> stem the plan's §D.2 already specified. This is the only deviation; it is fully consistent with the plan's design.

Self-describing substrate — no migration, no compat shim. Per CLAUDE.md's no-backwards-compat-shims posture and the plan §1.2: artifacts_uri stores the full URI, the web-UI serve route resolves any path under the artifacts-dir jail, and the admin listing os.walks recursively — all layout-agnostic. Pre-existing flat artifacts keep resolving untouched; only new writes go hierarchical. No migrate-artifacts.sh, no dual-read path. A one-shot migration recipe is documented in the plan §8 should a future external deployment ever need it (not filed as an issue — EDEN is pre-external-deployment-base and the substrate is self-describing).

Spec (reference-binding only). Physical layout is non-normative: 08-storage.md §5.1 keeps the artifact-store naming "implementation-defined" and 02-data-model.md §1.5 keeps artifacts_uri opaque. So the only spec surface touched is the reference binding: worker-host-subprocess.md gains §10 documenting the full reference artifact layout (and §2.3 forward-references it); 08-storage.md §5.1 gains an informative note pointing at it; docs/glossary.md §5 gains the artifact layout term. No JSON-schema, Pydantic-model, or wire-binding field changes. Conformance impact: zero — physical layout is not wire-observable, so no scenario / citation change. The opaque artifact URIs the evaluator conformance scenarios round-trip are deliberately left flat (coupling the suite to a reference-impl path shape would be the anti-pattern).

Tests. New test_artifacts.py (29 cases) covers the three (producer, entity_id) → dir mappings, the full per-branch filename policy, predict==write, and the FileExistsError no-overwrite guarantee. New test_eden_manual_artifacts.py loads the standalone CLI via SourceFileLoader and asserts the §D.4 full-nested-path write-side URI stamp, the CLI/helper layout parity, and CLI second-write FileExistsError. New test_whitespace_only_content_errors_not_stuck + a ideas/<idea_id>/content.md path-shape assertion in test_ideator_subprocess.py. The web-UI test_artifact_bundle.py was trimmed to the web-UI-specific serving-security tests (the writer coverage moved to _common); ideator/admin fixtures and test_ideator_multifile.py headline assertions updated to the hierarchical paths. Scripted-mode fictional URIs reshaped to the layout for legible admin listings (cosmetic — scripted mode never writes bytes).

Impl-stage codex-review record (2 rounds to convergence) at docs/plans/review/issue-168/impl/. Round 0 caught a real bug (whitespace-only ideator content would raise an uncaught ValueError through the shared writer, leaving the task stuck claimed — fixed by gating _write_content on content.strip()), plus restoring the write_idea_artifact public name on the shim and porting the no-overwrite exclusive-create into the CLI mirror; round 1 confirmed all three resolved with no new findings.

Redesign pending-task lists — executor + evaluator (issue #137)

Replaces the low-signal pending-task tables on the executor and evaluator list pages (task_id prominent, slug buried in small text, no sort / filter / grouping, an inline <details> content-preview per row, effectively-random ORDER BY task_id ordering) with a high-signal 5-column grid built for the operator's real decision — which pending task to claim. Web-ui-module-only: no spec, JSON-schema, Pydantic, wire-binding, or store-query change (verified, not assumed — §5 of the plan). Plan + impl-stage codex-review record at docs/plans/issue-137-pending-task-list-redesign.md + docs/plans/review/issue-137/.

Column shape (both lists, identical). Default collapsed row: slug (prominent, sortable) / priority (sortable, default sort key DESC) / target (any / worker:<id> / group:<id>) / created by (filterable + groupable) / claim (eligibility-aware button). The evaluator's variant-under-evaluation moves into the expansion (not a top-level column) so the two tables are visually identical.

Sort. Default (priority DESC, created_at ASC) — highest-priority first, oldest-first tiebreak. Sortable columns are slug and priority only (header links set ?sort=&dir=; clicking the active column flips direction). created_at is the always-on implicit secondary tiebreak, not a user-selectable axis. sort/dir are allow-listed (priority/slug, asc/desc); unknown/absent → default, so no raw ?sort= reflects into a link href (closes the query-param-injection surface). Degraded rows (idea/variant unreadable) sink to the bottom under both directions via partitioning (present rows sorted, degraded concatenated last in stable order) — not a -inf/"" sentinel, which would float them to the top under ascending sorts.

Filter + group (all URL-query-param driven — stateless, shareable; decision §2.2 chose query params over per-navigation session-cookie writes, deferred to #248). "Eligible for me" (?eligible=1, default ON) mirrors the full 04-task-protocol.md §3.5 claim ladder — registration first, then target — as an advisory projection (the claim write remains the enforcement point; the UI does not re-specify §3.5). Registration is resolved once per render (read_worker) with its own 3-outcome ladder; group membership is memoized by group_id (null/worker targets cost zero wire calls). Target tri-state (?target=all|targeted|untargeted). Group-by-creator (?group=1) wraps rows in a <details> per idea.created_by.

Eligibility-aware claim + two warning counters. Eligible rows get a live claim form; definitively-ineligible rows (filter OFF) get a disabled button + tooltip naming the target. Transport-indeterminate eligibility (a read_worker or resolve_worker_in_group transport/auth failure — not NotFound, which is a legitimate False) renders the row disabled + an "eligibility unknown" note and stays visible even under the eligible-only filter, so a transient outage never silently hides claimable work (the AGENTS.md narrow-exception-handling pitfall). The existing content-read warning counter is kept distinct from a new eligibility-resolution counter — folding them would let the copy lie ("idea read failed" when the idea read fine but the eligibility probe timed out).

Expansion replaces the inline preview. The toggle reveals a navigation-only context-links row (no inline content): task / creator / idea admin-detail links, the variant link + work-branch text (evaluator only), and artifact-shape-aware "view content" links covering both serve_artifact shapes — one ?uri=&entry= link per manifest entry for .tar.gz bundles, a single ?uri= link for direct single-file file:// artifacts (no hardcoded idea.md), and a direct external link for http(s):// URIs.

Server-side prong was already satisfied (the #134 StorageError-broadening already routes WorkerNotEligible/WorkerNotRegistered through wire_error_banner); this issue adds a forged/direct-POST regression test asserting an ineligible claim returns a 303 banner-redirect, not a 500 — no handler code change.

Impl. Shared helpers in routes/_helpers.py: parse_list_view (allow-listed ListView), build_list_links (all hrefs built from validated values), EligibilityResolver (per-render registration + group memoization, 3-outcome discipline), build_artifact_links, and arrange_pending_rows (filter → sort → optional group). Both row-builders in executor.py / evaluator.py attach eligible / eligibility_unknown / clean slug/priority/created_by fields. Templates share a _pending_list.html macro partial (one pending_section macro, role-parameterized) so executor/evaluator can't drift. New chip / disabled-button / expand-affordance / banner-warn CSS in static/style.css.

Tests. New test_executor_list.py / test_evaluator_list.py (rewritten from the retired inline-preview tests) cover columns, priority-default + slug sort, sort-param allow-listing (XSS-reflection guard), the eligibility / target / group filters, the disabled-button + tooltip, the expansion context links, degraded-row bottom-placement under both directions, and the full registration ladder (registered / NotFound / transport-unknown / group-probe-unknown). Forged-POST ineligible-claim regression added to test_executor_routes.py / test_evaluator_routes.py. Full pytest -q + conformance pass as regression gates (conformance does not exercise the web-ui list — no scenario change).

Deferred (tracked): cookie-backed cross-visit persistence + saved presets + pagination + ideator-list parity → #248; store-side priority ordering (postgres.py / sqlite.py / Store contract) → #249; browser-facing Forgejo browse-URL contract to make commit/branch refs clickable → #250. The created_by column ships worker_id now; the open id/name disambiguation work (#128) will later evolve the display — the two are independent and neither blocks the other. Closes #137.

In-stack log search UI: Loki + Alloy + Grafana overlay (issue #110)

Adds an opt-in Compose overlay (reference/compose/compose.logging.yaml) that ships Loki (log store + LogQL), Grafana Alloy (log shipper), and Grafana (search UI, pre-provisioned) so operators can search EDEN's logs across services + time windows from one UI instead of a multi-window docker compose logs session. Fills rung L3 of the log-persistence ladder in docs/observability.md §2.5. Pure deployment-binding addition: no EDEN service code, no spec / wire / schema / Pydantic / conformance changes.

Two facts reshaped the issue's design. (1) #109 has shipped — every long-running service already writes a structured JSON-line file to ${EDEN_EXPERIMENT_DATA_ROOT}/logs/<service>/<service>.jsonl. So the collector tails that bind-mount read-only and does not need the docker socket the issue's docker_sd_configs sketch implied (strictly lower privilege for the common case). (2) Promtail reached EOL 2026-03-02; the issue named Promtail, but Grafana's supported successor is Grafana Alloy. This chunk ships Alloy — the pipeline is identical (tail JSONL → ship to Loki → search in Grafana), only the collector binary + config language changed.

What shipped.

  • compose.logging.yaml — three services. alloy tails ${EDEN_EXPERIMENT_DATA_ROOT}/logs/**/*.jsonl read-only and ships to Loki, lifting service/level/experiment_id to labels (parsing the implicit eden_service_common/logging.py field contract); loki stores + indexes (internal-only, no host port); grafana is the only host-exposed surface (localhost:${GRAFANA_HOST_PORT:-3000}). Composes cleanly on top of any other overlay set (adds new services, edits none). Image tags pinned + validated against a live bring-up: grafana/loki:3.7.2, grafana/alloy:v1.16.1, grafana/grafana:12.4.3 (the plan sketched Grafana 11.x; 11 is EOL, so the then-current stable line was chosen per the plan's "exact tags at impl time" instruction). Outbound phone-home disabled on all three (Loki analytics.reporting_enabled: false, Alloy --disable-reporting, Grafana GF_ANALYTICS_*=false). Healthcheck note: grafana/loki:3.x is distroless (no shell) and grafana/alloy ships no HTTP client, so neither can carry a CMD HTTP healthcheck; only Grafana (which has curl) gets one, and loki+alloy use plain service_started ordering — the smoke validates Loki readiness + ingestion by exec-ing curl through the Grafana container (which reaches loki:3100 on the network).
  • logging/ config treeloki-config.yaml (single-binary, filesystem store, tsdb, finite 7-day retention_period + compactor retention so a long-lived demo's disk stays bounded, auth_enabled: false single-tenant), alloy-config.alloy (local.file_matchloki.source.fileloki.process json/timestamp/labels → loki.write; uses the record ts as the entry time via RFC3339Nano), and Grafana provisioning (Loki datasource isDefault + a starter eden-explore dashboard: one logs panel + service/level/experiment_id multi-select template variables). The service field is always emitted by logging.py, so the plan's path-derived label fallback is unnecessary and documented as such in the config header.
  • Optional infra-stdout overlay (compose.logging-infra.yaml + logging/alloy-config-infra.alloy) — layered on top, ALSO captures Postgres + Forgejo container stdout (not covered by the JSONL tail) via Alloy's discovery.docker + loki.source.docker. It mounts the host docker socket, so per the AGENTS.md privilege-isolation discipline it is a separate overlay (exactly how compose.docker-exec.yaml isolates the same privilege). Uses a dedicated :?-guarded EDEN_LOGGING_DOCKER_GID (the in-container socket gid) — deliberately not EDEN_DOCKER_GID, which defaults to 0 and would let the overlay come up with a wrong gid and fail confusingly at runtime.
  • Storage classification. ${EDEN_EXPERIMENT_DATA_ROOT}/loki/ + /alloy/ are DERIVED / non-protocol-owned (chapter 01 §13): a queryable projection of logs/, rebuild by re-ingesting, NOT covered by the §13 durability invariant and NOT exercised by the durability smoke. Documented in the overlay comments + observability.md + reference/compose/README.md.
  • setup-experiment.sh + .env.example — always generate EDEN_GRAFANA_ADMIN_PASSWORD (same idempotent secret path as every other secret; overrides Grafana's admin/admin) and always create the loki/+alloy/ data-root subdirs (chmod 0777; Loki runs as uid 10001). No --with-logging flag: both artifacts are teardown-cheap, so the issue's frictionless bring-up holds — just add -f compose.logging.yaml. Anonymous Grafana access is off by default (documented one-liner to enable for pure local demos).
  • smoke-logging.sh + compose-smoke-logging CI job — path-gated on the compose bucket, non-required status (same posture as compose-smoke-checkpoint), bash-3.2-safe. Statically merge-gates the privileged infra overlay (runs even socket-less), brings up base+subprocess+logging, asserts Loki ingests EDEN lines + Grafana healthy with the datasource + eden-explore dashboard provisioned + a {service="orchestrator"} LogQL query returns ≥1 line; when a docker socket is reachable, also layers the infra overlay and asserts postgres stdout reaches Loki.
  • Docsobservability.md (§1 cross-link, §2.5 ladder L3 flipped to In place + Promtail→Alloy note, new §2.8 walkthrough — appended as §2.8 rather than the plan's §2.7 since the doc grew two subsections since the plan was written; no renumbering), user-guide.md (§1 grafana row), reference/compose/README.md (overlay subsection + derived-storage classification), AGENTS.md Commands table (bring-up + smoke rows). The docs/roadmap.md row was not flipped: the roadmap is phase-organized and standalone issue-driven chunks (e.g. #109/#212/#237) are CHANGELOG-tracked, not roadmap-tracked; the authoritative status tracker for the log-persistence ladder is observability.md §2.5, which was flipped.

Deferred (each tracked as a GitHub issue per the AGENTS.md deferral-tracking rule). Loki per-experiment label namespacing / multi-tenancy for shared deployments — not needed for the single-experiment Compose stack (the experiment_id label disambiguates); follow-up #244. Helm/k8s equivalent of the overlay for the Phase 13 substrate — follow-up #245. External streaming (CloudWatch/Datadog/Grafana Cloud) remains rung L4 of the observability.md §2.5 ladder, unchanged by this chunk.

Refactor F-3: regroup eden-wire/server.py into per-resource APIRouter modules (issue #115)

Resolves the code-quality audit's F-3 / L-D / L-F entries (see docs/audits/2026-05-20-phase-c-disposition.md). The 1552-line server.py — a single 1424-line make_app nesting 43 route handlers + 9 exception handlers + 5 guard closures over store / admin_token / the subscribe + checkpoint substrate — is split into per-resource APIRouter modules. make_app is now a ~95-line assembler. Purely structural: no route path, method, query/header contract, request/response body, status code, or error envelope changes. Conformance (uv run pytest -q conformance/, 250 passed) is the contract enforcer and passes pre- and post-refactor.

New modules.

  • _dependencies.py — the frozen RouterDeps dataclass (constructed once per make_app, threaded into each build_router(deps); mirrors the pre-F-3 closure-capture set so the multi-app-in-one-process guarantee holds) plus the module-level guards lifted from make_app: check_experiment (drops the dead url parameter the closure never referenced — the ExperimentIdMismatch envelope's instance is built from str(request.url) at the handler boundary), enforce_worker, enforce_in_any_group, stamp_created_by, worker_id_from_request.
  • _artifact_fd.py — the descriptor-walk artifact-serving primitives (_open_artifact_fd + the O_NOFOLLOW/O_DIRECTORY walk, _check_not_symlink, _is_symlink, _build_content_disposition, open_and_read_artifact, artifact_response_headers, MAX_ARTIFACT_BYTES, _SymlinkRejected). Consolidates the security-sensitive walk in one auditable file (resolves L-F structurally — _serve_artifact carried no # slop-allow but was audit-flagged).
  • routers/ — 11 per-resource modules (tasks, ideas, variants, dispatch_mode, experiment_lifecycle, experiment_read, events, workers, groups, checkpoints, reference). Each exposes build_router(deps) -> APIRouter; handlers are module-level closure factories (_<route>(deps) returning the inner async def) so every handler is an independently-measured function for the complexity gate while build_router stays a thin assembler.

Auth-dispatch threading preserved verbatim per the audit-disposition's per-route matrix: direct require_admin / require_worker callers stay direct (workers / groups / checkpoints / reference), either-auth sites stay inline (GET /dispatch_mode, GET /state, GET {base}, GET /workers), POST /tasks's split-by-kind authority stays inline, and whoami remains the sole direct require_worker call. No enforce_either helper was introduced (would be feature-creep beyond F-3).

Behavior-preservation subtleties. _submission_from_wire keeps raising HTTPException(status_code=400) (NOT BadRequest) — preserving FastAPI's default {"detail": ...} body shape rather than the problem+json envelope. The 9 app-level exception handlers move to module-level functions registered via app.add_exception_handler in _install_exception_handlers; the 5 identical-bodied chapter-7 error handlers collapse onto one shared _error_envelope_handler registered against the same exact type set as before (so MRO resolution is unchanged and an unrecognized WireError still falls through to 500).

Tests. All existing eden-wire tests pass unchanged. test_artifact_route.py monkeypatches retarget from eden_wire.server to eden_wire._artifact_fd. New: test_multi_app_isolation + test_path_segment_scoping_no_shadow (pin Decision 2's no-module-state + Decision 5's path-segment-scoping invariants) in test_wire_roundtrip.py, and test_reference_validate.py (wire-level coverage for the two /_reference/ validate routes that had none). The complexity-gate runs clean with the two # slop-allow: annotations removed (file-level F-3 + function-level L-D). Closes #115.

Executor-side multi-file artifact upload UX (issue #212; follow-up to #120)

Extends the issue-#120 multi-file artifact upload UX (text box + file uploads → server-side .tar.gz bundle with a top-level manifest.json) to the executor submission form. #120 landed this on the ideator + evaluator forms and deferred the executor surface; this closes that gap so executors can attach build outputs, screenshots, profiling captures, etc. alongside their variant submission.

Template. executor_claim.html gains enctype="multipart/form-data" and an artifact (optional) fieldset mirroring the evaluator form: an artifact_text markdown textarea, a multiple artifact_files input, and the existing artifacts_uri field repositioned as an explicit-URI override ("when set, text / uploads above are ignored"). The fieldset surfaces both the artifact (bundling collision / filename rejection) and artifacts_uri per-field errors. executor_submitted.html now renders the resulting artifacts_uri as a scheme-aware link, matching evaluator_submitted.html.

Route. routes/executor.py reuses the predict_artifact_uri-adjacent write_artifact_bundle helper from artifacts.py. New _collect_uploads + _maybe_bundle_executor_artifact helpers mirror the evaluator route exactly; _parse_submit_form now collects uploads and rewrites the draft's artifacts_uri to the freshly-bundled artifact (single-file .md/<ext> or .tar.gz) when the operator supplies text/uploads instead of an explicit URI. The bundle text entry lands at the role-coherent variant.md headline (cf. idea.md / evaluation.md). The artifacts_uri already flowed through VariantSubmission onto Variant.executor_artifacts_uri (spec 03-roles.md §3.4, issues #164/#165) — only the URI-production path is new. The executor's separate Variant.description field is unchanged.

Tests. New test_executor_multifile.py mirrors test_evaluator_multifile.py / test_ideator_multifile.py: text+files → .tar.gz (with per-entry serving round-trip), text-only → .md, single-file → raw <ext>, explicit URI overrides bundling, and the no-artifact path stays artifacts_uri == None. All cases drive the executor's status="success" git-reachability gates with a real child commit.

Refactor F-1: split eden-storage._StoreBase into a mixin family (issue #114)

Code-quality follow-up from the Phase-A audit (docs/audits/2026-05-20-phase-c-disposition.md §1 F-1). The 1,638-SLOC _base.py monolith (MI 0.00, carried under a # slop-allow-file: annotation deferring the split to #114) is split into a thin core plus a per-resource mixin family. Behavior-preserving by construction — no spec, schema, wire, or Store-Protocol change; the only observable difference is internal file layout.

Shape. _base.py is now 235 SLOC: module constants, the _Tx dataclass, the abstract _StoreCore (owns __init__, the event-id factory, _event/_ts/_maybe_ts, the cross-resource read-side predicates _require_* / _find_starting_variant_for_implement_task / _validate_registry_id, the backend-primitive declarations, and abstract stubs for the two split-body helpers), and the composite _StoreBase. Eight mixins live under _ops/:

  • _TaskCreateOpsMixin (tasks_create.py, 258 SLOC) and _TaskLifecycleOpsMixin (tasks_lifecycle.py, 715 SLOC) — the combined _TaskOpsMixin measured 955 SLOC, over the 800 file gate, so the plan §3.7 fallback split fired (create vs lifecycle).
  • _IdeaOpsMixin, _VariantOpsMixin, _EventOpsMixin, _ExperimentOpsMixin, _WorkerOpsMixin, _GroupOpsMixin (45 / 78 / 21 / 176 / 90 / 142 SLOC).
  • Module-free pure helpers in _helpers.py.

Each mixin inherits _StoreCore so method bodies resolve self._get_* / self._event / self._apply_commit under pyright; _StoreBase flattens the mixin MRO. A module-load-time assert _StoreBase.__mro__[1:10] == (...) guard (plan §6/§8.1) fails loud on any future bases reorder. The composite-commit invariant (one _Tx per public method, one self._apply_commit(tx), deterministic field-walk order) is preserved unchanged across all three backends.

Backends untouched. InMemoryStore / SqliteStore / PostgresStore still subclass _StoreBase verbatim (zero LOC change); _checkpoint.py still binds to the abstract primitives (now on _StoreCore) and imports _StoreBase / _Tx from _base unchanged.

Plan deviations (surfaced for review). (1) The disposition doc's proposed _ValidationOps mixin was rejected — each _validate_* helper co-locates with its consumer (plan §D.3). (2) L-O (reassign_task, audit LEN=122) closes here: the method now measures 67 LEN / CC 6 on _TaskLifecycleOpsMixin, already inside the threshold thanks to the pre-existing _stage_reassign_reclaim / _reassign_event_payload helpers; per plan §D.6 no cosmetic per-state split was added. (3) The wave-1 _validated_update re-export from _base was retired (it became a dead # noqa: F401 shim once _base stopped using the helper — counter to BASE.md's no-greenfield-shims posture); the 4 in-repo eden-git test import sites now reference the helper's canonical home eden_storage._ops._helpers.

Acceptance. The # slop-allow-file: annotation on _base.py is removed and python3 scripts/check-complexity.py passes without it. Storage suite: 571 passed across memory + sqlite + postgres (EDEN_TEST_POSTGRES_DSN set), full pyright clean, conformance suite + Compose smoke green.

What this does NOT cover. M-4 (extract the _apply_commit _Tx-walk into a shared backend-hook helper) → issue #230; L-4 (reduce _validate_non_no_op_variant CC=17 readability) → issue #231. Both are independent audit items, never in F-1's scope. No per-mixin unit tests are added (the Protocol-level parametrized suite is the conformance contract; per-mixin tests would create a second source of truth that drifts — plan §3.9). F-3 (#115) and F-4 (#116) are sibling Phase-C refactors with separate plans.

Backfill: /admin/control/workers + /admin/control/groups deployment-scoped pages (issue #146)

Backfills the Phase 12c CHANGELOG-narrated deferral ("Deployment-scoped worker/group registry admin pages not shipped: the chapter 11 §6 registry remains admin-only via direct API calls"). Operators administering deployment-level workers / groups no longer need raw curl against the control plane.

Routes. New package reference/services/web-ui/src/eden_web_ui/routes/admin/control/ with two modules:

  • workers.pyGET /admin/control/workers/ (list + filter + group-membership column), POST /admin/control/workers/ (register), GET /admin/control/workers/{worker_id}/ (detail), POST /admin/control/workers/{worker_id}/reissue-credential. Token-page renders the one-shot registration_token (Cache-Control: no-store).
  • groups.pyGET /admin/control/groups/ (list + transitive-worker counts), POST /admin/control/groups/ (register with optional initial members), GET /admin/control/groups/{group_id}/ (detail + transitive closure), POST .../members (add), POST .../members/{member_id}/remove, POST .../delete. Reuses admin_groups.walk_transitive_workers for the DAG walk — it only needs read_group + read_worker, both of which ControlPlaneClient exposes.

Both modules mirror the existing per-experiment /admin/workers/ + /admin/groups/ patterns: server-side Jinja, auth-first POST (get_session runs before CSRF), closed-allowlist banners surfaced via ?ok= / ?warn= / ?error= query params.

Templates. Five new files — admin_control_workers.html, admin_control_worker_detail.html, admin_control_worker_token.html, admin_control_groups.html, admin_control_group_detail.html. Each carries a "deployment-scoped" banner + a back-link to the per-experiment counterpart so operators can tell which registry they are viewing.

Wiring. app.py registers the new router only when make_app(control_plane=...) is set (same gate as /admin/experiments/); the control plane client carries the admin bearer at startup (cli._build_control_plane_client), so admin-gated reads + writes go through one client. base.html gains two new top-nav links ("deployment workers" / "deployment groups") behind the same control_plane_enabled flag that surfaces "experiments".

Tests. New test_admin_control_workers_routes.py + test_admin_control_groups_routes.py following the test_admin_experiments_routes.py shape — a thin _StoreBackedClient adapter wraps an InMemoryControlPlaneStore and ducktypes as ControlPlaneClient for the routes. Covers auth-first redirects, CSRF failure → 403, list / register / detail / reissue / membership flows, reserved-identifier + cross-registry collision branches, and the control_plane=None → 404 wiring gate.

Admin gating. Follows the existing /admin/experiments/ posture: route handlers gate on get_session only; the deployment-level admins-group enforcement lives in the wire layer (chapter 7 §15 _enforce_admin). Issue #144 will add an additional web-ui-side check across every /admin/* route; this PR does not pre-empt that work. Closes #146.

Conformance suite parallelization (pytest-xdist)

CI tooling: the conformance job was the longest single CI job (~8 min wall locally, similar on the runner), and it gates cascade latency. Profiling (uv run pytest -q conformance/ --durations=30) showed the cost is almost entirely per-scenario subprocess boot, not compute: the run logged ~671 s of user CPU at only ~150% utilization, with each scenario's setup (not call) dominating at ~1.5–2.9 s. That is because the per-scenario iut fixture spawns a fresh eden_task_store_server and eden_control_plane_server subprocess against in-memory storage on random ports (--port 0); there is no giant inherently-serial test (the slowest, _meta/test_self_validation, is ~10 s). That profile is the textbook pytest-xdist case — boot-bound, embarrassingly parallel.

Parallelization. pytest-xdist>=3.6 is added to conformance/pyproject.toml and the CI conformance job now runs uv run pytest -q conformance/ -n auto (the AGENTS.md Commands table is updated to match). Locally this cut wall time from ~479 s to ~120 s (~4×, ~9 cores); on the runner's 4 vCPUs the expected floor is ~user-CPU/4 ≈ 3 min, comfortably under the speedup target. Scenarios are independent — each owns its subprocess stack on OS-assigned ports and a per-test tmp_path, and the file:///tmp/... strings in some scenarios are inert artifact-URI payloads stored in-memory, not real filesystem writes — so no port or fixture-file contention surfaced across repeated full runs.

The one cross-test shared-state surface, made xdist-safe. spec/v0/07-wire-protocol.md §7's vocabulary-closure assertion (conformance/scenarios/test_error_vocabulary.py) is a suite-level check: it reads a session-scoped observed_problem_types accumulator that every other scenario populates through its WireClient. Under xdist that accumulator is per-worker, so no single worker observes the whole run — a naive -n auto made test_v0_vocabulary_each_observed_at_least_once fail (it saw only the types emitted on its own worker). Per the stop-condition discipline this was surfaced and designed-around rather than papered over with a serial marker:

  • The closed-vocabulary tables and closure logic moved to a shared harness module conformance/src/conformance/harness/error_vocabulary.py (consumed by both the scenario and the plugin).
  • conformance/src/conformance/harness/plugin.py gained xdist hooks: each worker publishes its observed types via workeroutput at pytest_sessionfinish; the controller folds them into a union in pytest_testnodedown and asserts BOTH closure directions (no out-of-vocab type; every core type observed) over the union at controller pytest_sessionfinish, failing the session (ExitCode.TESTS_FAILED, without clobbering a pre-existing failure) on violation. Because pytest_testnodedown also fires on the worker-crash path (worker_errordown, where workeroutput may be missing/partial), the controller marks the union incomplete on any error-or-missing-key node-down and skips the closure assertion (emitting a yellow "closure NOT asserted" note) rather than report a misleading "core type never observed" failure on top of the already-red run — the crash itself fails the run.
  • The two scenario tests skip on a worker (idiom: hasattr(config, "workerinput")) with a reason pointing at the plugin; under serial runs they assert in-process exactly as before. The skip preserves chapter-9 §5 group citation coverage because check_citations.py reads docstrings statically. The closure retains teeth under xdist — verified by a deliberate incomplete-coverage subset run that exits non-zero with the missing-types diagnostic.

test_observed_types_are_in_v0_vocabulary (the no-out-of-vocab direction) is monotonic, so it stayed green per-worker even before the controller-side check; the controller assertion now covers both directions authoritatively under xdist for a single source of truth.

Out of scope / not pursued: sharing one IUT subprocess stack across compatible scenarios (rejected — per-scenario fresh state is a deliberate isolation property of the harness, and xdist already recovers the wall-time win without weakening it); replacing poll/sleep waits (the suite is boot-bound, not poll-bound — no hot poll loops surfaced in the durations). The reference suite passes under -n auto across repeated runs with no order-dependent flakes.

Slug-uniqueness soft-check on idea submission (issue #121)

Polish item: when an operator submits an idea whose slug collides with an existing idea in the same experiment, the system now surfaces an advisory warning at submission time rather than silently accepting the collision. Slug uniqueness is not a protocol invariant — idea identity is by idea_id (spec/v0/02-data-model.md §5.1), and variant branches embed the unique variant_id so collisions are harmless in lineage tracking. The check is soft: the request still succeeds 200, and operators can deliberately reuse a slug (e.g., sibling variations grouped by slug) by ignoring the warning.

Spec amendment. spec/v0/07-wire-protocol.md §3 documents an OPTIONAL warnings: list[string] field on create_idea's response body, alongside the idea fields per idea.schema.json. Warnings are non-normative diagnostic strings; implementations MAY omit the field, and clients MUST NOT rely on its presence or contents for correctness. (Conformance does NOT assert this field — it remains a reference-impl convenience.)

Wire layer. reference/packages/eden-wire/src/eden_wire/server.py's _create_idea calls a new module-level helper _slug_conflict_warnings(store, idea) that scans the experiment's existing ideas (store.list_ideas() is single-experiment scoped) for slug matches excluding the just-created idea. When matches are present, the response body grows a warnings array listing the colliding idea_id values; when none, the field is omitted (Pydantic-equivalent exclude_none semantics).

eden-manual CLI. reference/scripts/manual-ui/eden-manual's cmd_ideation_submit now captures the create_idea response, prints any per-idea warning to stderr at creation time, and includes a warnings array in the final JSON output when any collisions were detected.

Web UI ideator. The web-ui's submit_idea route (reference/services/web-ui/src/eden_web_ui/routes/ideator.py) doesn't go through the wire — it calls store.create_idea directly — so _persist_idea_drafts now performs the same slug scan locally and returns a slug_warnings list alongside the idea_ids. Warnings are passed through to _render_submitted and rendered as a <div class="slug-warnings" role="alert"> block on ideator_submitted.html. Pre-submit AJAX validation (an alternative shape proposed in the issue) was deferred — the post-submit warning is sufficient for the polish goal and avoids the additional HTMX wiring.

Tests. New TestCreateIdeaSlugWarnings class in reference/packages/eden-wire/tests/test_lifecycle_wire.py covers three cases: unique slug → no warnings key; duplicate slug → warnings array names the prior idea_id; triple-collision → all prior IDs listed. New TestSlugSoftCheck class in reference/services/web-ui/tests/test_ideator_flow.py asserts the warning block renders on the submitted page on collision and is absent on unique-slug submissions.

Out of scope (followups): Pre-submit AJAX warning + confirmation step in the web-ui ideator form (the issue suggested this; deferred since the post-submit warning is sufficient and the form's HTMX surface is being reworked in adjacent issues — revisit if operator feedback shows the post-submit timing is too late). docs/user-guide.md §5 and .claude/skills/eden-manual-ideator/SKILL.md Phase 4 mentions of the soft-check are intentionally omitted since the warning is self-describing and the user-guide entry would just restate the behavior. Closes #121.

Fix #133: surface ideation policy in experiment config

Closes #133. Moves the ideation-creation policy from a deployment-time CLI flag (--ideation-policy <module:callable> on the orchestrator) plus two env-var knobs (EDEN_IDEATION_POLICY_TARGET_PENDING / EDEN_IDEATION_POLICY_MAX_TOTAL) to a declarative block in the experiment config YAML. Operators authoring an experiment now choose between open-ended exploration (maintain_pending) and bounded-budget experiments (fixed_total) in the same file that already carries parallel_variants / objective / dispatch_mode — no more reading orchestrator container args to discover why pending tasks refill.

Schema. spec/v0/schemas/experiment-config.schema.json gains an optional ideation_policy block discriminated by kind: maintain_pending (with optional target / max_total; defaults target=3, max_total=null for unbounded) or fixed_total (requires total >= 1). Per the dispatch_mode precedent, unknown extra keys are tolerated and ignored; an unrecognized kind fails validation rather than silently falling back. spec/v0/02-data-model.md §2.4 documents the block; spec/v0/03-roles.md §6.2 step 1 now points at the config block as the policy-selection mechanism.

Pydantic. reference/packages/eden-contracts/src/eden_contracts/config.py adds MaintainPendingPolicyConfig / FixedTotalPolicyConfig (discriminated union IdeationPolicyConfig) and threads ExperimentConfig.ideation_policy: IdeationPolicyConfig | None = None. The max_total field intentionally does NOT carry the repo's NotNone wrapper because the schema explicitly types it as ["integer", "null"] (null means "no cap"); other optionals keep the wrapper. Twelve new fixture cases in reference/packages/eden-contracts/tests/cases.py cover both kinds' valid shapes, the unknown-kind reject, the missing-required-arg reject, the unknown-extra-key tolerance, and explicit-null rejection.

Factory. reference/packages/eden-dispatch/src/eden_dispatch/policies.py adds build_policy(config: IdeationPolicyConfig | None) -> IdeationPolicy that dispatches on kind and constructs the matching factory. default_policy() was simplified from its env-var-reading form to a plain maintain_pending(target=3) — env-var configuration is gone, the config is authoritative.

Orchestrator. reference/services/orchestrator/src/eden_orchestrator/cli.py drops the --ideation-policy CLI flag and adds a required --experiment-config <path> flag. The orchestrator loads the config via eden_service_common.load_experiment_config, calls build_policy(config.ideation_policy), and passes the resulting callable into run_orchestrator_loop / run_multi_experiment_loop. In multi-experiment mode the same single config still applies to all held experiments — per-experiment configs from the control-plane registry projection are a separate follow-up. The --ideation-policy <module:callable> shape is removed (not deprecated) per the project's no-backwards-compat-shims rule (AGENTS.md). Custom callable specs from the config are explicitly out of scope (security: arbitrary code load).

Compose + setup-experiment. reference/compose/compose.yaml adds --experiment-config /etc/eden/experiment-config.yaml to the orchestrator's command and mounts the config via configs: (same eden-experiment-config source the task-store-server / web-ui already use). The EDEN_IDEATION_POLICY_* env vars are removed from the orchestrator's environment: block, .env.example, and setup-experiment.sh. compose.multi-orchestrator.yaml gets the same flag + config mount on orchestrator-2.

Smoke scripts. All six smokes that previously sed-edited EDEN_IDEATION_POLICY_* in .env (smoke.sh, smoke-subprocess.sh, smoke-subprocess-docker.sh, smoke-multi-orchestrator.sh, smoke-manual-mode.sh, e2e.sh) now append a fixed_total block directly to the copied reference/compose/experiment-config.yaml. smoke.sh pinned at total: 3, smoke-manual-mode.sh at total: 1, e2e.sh at total: 4.

Tests. Orchestrator's test_e2e.py / test_subprocess_e2e.py write a per-test config with ideation_policy: { kind: fixed_total, total: 3 } appended (replacing the prior env={EDEN_IDEATION_POLICY_MAX_TOTAL=3} posture). test_e2e_real_subprocess.py in the web-ui suite does the same with maintain_pending(max_total=0) for the no-create-more-tasks shape. test_dispatch_mode_gating.py swaps the two env-var tests for three new ones covering default_policy shape + build_policy(None) + build_policy(MaintainPendingPolicyConfig) + build_policy(FixedTotalPolicyConfig).

What this does NOT cover:

  • Per-experiment ideation policy in multi-experiment mode. When the orchestrator runs under --control-plane-url, the single --experiment-config applies uniformly to every held lease. Surfacing per-experiment configs (so one orchestrator can run a maintain_pending experiment alongside a fixed_total one) requires plumbing per-experiment YAML access through the control-plane registry projection's config_uri. Follow-up: #214.

Explicit non-goals (called out because the original issue listed them; not deferred — they will not land at all in this shape):

  • Custom callable specs in the experiment config. The pre-#133 --ideation-policy module:callable form could load arbitrary code; surfacing that in YAML would let operator-authored config import code from any importable module — a security regression. The two named kinds (maintain_pending + fixed_total) cover the documented use cases; deployments that need something else extend the schema with a new kind rather than reintroducing the callable escape hatch.

Fix #144: enforce admins-group at web-ui /admin/* route layer

Closes #144. The wire layer already gates the mutating admin endpoints on transitive membership in the admins group (server.py:_enforce_in_any_group), but the web-ui's /admin/* route handlers gated only on get_session(request) is None. Under the pre-#140/#143 single-bearer model the gap was invisible (every signed-in user was implicitly admin), but it becomes a live read-side leak the moment per-user session bearers + non-admin defaults land. This change closes the gap now so the auth-model changes can ship without a coordinated cutover.

Middleware approach. A new reference/services/web-ui/src/eden_web_ui/middleware.py AdminGateMiddleware runs before every request and short-circuits on the /admin prefix: missing session → 303 to /signin (preserves the existing redirect-to-signin behavior); Store.resolve_worker_in_group(worker_id, "admins") raises → 502 _error.html (matches the chunk-9e dashboard read-failure shape); membership returns False → 403 _error.html ("This page requires membership in the admins group"). Per-handler get_session(...) is None: redirect checks remain in place as defense-in-depth and to keep the type-narrowing pattern intact across the 40+ admin handlers; the middleware is the load-bearing check, but a handler also seeing None is treated as belt-and-suspenders rather than a no-op replacement. The middleware is registered in app.py before include_router so it wraps every admin sub-router (admin/, admin_workers/, admin_groups/, admin_artifacts/, admin_experiments/). A new /admin/* route added in any future PR is gated automatically — no per-handler patching required.

Tests. New test_admin_gate.py parametrizes nine representative GETs (one per admin sub-router prefix) across three postures: unauthenticated → 303 to /signin; signed-in non-admin → 403 with the HTML forbidden page; signed-in admin → 200 (positive control). A separate test asserts that a POST to /admin/tasks/<id>/reclaim from a non-admin session is rejected by the gate before reaching the handler's CSRF check (the body matches the gate's forbidden copy, not the CSRF-failure response). The transport-failure branch is covered by monkey-patching store.resolve_worker_in_group to raise; the gate returns 502 with the "transport failure" page instead of crashing. The shared conftest.py store fixture now registers an admins group containing every test worker — mirroring the production deployment shape (setup-experiment.sh puts the web-ui worker in admins) — so existing admin tests pass the gate without per-test changes. The e2e tests (test_admin_e2e.py, test_admin_workers_e2e.py, test_admin_groups_e2e.py, test_admin_ideas_e2e.py) explicitly add their web-ui worker (ui-admin / ui-admin-workers / ui-admin-groups / ui-admin-ideas) to admins via their existing admin StoreClient seed step.

Docs. docs/observability.md §2.1 admin dashboards section gains an explicit "Auth" note: every /admin/* page load requires the signed-in worker to be a transitive member of admins, with a pointer to /admin/groups/admins/ for adding new members. The setup-experiment script's existing add_to_group("admins", "${WEB_UI_WORKER}") step already meets the requirement for the default Compose deployment — no operator-facing migration is needed for existing experiments.

Out of scope. Per-route fine-grained permissions (e.g. "non-admins see /admin/events/ but not /admin/workers/") remain deferred — today's all-or-nothing posture matches issue #144's "enough for v0" scope. UI-level audit logging of admin-page access is also deferred; the wire layer already logs admin-gated mutations, and adding a parallel per-page-load log is a separate observability concern.

Executor-host substrate access + DooD env-var forwarding (issues #154 + #155)

Backfills two Phase 12a-1f deferrals together (single PR — both touch the DooD code path and are simpler to reason about as one change).

Issue #154 — executor-host substrate access. Phase 12a-1f opened three read-side substrates (git bare clone, artifact HTTP route, readonly Postgres) to ideator + evaluator subprocesses but skipped the executor. reference/services/executor/src/eden_executor_host/cli.py now registers add_substrate_arguments(parser) and threads the resolved SubstrateArgs into the spawned execution_command's env (mirroring the evaluator's pattern). User env-file content is funnelled through strip_reserved_substrate_keys before merging so a user file cannot re-inject substrate keys the host suppressed. The subprocess-mode block was extracted into a _run_subprocess_mode(args, ...) helper to stay under the complexity gate after the substrate logic landed. The reference compose stack now sets EDEN_ARTIFACT_URL / EDEN_ARTIFACT_PATH_ROOT / EDEN_READONLY_STORE_URL / EDEN_REPO_DIR on the executor-host service via an environment: block in reference/compose/compose.subprocess.yaml (matching the existing evaluator wiring).

Issue #155 — DooD --exec-network un-suppression. reference/services/_common/src/eden_service_common/cli.py adds a new --exec-network flag (also read from $EDEN_EXEC_NETWORK) plumbed onto ExecArgs.network. container_exec.py wrap_command accepts a network: str | None kwarg and emits --network <name> on the docker run invocation when set. substrate_args_for_exec_mode(substrate, *, exec_mode, exec_network) now only suppresses substrate keys when exec_mode == "docker" AND exec_network is None — when the operator attaches the spawned sibling to a reachable compose network, the substrate URLs are forwarded normally. All three hosts (ideator/executor/evaluator) thread the resolved network into their per-spawn wrap_command call. The WARN log key (substrate_access_disabled_in_exec_mode_docker) is unchanged, but now carries a hint: field pointing operators at --exec-network.

Compose wiring. reference/compose/compose.docker-exec.yaml passes --exec-network ${EDEN_EXEC_NETWORK:-eden-reference_default} to each of ideator-host / executor-host / evaluator-host. The default matches docker compose's automatic <project>_default network; operators with custom project names override via $EDEN_EXEC_NETWORK. Substrate env vars carry into the docker-exec overlay through compose's additive map-merge of environment: blocks.

Tests. New test_executor_subprocess_env.py mirrors test_ideator_subprocess_env.py / test_evaluator_subprocess_env.py. test_substrate_arguments.py gains coverage for the new exec_network parameter (docker-with-network un-suppresses; host-mode ignores the param). test_container_exec.py asserts wrap_command emits --network <name> iff network= is passed. smoke-subprocess-docker.sh gains a post-quiescence log assertion that none of the three host services emitted substrate_access_disabled_in_exec_mode_docker — proves the --exec-network plumbing reached the hosts end-to-end.

Spec binding doc. spec/v0/reference-bindings/worker-host-subprocess.md §9 updated: the four substrate env vars are now described as available to all three subprocess-mode roles (was: ideator + evaluator only); §9.3's DooD-suppression bullet documents the new --exec-network opt-in path.

Out of scope (followups): Container-image substrate baking (e.g., bundling a postgres client in the runtime image) and per-worker Forgejo tokens with branch ACLs (deferred to Phase 13e). The ideator's docker-exec command intentionally still omits --repo-path (no git-substrate forwarding for DooD ideator); adding it would require also threading Forgejo credentials into the spawned sibling, which is a separate trust-boundary decision. Closes #154 / #155.

Fix #109: EDEN_LOG_DIR file-handler log persistence

Closes #109. The shared logger at reference/services/_common/src/eden_service_common/logging.py configure_logging now installs a logging.handlers.RotatingFileHandler alongside the existing stdout StreamHandler whenever EDEN_LOG_DIR is set in the environment. The file handler writes ${EDEN_LOG_DIR}/<service>.jsonl using the same _JsonFormatter as stdout (no schema divergence), in append mode (mode="a"), flushed after every record (the stdlib StreamHandler.emit default — so a SIGKILL'd process loses at most the bytes of one not-yet-flushed JSON line, not minutes of log). Rotation is controlled by EDEN_LOG_MAX_BYTES (default 50 * 1024 * 1024) × EDEN_LOG_BACKUP_COUNT (default 5); both env vars accept positive ints and fall back to defaults on empty / non-integer / zero values so misconfiguration cannot prevent the logger from coming up. A non-creatable EDEN_LOG_DIR (permission denied, parent is a regular file) falls back to stdout-only — the service still boots.

Compose wiring. Each long-running service (task-store-server, orchestrator, ideator-host, executor-host, evaluator-host, web-ui) in reference/compose/compose.yaml gains EDEN_LOG_DIR=/var/lib/eden/logs in its environment: block and a ${EDEN_EXPERIMENT_DATA_ROOT}/logs/<service>:/var/lib/eden/logs host bind-mount (12a-1g posture). reference/scripts/setup-experiment/setup-experiment.sh mkdir -p + chmod 0777 the per-service log dirs alongside the existing substrate dirs. Because the log dirs are host bind-mounts, docker compose down -v does not wipe them — only an explicit rm -rf ${EDEN_EXPERIMENT_DATA_ROOT} does, mirroring the rest of the 12a-1g substrate tree.

Tests. New reference/services/_common/tests/test_logging.py covers nine scenarios: stdout-only when EDEN_LOG_DIR unset, per-service .jsonl creation when set, parent-dir creation, append-across-reinit (crash forensics survive a restart), idempotent reconfigure (no handler stacking, no duplicate writes), rotation kicks in at the configured EDEN_LOG_MAX_BYTES, invalid env-var values fall back to defaults, an unwritable EDEN_LOG_DIR doesn't break stdout, and EDEN_LOG_DIR= whitespace is treated as unset. reference/compose/healthcheck/smoke.sh gains two assertions: the six per-service log dirs exist post-setup-experiment, and each <service>.jsonl is non-empty + parses as JSON-lines after the smoke run.

Docs. docs/observability.md §1 substrate table now lists the JSON-line bind-mount alongside the docker-driver logs; §2.5 is rewritten to cover both destinations, the jq patterns for reading the host-side files, and the crash-survival caveats (compose-down survival vs rm -rf reset). docs/user-guide.md §1 components table gains a service logs row; §11 Gotchas + resets notes that rm -rf $EDEN_EXPERIMENT_DATA_ROOT wipes log history alongside the rest of the substrate tree.

Backfill: compose-smoke-checkpoint CI job (issue #152)

Backfills the Phase 12b CHANGELOG-narrated deferral ("Compose smoke for checkpoint — no compose-smoke-checkpoint job yet; deferred until the deployment-substrate integration lands"). The portable-checkpoint export/import round-trip (spec/v0/10-checkpoints.md §9) is now exercised end-to-end through the Compose stack on every CI run.

Smoke script. New reference/compose/healthcheck/smoke-checkpoint.sh drives the round-trip in six phases: (1) setup-experiment + bring up the full stack, run to orchestrator quiescence (same posture as smoke.sh); (2) snapshot pre-checkpoint wire state (event/variant/idea/task counts + sorted id sets); (3) export via POST /v0/experiments/<id>/checkpoint (admin-gated per chapter 7 §13.1), assert the tar archive parses and carries manifest.json; (4) compose down -v + wipe the substrate bind-mount data root + recreate the substrate subdir layout; (5) bring up ONLY postgres + task-store-server against the same .env (no setup-experiment re-run so no reserved-group / initial-admin bootstrap runs against the receiver, satisfying chapter 10 §9's "store-must-be-empty" import precondition; no orchestrator/workers/forgejo so nothing races against the import), POST the archive to /v0/checkpoints/import, assert the spec-pinned 201 Created response (chapter 7 §14.2); (6) re-read wire state, assert every count + id-set matches the pre-checkpoint snapshot, and confirm imported_from.checkpoint_exported_at is stamped on the receiver-side experiment row (chapter 10 §10).

CI job. New compose-smoke-checkpoint job in .github/workflows/ci.yml mirrors the existing compose-smoke* shape (20-minute timeout; verifies docker compose + jq + python3 availability; runs the smoke from reference/compose). Not required by branch protection in this PR; same posture as the other newly-added smoke jobs (compose-smoke-manual-mode, compose-smoke-multi-orchestrator) — bump to required-status after staying clean on main for ~2 weeks.

Posture. Asserts state-preservation correctness, not worker-resume; the receiver phase intentionally skips forgejo + orchestrator + worker hosts. The reference compose deployment leaves --checkpoint-import-credentials-dir unset (issue #150 deployment posture), so the import response's warnings array carries the expected "tokens NOT persisted — operators must reissue via the admin endpoint" message; the smoke surfaces the warnings in its log but does not assert on the specific message text (a future deployment that wires up the credentials dir would emit a "persisted" warning instead; both are valid round-trip outcomes).

Multi-file artifacts via web UI + eden-manual (issue #120)

Extend the ideator + evaluator submission surfaces to accept multi-file artifacts. Before this PR, an idea / evaluation was a single markdown blob: operators wanting to bundle a design doc with a diagram / proto / screenshot had to consolidate into one markdown (lossy) or omit supporting context. Now the Web UI's ideator and evaluator draft forms — and the eden-manual CLI — accept a markdown body plus any number of file uploads per submission, and the bundler picks the right artifact shape:

  • text only → <id>.md (existing single-file shape; back-compat with all pre-#120 submissions).
  • one upload, no text → <id>.<ext> (the upload stored as-is under the original extension).
  • text + 1+ uploads OR 2+ uploads → <id>.tar.gz with a top-level manifest.json enumerating each entry's path / size / content_type. The role's text body lands at a role-specific basename inside the archive (idea.md / evaluation.md) so the viewer can render a headline above the per-file link table.

Shared bundler. New module-level functions in reference/services/web-ui/src/eden_web_ui/artifacts.py: write_artifact_bundle(...) drives the three branches above; predict_artifact_uri(...) returns the final URI without writing (so the route handler can construct the Idea / EvaluationDraft with the final URI before any disk side effect — keeps the existing validation barrier intact). Filename collisions, duplicate entries, the reserved manifest.json slot, and path-traversal upload names (../../etc/passwd) are rejected as ValueError and surface as per-row field errors. Single-file writes go through atomic tmp → os.replace. Bundles are gzipped tar with deterministic mtime=0 and mode=0o644 so the reproducible-output property holds.

Reader side. New helpers in _helpers.py: read_idea_content / read_variant_artifact / read_evaluation_artifact now detect .tar.gz bundles and return the role's headline entry as inline text (preserving the existing pre-#120 contract for non-bundles). Sibling read_idea_manifest / read_variant_artifact_manifest / read_evaluation_artifact_manifest return the manifest dict for bundles (None otherwise). The evaluator and executor draft templates render a new _artifact_manifest.html partial as a per-entry link table when a manifest is present. Pure-bundle artifacts (no headline entry) get a "multi-file bundle; see links below" note.

Per-entry serving. reference/services/web-ui/src/eden_web_ui/routes/artifacts.py's GET /artifacts now honors an optional &entry=<name> query param: when the resolved file is a .tar.gz AND entry is a single safe basename (no slashes, no .., no NUL), the route streams that entry's bytes out of the archive via tarfile.extractfile. Entry names containing path components return 400; non-.tar.gz URIs reject the entry= param. There is no automatic unpacking on disk (issue #120 option (1) — per the spec's note "Stick with option (1) — manifest only — unless option (2) is explicitly approved").

Ideator route. routes/ideator.py reads per-row uploads from files_<i> form fields (one input per row, multiple enabled). parse_idea_rows now accepts a parallel has_uploads_per_row: list[bool] and treats content as optional when uploads are present (otherwise the existing "content markdown is required" check still fires). _persist_idea_drafts predicts the URI via predict_artifact_uri (so a ValueError on a bundling collision is surfaced before the Idea is constructed, keeping disk clean on validation failure), then writes the bundle and creates the idea record. The HTML form gains enctype="multipart/form-data" and an inline note that browser file inputs do not survive the "add another row" round-trip.

Evaluator route. routes/evaluator.py collects uploads from a single artifact_files field; the new _maybe_bundle_evaluator_artifact helper returns (uri, error). An explicit operator-supplied artifacts_uri (e.g. for eval output that already lives somewhere the web-ui can reach) takes precedence — text/uploads are only bundled when no URI was pasted. The submit handler's body length is held under the 100-line complexity gate by extracting _parse_evaluator_submit_form.

eden-manual CLI. reference/scripts/manual-ui/eden-manual — ideation-submit's .ideas.json schema gains an optional content_files: [path, ...] per idea; evaluation-submit gains parallel --content (markdown body) and --content-file PATH (repeatable) flags. The bundling logic is duplicated inline rather than imported from eden_web_ui because the CLI runs under system python3 (shebang) without the workspace venv on PATH; the duplicated block is small (~80 lines) and a comment in the CLI flags both copies for lockstep maintenance.

Tests. New test_artifact_bundle.py covers the bundler unit-test surface: each branch, filename-collision rejection, path-traversal sanitization, zero-byte uploads, atomic-write (no .tmp siblings post-write), oversized-entry rejection, corrupt-tarball read returns None, and the per-entry serving security boundary (entry slash / .. / non-bundle URI / missing entry). New test_ideator_multifile.py covers route-level happy paths (text-only, single upload, text+upload bundle, two-uploads bundle, per-row isolation) and error paths (no text + no files re-renders with field error; duplicate upload filenames surface as form error without creating any idea). New test_evaluator_multifile.py covers the parallel evaluator surface plus the explicit-URI-takes-precedence branch.

Out of scope for this PR. Executor-side multi-file uploads — #164 (executor VariantSubmission.artifacts_uri) landed mid-flight on this branch; rather than scope-creep this PR, executor wiring (web UI executor draft form + eden-manual execution-submit --content-file) is left to a follow-up that reuses the same shared bundler. The new executor_claim.html already renders bundle manifests from the idea side via _artifact_manifest.html; only the submit-side upload UX is the follow-up. Tracked as issue #212. Server-side rendering of non-markdown formats (PDF preview, image inline) is deferred — the manifest table's per-file links open each entry in a new tab. Auto-unpacking on disk (issue #120 option (2)) is deferred — bytes stream out of the archive on demand. The artifact viewer panel does not yet render evaluation.md / variant.md headline entries inline (they appear in the per-entry link table); follow-up if the headline-rendering UX is wanted.

Auto-reissue worker credentials on checkpoint import (issue #150)

Backfills the Phase 12b deferral that left receiver-side credential reissue as manual operator work. The import endpoint now drives spec/v0/10-checkpoints.md §8 step 4 to its normative conclusion: every imported worker gets a freshly-minted bearer atomically with the rest of the import commit (no more sentinel-hash placeholder rows), and the reference wire binding optionally persists each <worker_id>.token to an operator-configured credentials directory so worker hosts pick up valid bearers at startup with no manual reissue_credential round-trip.

Storage layer. reference/packages/eden-storage/src/eden_storage/_checkpoint.py's _apply_archive_commit removes the $reissue-required$<random> sentinel hash. Every imported worker now gets store._generate_credential_token() + store._hash_credential(...) minted inside the same _apply_commit transaction (so a failure at any step rolls the entire import back per chapter 10 §8 "atomic with the rest of the import"). The plaintext tokens are returned through the new ImportResult.reissued_credentials: Mapping[worker_id, token] field (an immutable MappingProxyType).

Wire layer. reference/packages/eden-wire/src/eden_wire/server.py make_app accepts a new checkpoint_import_credentials_dir kwarg. When set, the POST /v0/checkpoints/import handler writes each <worker_id>.token file to that directory using the same atomic-write + 0o600 semantics as eden_service_common.auth.bootstrap_worker_credential (random-suffixed tmp + os.replace), then surfaces the file paths in the response's warnings array. When unset, the import still mints fresh credentials (§8 is normative) but the response carries an explicit "not persisted" warning so operators know to drive reissue_credential manually.

Task-store-server CLI. reference/services/task-store-server/src/eden_task_store_server/cli.py gains --checkpoint-import-credentials-dir (also read from EDEN_CHECKPOINT_IMPORT_CREDENTIALS_DIR). app.py build_app forwards it to make_app.

Tests. Storage-side: test_workers_credentials_reissued_on_import asserts the source plaintext is no longer accepted, every imported worker carries a fresh token on ImportResult.reissued_credentials, and each token authenticates against the receiver. Wire-side: three new tests in test_checkpoint_wire.pytest_import_persists_reissued_credentials_to_dir asserts file mode + contents + receiver-side verification, test_import_warns_when_no_credentials_dir asserts the unset-dir warning, test_import_then_worker_bootstrap_reuses_persisted_bearer is the end-to-end issue #150 happy-path (import drops a token; bootstrap_worker_credential reads it back and returns the bearer without needing an admin token — the persisted-token-verifies-via-/whoami branch).

Deployment posture. The new flag is opt-in: the reference compose stack leaves --checkpoint-import-credentials-dir unset by default (so post-import in the existing compose layout still surfaces the manual-reissue warning), and operators wire the flag against their own credentials substrate when their deployment is ready for it. The wire-layer mechanism is complete and exercised by tests; the deployment-side credentials-dir layout decision (shared dir vs per-host with sidecar import-dir, plus a parallel bootstrap_worker_credential extension to consult both) is a deployment-substrate concern operators decide independently of the wire-side machinery.

Retire X-Eden-Worker-Id test-fixture header (closes #148)

Backfill of a Phase 12a-1 deferral: the pre-12a-1b X-Eden-Worker-Id header was the wire's auth-disabled-mode worker-id source for the conformance harness (~25 scenario files) and the eden-wire roundtrip tests. The full retirement migrates the reference conformance adapter to spawn the task-store-server with --admin-token enabled, has the harness's default_workers fixture register the conventional worker ids (plus admins / orchestrators group memberships for the admin / orchestrator actor identities the scenarios name) through the admin bearer, and stashes each issued §6.3 registration token on WireClient so per-call as_worker=<wid> swaps the Authorization header for that worker's bearer. The scenario files drop their headers={"X-Eden-Worker-Id": …} callsites in favor of the new as_worker= kwarg. The wire server's _worker_id_from_request and _enforce_in_any_group fallbacks no longer read the header (auth-disabled mode collapses every caller onto the anonymous sentinel); the StoreClient stops sending the header on claim / submit; the control-plane app drops the parallel fallback. The eden-wire test_wrong_claimant roundtrip case spawns an auth-enabled mini-fixture inline because distinguishing claimants requires distinct authenticated identities. No spec change; internal cleanup with no contract-coverage impact (wave-5 RBAC scenarios already exercise the §13.3 ladder).

Rationalize executor 3-branch-per-variant flow to 2

Closes #169. Discovered during the 2026-05-22 manual demo: a single executor → evaluator → integrator cycle left three branches on Forgejo for one variant — <slug> (operator-leaking artifact from the eden-manual CLI's push step), work/<slug>-<variant_id> (the canonical executor branch), and variant/<variant_id>-<slug> (the integrator's squash). The bare-slug branch was redundant (same SHA as work/*) and never cleaned up. The two surviving branches also had mirrored field orders — work/<slug>-<variant_id> vs variant/<variant_id>-<slug> — so operators reading Forgejo had to mentally parse which prefix was which.

Fix 1 — eliminate the bare-slug branch. reference/scripts/manual-ui/eden-manual's push subcommand now takes a task_id (replacing the freeform <workdir> first positional) and pushes directly to refs/heads/work/<variant_id>-<slug> — derived from the claim record's variant_id plus the task's idea slug, both already known by that point in the executor flow. The --branch flag is removed (operator-leaking implementation detail per the issue framing). execution-submit's subsequent push is now idempotent against the same ref. No more bare-slug branches accumulate on Forgejo.

Fix 2 — flip work-branch field order. Reference impl's work-branch name changes from work/<slug>-<variant_id> to work/<variant_id>-<slug>, matching the integrator's variant/<variant_id>-<slug> shape from spec/v0/06-integrator.md §3.2. Pure rename — the spec leaves naming under work/* implementation-defined (chapter 03 §3.2), so no spec change. Five production sites updated together (eden-manual CLI, web-ui/routes/executor.py, executor-host/subprocess_mode.py, _common/scripted.py, eden-dispatch/workers.py); tests that pinned the legacy ordering (test_executor_routes, test_executor_e2e, test_executor_flow, test_evaluator_list_preview, test_executor_partial_write, test_executor_subprocess, web-ui/conftest) updated to assert the new shape. Pre-fix variants keep their old branch names (no rewriting of git history).

Fix 3 — operator-facing docs. docs/glossary.md §6 expanded with a clearer "work branch" vs "variant branch" distinction — work records what the executor wrote (tip + intermediate commits); variant records the integrator-produced squash. Reference-binding spec/v0/reference-bindings/worker-host-subprocess.md §3 step 1 surfaces the new field-order rationale (mirroring the integrator's shape). docs/user-guide.md §6 and .claude/skills/eden-manual-executor/SKILL.md updated for the new push CLI shape.

Naming under work/* remains a reference-impl convention; conformance assertions continue to treat variant.branch as opaque (per chapter 02 §1.3 variant_id-opacity rule).

Fix #124: Adminer-convenience variant_unpacked Postgres view

Add a Postgres view named variant_unpacked that the task-store-server creates at startup whenever the backend is Postgres (issue #124, cluster:observability). The view unpacks the variant.data JSON blob into typed scalar columns — one column per public Variant field plus one column per metric declared in the experiment's evaluation_schema — so operators in Adminer or psql can write SELECT correctness FROM variant_unpacked instead of (data::jsonb -> 'evaluation' ->> 'correctness')::real. The base variant table is untouched; the view is a read-only convenience layer that lives alongside it. Per-metric column types follow the schema's declared types: integer → Postgres integer, real → Postgres double precision (widened so high-precision floats round-trip without single-precision truncation), text → Postgres text. The eden_readonly role gets a GRANT SELECT on the view in addition to its existing GRANT on the base table. The view is DROP VIEW IF EXISTS + CREATE VIEW on every store open so a fresh experiment with a different evaluation_schema (the only legal schema-change path, per chapter 8 §4.2) picks up the new metric columns. New module reference/packages/eden-storage/src/eden_storage/_postgres_views.py carries the view-DDL helper; PostgresStore.__init__ calls it after _initialize_experiment (so the canonical evaluation_schema, persisted at first open, drives the view shape on every reopen); _grant_readonly_safe_set extends to the view via a new _READONLY_GRANT_VIEWS tuple. Integration tests against a real Postgres in reference/packages/eden-storage/tests/test_postgres_store.py cover common-column extraction, per-metric typed columns (via information_schema.columns), no-schema deployments, and the schema-replacement path; pure-Python tests in reference/packages/eden-storage/tests/test_postgres_views.py guard the Variant-field-coverage contract (so adding a field to Variant without extending the view fails CI) and metric-type mapping. Docs: new §5 in docs/operations/agent-readonly-db.md and a paragraph in docs/observability.md §2.7 point operators at the view; the InMemory and SQLite backends are unaffected (no JSONB, no view).

Add artifacts_uri to VariantSubmission + intended_evaluator to Idea (#164 + #165)

Two symmetric additive fields surfaced during the 2026-05-22 manual demo session.

#164 — VariantSubmission.artifacts_uri. Executors gain an optional submission field for non-committed output (build logs, coverage reports, screenshots, profiling captures) — the same channel evaluators already had via EvaluationSubmission.artifacts_uri. On accept the orchestrator writes the URI onto a new variant field, Variant.executor_artifacts_uri (spec/v0/02-data-model.md §9.1, 03-roles.md §3.4), disjoint from the existing Variant.artifacts_uri (evaluator-written via EvaluationSubmission.artifacts_uri per §4.4). The two fields are preserved independently — operators wanting multi-file uploads from a single producer wrap them in a tarball, matching the evaluator convention. Spec amendments cover the 02-data-model.md §9.1 field table + §8.2 reserved-names list, 03-roles.md §3.2/§3.4 (executor submission shape), and schemas/variant.schema.json (Draft 2020-12). artifacts_uri is NOT part of executor-submission resubmission equivalence — the first submission's value wins (mirrors §4.2's evaluator-side rule).

#165 — Idea.intended_evaluator. Symmetric with the existing Idea.intended_executor routing hint: ideators MAY name a worker/group whose task.target the orchestrator copies onto evaluation tasks created from variants of this idea (spec/v0/02-data-model.md §5.1, 03-roles.md §6.2 decision-type 3). Same claim-time resolution semantics, same admin-override discipline, same TaskTarget shape. The orchestrator's _insert_evaluation_task / create_evaluation_task both consult the originating idea's hint when no explicit target is supplied.

Spec, schemas, contracts. Spec edits to 02-data-model.md §5.1 + §8.2 + §9.1, 03-roles.md §3.2 + §3.4 + §6.2 decision-type 3, 09-conformance.md §5 index (new "Intended-evaluator flow-through" scenario group). JSON Schema additions to schemas/idea.schema.json (intended_evaluator) and schemas/variant.schema.json (executor_artifacts_uri). Pydantic mirror lockstep in reference/packages/eden-contracts/src/eden_contracts/{idea,variant}.py + accept/reject fixtures in tests/cases.py covering both new fields. Schema-parity + round-trip tests pass.

Storage layer. VariantSubmission gains artifacts_uri: str | None; submission_to_payload / submission_from_payload thread the field; _accept_execution writes Variant.executor_artifacts_uri; _reject_execution preserves the executor's URI on the error path (excluding validation_error, matching the evaluator-side rule). _validate_execution_acceptance dry-runs the executor URI through the model so an invalid URI surfaces as validation_error instead of crashing accept (test message string updated). _insert_evaluation_task + create_evaluation_task inherit idea.intended_evaluator when the caller doesn't supply an explicit target.

Wire + services. The wire layer changes are zero-additional-code because submissions go through submission_to_payload / _from_payload (covered) and ideas/variants go through Idea.model_validate / Variant.model_validate (covered by the model edits). eden-manual execution-submit gains --artifacts-uri; eden-manual ideation-submit passes intended_evaluator (and intended_executor) through verbatim from the ideas-file JSON. Web-UI executor form gains an artifacts_uri input.

Conformance. New scenario file conformance/scenarios/test_intended_evaluator.py with three tests covering absent / worker / group flows for the intended_evaluator → evaluation task target.

What this does NOT cover

  • Web-UI ideator form intended_evaluator input — the existing intended_executor plumbing is a per-row parallel-list form surface (~100 lines across forms.py, routes/ideator.py, and ideator_claim.html). Mirroring it for intended_evaluator is mechanically straightforward but out of scope for this PR; the CLI path (eden-manual ideation-submit) supports it today. Follow-up: #205.
  • Variant.artifacts_uri rename to evaluator_artifacts_uri — issue #164 recommended Option B with both fields renamed for clarity. This PR took the additive path (new executor_artifacts_uri field only; existing artifacts_uri retains evaluator-written semantics) to keep spec/manifest/checkpoint disruption contained. A follow-up rename would touch 06-integrator.md §4 (manifest field name), 10-checkpoints.md (export rewrites), and 5+ test fixtures. Follow-up: #206.
  • Conformance scenarios for VariantSubmission.artifacts_uriVariant.executor_artifacts_uri — the wire surface is exercised by reference storage tests; a dedicated conformance scenario would assert the field round-trips through read_variant for any IUT. Follow-up: #207.

Codify demo lessons into permanent process

Codify six lessons from the Phase 12 demo manual-UI session into durable process artifacts. New docs/triage.md anchors the issue-label scheme (type / triage / priority / cluster) with application rules, heuristics, the deferral-tracking rule (§4), and a standard backlog-view catalog. New .github/PULL_REQUEST_TEMPLATE.md requires a "What this does NOT cover" section (each deferred follow-up linked to a filed GH issue) and a "Fresh-operator walkthrough" section (required when the PR touches operator-facing surfaces). New .github/ISSUE_TEMPLATE/ carries bug-report + feature-request templates that surface the triage conventions at issue-file time. AGENTS.md gains a new "Deferrals MUST be tracked as GitHub issues" sub-section under "Recording chunk completions" (closes #149), a new "Substrate migrations need a same-PR audit" entry under "Plan-writing pitfalls" (closes #178), and cross-references from "Commit guidelines" / "Related docs" to the new templates + triage doc (closes #162). CONTRIBUTING.md gains a new "Operator-facing disciplines" section documenting the fresh-operator walkthrough discipline per surface change (closes #180) and the scheduled operator-dogfooding ritual (closes #179), plus a new "PR descriptions" section pointing at the template and explaining why each required section closes a class of past escape (closes #181). The bundle closes #149 / #162 / #178 / #179 / #180 / #181 together — the lessons reinforce each other (deferral tracking + triage labels + fresh-walkthrough notes + dogfooding all feed the same triaged backlog) and codifying them piecemeal would risk stale cross-references between artifacts.

Web-UI form-error handling — issues #134 + #158

Pre-fix, the web-ui's role routes (ideator / executor / evaluator) leaked two classes of uncaught exception as generic HTTP 500: a pydantic.ValidationError on Idea(**kwargs) (the demo trigger: slug="Spanish"), and any StorageError subclass outside the narrow pair the claim handlers caught. Bundled this PR closes both gaps:

  • #134 (Pydantic ValidationError). Idea(**kwargs) in reference/services/web-ui/src/eden_web_ui/routes/ideator.py _persist_idea_drafts now wraps construction in try/except ValidationError. The new shared helper format_validation_errors(exc, *, row=0) in reference/services/web-ui/src/eden_web_ui/forms.py maps pydantic's loc tuples to the form's per-field FormErrors shape (parse_idea_rows now returns a parallel draft_rows: list[int] so per-row errors land on the right row in the multi-row form). The canonical demo trigger — operator types slug="Spanish" (the form grammar accepts mixed-case but Idea.slug's ^[a-z0-9][a-z0-9-]*$ rejects it) — now re-renders the form with the field error and HTTP 400 instead of 500. The issue's table also named IdeaSubmission / VariantSubmission / EvaluationSubmission constructions as targets, but those types are @dataclass(frozen=True) (not Pydantic) — they cannot raise ValidationError, so no wrap is needed; Variant(**kwargs) is Pydantic but all its inputs at Phase 1 come from already-validated upstream sources, so no operator-recoverable failure exists. The codex-review pass caught this distinction; the implementation lands only the reachable case.
  • Artifact leak fix (regression caught by codex-review). Pre-fix, _persist_idea_drafts wrote the artifact file BEFORE constructing the Idea, so a ValidationError left an orphan file on disk that nothing referenced. Post-fix, the artifact URI is computed (not written) before validation, the Idea is constructed (validation barrier), and the artifact is written only after the Idea has passed validation. New regression test asserts the artifacts dir stays empty on slug rejection.
  • #158 (broadened StorageError coverage). The three claim handlers previously caught only (IllegalTransition, InvalidPrecondition), so a claim against a missing task (NotFound), an unregistered worker (WorkerNotRegistered), or a target-ineligible worker (WorkerNotEligible) leaked as 500. Each handler now catches the base StorageError, mapped through the extended WIRE_ERROR_NAMES table in reference/services/web-ui/src/eden_web_ui/routes/_submit_readback.py (now covers all 13 chapter-7 §9 closed-vocabulary subclasses), to a banner redirect (303 → /<role>/?banner=eden://error/<name>).

Tests. New reference/services/web-ui/tests/test_form_errors.py covers the canonical demo bug (uppercase slug), an underscore-slug variant, the artifact-leak regression, an executor happy-path smoke (to confirm the helper extractions don't disturb the working flow), and the three role claim handlers against a missing task_id — each asserting no 500 and the expected re-render / banner outcome. Existing 514 web-ui tests + full-repo tests continue to pass.

Slop-gate housekeeping. Extracted _drive_submit_phases (executor) and _build_and_submit_evaluation (evaluator) helpers from the per-route submit() functions to keep them under the 100-line complexity gate — both helpers are behavior-preserving.

Round-2 codex-review. Codex round 2 flagged that Variant (the executor's Phase-1 construction) IS a Pydantic model and is technically reachable via --experiment-id "" at service startup (would fail Field(min_length=1) mid-request). The higher-tier codification fix lands here instead of a route-side wrap: reference/services/_common/src/eden_service_common/cli.py gains a _non_empty_str argparse validator on --experiment-id so an empty/whitespace value is rejected at parse time across every reference service (orchestrator, ideator-host, executor-host, evaluator-host, task-store-server, web-ui). New unit tests assert the parse-time rejection.

Follow-up surfaced (out of scope here): the executor draft template (executor_claim.html) renders commit_sha field errors but has no renderer for the status radio group, so a parse_implement_form status rejection silently drops the field error on re-render. Pre-existing latent bug, untouched by this PR; file a follow-up when planning the executor form polish.

Fix #132 — eden-manual writes artifacts to shared substrate path

reference/scripts/manual-ui/eden-manual previously wrote idea-content artifacts to /tmp/eden-manual/artifacts/<idea_id>.md and stamped the wire submission with a host-local file:///tmp/eden-manual/artifacts/<idea_id>.md URI. The path is invisible to containerized consumers — the web-ui's /artifacts?uri=… route rejected it (out of --artifacts-dir jail) and a containerized executor could not read the idea content via idea.artifacts_uri. Mixed-mode demos (CLI ideator + web-ui or auto-host executor) broke at the first read; pure-CLI workflows happened to work because the CLI knew how to read its own /tmp path. Fix (issue #132 Option A): the CLI now writes the bytes to ${EDEN_EXPERIMENT_DATA_ROOT}/artifacts/<idea_id>.md (host side of the artifacts bind-mount) and stamps the URI with the container-internal path file:///var/lib/eden/artifacts/<idea_id>.md — the canonical reference every consumer (web-ui, scripted workers, the CLI itself) understands. A new _translate_artifacts_uri_to_host helper rewrites container paths back to the host bind-mount when the CLI reads back its own artifacts (cmd_show's inline-content view); legacy host-path URIs (/tmp/eden-manual/artifacts/…) pass through unchanged. EDEN_EXPERIMENT_DATA_ROOT is now required in .env (setup-experiment always populates it). The CLI's worker-state files (.claims.json, .credentials.json) and per-task git workdirs continue to live under /tmp/eden-manual/ — only the artifact bytes move to the shared substrate.

Migrate Gitea → Forgejo

Switch the in-network git server from Gitea (gitea/gitea:1.22.6-rootless) to Forgejo (codeberg.org/forgejo/forgejo:11-rootless). Pure rename — service + container names, env vars (GITEA_* → FORGEJO_*), CLI flags (--gitea-url → --forgejo-url), code identifiers, docs, spec text, manual-UI skills, smoke scripts, setup-experiment, and compose overlays all updated together. Operators with running deployments must compose down -v and re-run setup-experiment.sh against the new shape — no backwards-compat shims (per the project's pre-external-user posture). Historical artifacts (docs/archive/, docs/plans/review/) and changelog entries documenting prior chunks (Phase 10d follow-up B "Gitea-as-the-workers'-git-remote", etc.) are preserved as-is — they record what shipped at the time. The plan file docs/plans/eden-phase-13e-gitea-hardening.md keeps its filename (the chunk hasn't been implemented yet; impl PR will rename when it lands).

Phase 12c — Control plane (leases + multi-experiment + cross-experiment dashboard)

Adds the deployment-level coordination layer that completes Phase 12: a new eden-control-plane package + reference/services/control-plane/ reference service host the chapter 11 experiment registry, time-bounded leases (chapter 11 §4), and the deployment-scoped worker/group registry (chapter 11 §6). Orchestrator replicas subscribe via the new LeaseManager (chapter 11 §5), run the per-lease iteration loop, and apply the §5.3 partition self-fence. The web-ui surfaces a cross-experiment dashboard at /admin/experiments/. Six waves shipped: 00a1335 (spec — chapter 11 + chapter 02/03/07/09 amendments + lease.schema.json) / c586928 (eden-control-plane package — models + client + LeaseError vocabulary) / 945eafa (storage Protocol + InMemory + Postgres backends + FastAPI server) / 8990854 (orchestrator integration — LeaseManager + run_multi_experiment_loop + §5.2 startup probe + §5.3 self-fence + §5.5 release-after-drain) / 192f7ac (web-ui /admin/experiments/ dashboard + control-plane state-sync poller per chapter 11 §3) / wave 6 (conformance scenarios + docs).

Spec. New normative spec/v0/11-control-plane.md (Purpose / experiment registry per §2 / pull-based state sync with bounded staleness per §3 / lease shape + lifecycle + at-most-one-active-lease invariant per §4 / holder-instance fencing per §4.7 / orchestrator lease-ownership invariant + startup duplicate-worker_id probe + acquisition + renewal threads + control-plane-partition self-fence + hand-off + release-after-drain per §5 / deployment-scoped worker/group registry distinct from chapter 02's per-experiment registries per §6 / partial-success contract on import_checkpoint auto-register per §7 / implementation latitude per §8 / future amendments per §9). Chapter 02 §2.6 (new, informative) — control-plane registry projection (last_known_state, lease) observed only through control-plane endpoints, NOT on task-store-server's read_experiment. Chapter 03 §6.4 amendment (the pre-12c "v0 does not introduce a lease primitive" disclaimer distinguished: per-decision-type leases still absent; chapter 11's per-experiment ownership leases are a separate concern) + new §6.6 "Lease ownership" pinning that orchestrator decisions MUST be lease-gated under deployments running a control plane. Chapter 07 §1.3 carve-out for /v0/control/ paths + new §15 (Control-plane operations, 19 endpoints across experiment registry, lease ops, deployment-scoped worker/group registry, whoami) + §9 error vocabulary additions (lease-held-by-other 409, lease-not-held 410, lease-expired 410, lease-instance-mismatch 409). Chapter 09 §4 new parallel v1+multi-experiment conformance level with 11 scenario groups.

eden-control-plane package (reference/packages/eden-control-plane/) — ControlPlaneStore Protocol mirroring eden-storage's pattern; Pydantic models (ExperimentLease per lease.schema.json, RegisteredExperiment, request bodies for the four lease ops); typed LeaseError hierarchy (LeaseHeldByOther / LeaseNotHeld / LeaseExpired / LeaseInstanceMismatch) + raise_for_control_plane_envelope that routes the four chapter 11 §4.5 codes and defers every other code to eden_wire.errors.raise_for_envelope; ControlPlaneClient HTTP client covering all 19 §15 endpoints; InMemoryControlPlaneStore (thread-safe via single lock, clock injection for tests) + PostgresControlPlaneStore (per-op BEGIN ISOLATION LEVEL SERIALIZABLE READ WRITE; lease atomicity via INSERT … ON CONFLICT DO UPDATE … WHERE expires_at < EXCLUDED.acquired_at so "acquire or replace-expired" is one statement; PRIMARY KEY on experiment_id enforces at-most-one-lease-per-experiment); shared _credentials.py argon2id + grammar helpers (mirrors eden_storage._base's patterns at the deployment scope). 128 unit tests (35 store-protocol tests parametrized memory/postgres; Postgres skips without EDEN_TEST_POSTGRES_DSN).

Control-plane server (reference/services/control-plane/) — make_app(store, *, admin_token=None, lease_duration_seconds=30, state_poller=None) exposes the 19 §15 endpoints. Auth dispatch via control-plane-local auth.py (Principal + authenticate, parallel to eden_wire.auth but with the deployment-scoped registry as the verify_worker_credential source). Lease ops enforce orchestrators-group membership; acquire_lease rejects holder impersonation; renew/release reject non-holder workers; list_active_leases?holder=X admits the holder OR admin only. CLI announces EDEN_CONTROL_PLANE_LISTENING host=… port=… on --port 0. 28 wire-layer tests.

State-sync poller (state_sync.py) — chapter 11 §3 background thread polling each registered experiment's task-store-server every EDEN_STATE_SYNC_INTERVAL_SECONDS (default 30). WarningsTracker accumulates per-experiment consecutive-failure counts and emits the §3.4 stale-state warning string once threshold (default 10) is exceeded. app.py wires it: read_experiment_metadata injects warnings; acquire_lease triggers the §3.3 on-demand refresh. CLI lifespan hooks start/stop the daemon thread. 10 tests.

Orchestrator integration (reference/services/orchestrator/src/eden_orchestrator/lease_manager.py + multi_loop.py) — new LeaseManager (per-replica lease state owner: startup_probe() for §5.2 duplicate-worker_id detection; refresh() for renew/acquire/§5.3-self-fence; mark_drained_terminated() for §5.5; release_all() for graceful shutdown). New run_multi_experiment_loop outer driver iterates held experiments and dispatches run_orchestrator_iteration per held lease; per-experiment exceptions don't bring down the loop. cli.py dispatches: --control-plane-url set → multi-experiment driver; unset → existing single-experiment driver (backward compat). New CLI flags --lease-duration-seconds, --control-plane-admin-token. DuplicateWorkerInstancereturn 2. 19 unit tests (14 lease + 5 multi-loop).

Web-UI cross-experiment dashboard (reference/services/web-ui/src/eden_web_ui/routes/admin_experiments.py) — when --control-plane-url is set, registers five routes under /admin/experiments/: read-only dashboard listing every registered experiment + lease holder + warnings + actions; admin-form register/unregister/release-lease/select (all CSRF-protected). Session.selected_experiment_id (new, optional) records the operator's per-session selection; make_app(control_plane=None) keyword surfaces the cross-experiment routes + top-nav link. Single-experiment Compose deployments leave control_plane=None and the chapter 11 surface is hidden entirely. 13 tests.

Conformance. 11 new scenario files at conformance/scenarios/test_*.py backed by a new control_plane_handle + control_plane_client fixture pair in conformance/scenarios/conftest.py (spawns python -m eden_control_plane_server --store-url :memory: --port 0 per test). 28 of the 37 v1+multi-experiment cases pass directly against the wire surface; 9 are documented @pytest.mark.skip with rationale where the IUT contract (chapter 9 §6) doesn't admit direct assertion — auth gating requires the conformance harness to drive admin tokens (Lease-ownership authority); event-log non-emission requires an orchestrator running alongside the control plane (Lease decision gating event-log half + Multi-experiment dispatch event disjointness); two-service hand-off requires the task-store-server bound to the poller (State synchronization + Checkpoint import auto-register). The contracts those skipped assertions test are covered in-process by the wire-layer + integration tests; documented in each skipped scenario's reason. Citation discipline preserved — every test docstring cites a chapter-11 §X MUST. The reference impl is now v1+roles+orchestrator-substrate+lifecycle+checkpoints+multi-experiment conformant; 246/246 conformance scenarios pass (11 skipped — 2 pre-existing + 9 new documented skips).

Plan deviations explicitly recorded.

  • Plan §5.1 schema-table conflict resolved in favor of the §3.4 Option A architecture: the plan's spec table called for adding lease to experiment.schema.json, but the plan's chapter 11 §2.1 design + §3.4 dedicated-control-plane-Postgres architecture keep lease ONLY on the control-plane registry projection (NOT on the task-store-server's runtime experiment object). The cleaner design lands: lease.schema.json (new) defines ExperimentLease; experiment.schema.json is unchanged.
  • Per-route store swapping for the experiment switcher (plan §3.6) deferred: the switcher requires every existing per-experiment route (ideator, executor, evaluator, /admin/tasks, /admin/variants, /admin/workers, /admin/groups, …) to look up the active experiment from session state. The refactor is ~12 route files + their tests; out of scope this chunk. Wave 5 surfaces the session field + the cross-experiment dashboard records the selection; per-route store swapping is a follow-up, with a footer note on the dashboard explaining the v0 limitation to operators.
  • Deployment-scoped worker/group registry admin pages not shipped: the chapter 11 §6 registry remains admin-only via direct API calls. A parallel /admin/control/workers/ / /admin/control/groups/ surface is a follow-up.
  • Compose smoke compose-smoke-multi-experiment deferred: the chapter 11 surface is fully exercised by unit + wire + conformance tests. The existing 6 Compose smokes are unchanged in posture (when --control-plane-url is unset, behavior is unchanged) and continue to pass. A control-plane Compose service + multi-experiment smoke lands in a follow-up substrate chunk (Phase 13a Helm path is the natural home).

Codex review. Eight rounds. Round 1 (040af4a) surfaced 7 findings (4 BLOCKER + 2 MAJOR + 1 MINOR): control-plane worker bootstrap + worker-bearer plumbing through the multi-loop factory, per-experiment is_held() revalidation, list_experiments warnings injection, release-lease admin route removed (deferred to follow-up #104), and spec §3.4 wording. Round 2 (3aa32b4) surfaced 6 (3 BLOCKER + 1 MAJOR + 1 REGRESSION + 1 MINOR): per-experiment credential subdir isolation, atomic credential write, ControlPlaneWireClient extra_headers propagation, expiry-aware is_held, auth-disabled regression fix, error-vocab whitelist. Round 3 (62a84b8) caught two follow-ons: an auth-enabled fresh-deployment regression (unauthenticated whoami probe), and the §3.4 state-sync counter ordering (record_success only after store write commits). Round 4 (9315629) addressed 3 (2 MAJOR + 1 MINOR): per-experiment task-store credential release_for(experiment_id) on bootstrap failure (no fail-open under auth-enabled task-stores), expired-lease filtering at both storage backends, and probe-first control-plane bootstrap (auth-disabled restart with leftover persisted credential). Round 5 (6f88a47) reconciled a spec/impl split surfaced by round 4's lease-filter: chapter 11 §4.4 amended to make active-only lease visibility normative (matches chapter 02 §2.6's "currently-active lease holder" framing). Round 6 (bc88f1c) tightened the §4.4 wire shape: server emits explicit lease: null (was using exclude_none=True which omitted the key); POST /v0/control/experiments returns 200 on idempotent replay per chapter 07 §15 (was always 201). Round 7 (9345006) addressed 4 (2 MAJOR + 2 MINOR): atomic 201/200 via ControlPlaneStore.register_experiment returning (entry, created) (concurrent-register test added); §15.3 register_worker / register_group return 200 per the §6.1 verbatim-mirror; RegisteredExperiment.lease required-but-nullable (no Pydantic default — distinguishes "absent" from "explicit null"); explicit lease=null assertions added across registry conformance scenarios. Round 8 declared convergence — no remaining material findings.

Plan + per-wave commit record at docs/plans/eden-phase-12c-control-plane.md.

Phase 12b — Portable checkpoints — 2026-05-19

Adds a spec-defined export/import format for whole-experiment state: every Store-managed protocol object (tasks, ideas, variants, submissions, events, workers, groups, runtime experiment) plus a git bundle + experiment-config flow through a single self-describing application/x-eden-checkpoint+tar archive. The "portable" claim is anchored at chapter 10: a conforming archive emitted by any impl is importable by any other at the same spec_version. Five waves shipped (plus two codex-review fix rounds): 30f9219 (spec — new chapter 10 + manifest schema + chapter 02/07/08/09 amendments) / d459a21 (eden-checkpoint format package + ImportProvenance) / fa1f9ad (Store Protocol + backend export/import + V6 migration) / 7d75f94 (wire endpoints + StoreClient + auth amendment) / 1966b0c (wave-5 conformance + docs) / d1a4903 (wave-5 review fix — wire substrate plumbing + bundle cross-ref validation) / 827bb90 (round-1 fixes — 4 blockers + 1 major) / 6acbc87 (round-2 fix — stream-read manifest in recovery probe).

Spec. New normative spec/v0/10-checkpoints.md (Purpose / logical contents / format / tar transport envelope / manifest schema / atomicity contract per §6 / content-addressed artifacts per §7 — v0-deferred to a future v1+checkpoints+artifacts revision; see #102 / worker+group portability per §8 / round-trip semantics per §9 / lost-201 recovery probe per §10 / experiment-id collision + override per §11 / cross-reference validation per §12 / versioning per §13 / conformance per §15). Chapter 02 §1.5 deployment-local artifacts_uri note + §2.5 Experiment.imported_from runtime field. Chapter 07 §1.3 carve-out for /v0/checkpoints/ paths (X-Eden-Experiment-Id OPTIONAL; when present MUST equal post-rewrite experiment_id) + new §14 wire bindings (POST /v0/experiments/{E}/checkpoint, POST /v0/checkpoints/import returning 201 Created per round-1 fix, GET /v0/experiments/{E}) + §9 error vocabulary additions (checkpoint-invalid 400 / experiment-id-conflict 409 / spec-version-mismatch 409 / unsupported-checkpoint-version 409 / checkpoint-in-progress 409 OPTIONAL). Chapter 08 §1.9 Store Protocol ops (read_experiment subsumes the 12a-3 read_experiment_state projection + export_checkpoint + import_checkpoint). Chapter 09 §5 new parallel v1+checkpoints conformance level with 7 scenario groups (Checkpoint round-trip / cross-impl interop / preconditions / atomicity / Recovery probe / Checkpoint authority / Terminated-experiment round-trip).

Authority on the three §14 endpoints is gated on the LITERAL admin principal (per §13.1), not the admins group: bootstrap-class semantics — a fresh receiver deployment has no admins-group member registered before its first import_checkpoint (the import is what populates the worker/group registries); gating on the group would be uncallable.

eden-checkpoint workspace member (reference/packages/eden-checkpoint/) — Pydantic-mirror manifest model (CheckpointManifest / ManifestCounts / ManifestFiles / ExporterInfo); CheckpointWriter streaming tar producer with content-addressed dedup; CheckpointReader extracted-dir reader; extract_checkpoint(stream, dest_dir) using Python 3.12+ tarfile.data filter for path-traversal safety; typed errors CheckpointInvalid / UnsupportedCheckpointVersion / SpecVersionMismatch / ExperimentIdConflict / ExperimentIdMismatch; repo_bundle.py subprocess wrappers (create_bundle / verify_bundle / fetch_bundle / list_bundle_refs / verify_commits_reachable) with pinned author identity + commit.gpgsign=false. 49 unit tests.

eden-contracts. New ImportProvenance model carrying {checkpoint_exported_at, checkpoint_format_version}; Experiment.imported_from: ImportProvenance | None matches the schema's oneOf: [null, object] so the wire response carries the key uniformly (callers detect "present, no import" without branching on key-presence).

Store Protocol. export_checkpoint(stream, *, experiment_config, repo_bundle, exporter_info=None) snapshots state inside _atomic_operation (transactional-snapshot atomicity per chapter 10 §6); import_checkpoint(stream, *, as_experiment_id=None, extract_dir=None) validates spec_version + target experiment_id + store-must-be-empty preconditions BEFORE any write, then bulk-inserts every entity in a single atomic commit and sets imported_from per chapter 10 §10; shared logic in eden_storage._checkpoint. Bundle cross-reference validation (chapter 10 §12) covers all four items: variant.branch ref check via list_bundle_refs, plus per-SHA reachability via verify_commits_reachable for variant.commit_sha / variant.variant_commit_sha / idea.parent_commits. Workers imported with a $reissue-required$<random> sentinel credential hash so source plaintext tokens never authenticate (chapter 10 §8); receiver MUST run reissue_credential per worker before resuming. Default event-id counter reseeds past the imported max via _StoreBase._reseed_default_event_counter to prevent UNIQUE collisions on subsequent emit. SQLite + Postgres V6 migration: adds experiment.imported_from text NULL column. 13 parametrized backend tests + 2 cross-backend interop tests (SQLite↔InMemory) + 1 event-counter regression test.

Wire. Task-store-server CLI threads --experiment-config text + --repo-path into make_app so export composes real bytes (per-request git bundle create --all); when either is unset, route emits zero-byte placeholders (test posture). New @app.exception_handler(CheckpointError) covers the four checkpoint wire-error types; eden-checkpoint ExperimentIdMismatch is re-raised through the wire-layer class so a single class maps each wire type. The three §14 routes call require_admin(request). StoreClient.read_experiment() now hits GET /v0/experiments/{E} and parses the full Experiment shape including imported_from (replaces the prior /state-projection synthesis). StoreClient.export_checkpoint(stream) and StoreClient.import_checkpoint(stream, as_experiment_id=None) round-trip archive bytes; the import client implements the chapter-10 §10 3-outcome recovery-probe ladder (confirmed-success via imported_from match → synthesized response with recovery warning; divergence → ExperimentIdConflict; indeterminate → new IndeterminateImport exception). Recovery-probe stream-reads only manifest.json from the tar — no temp-dir extraction (round-2 fix). 15 wire-level tests.

Conformance. 7 new scenario files at conformance/scenarios/test_checkpoint_*.py backed by a new conformance/scenarios/conftest.py fixture pair (sender_wire_client + receiver_wire_client spawns a second IUT per test so the receiver is empty for import); test_checkpoint_authority spawns its own auth-enabled task-store-server inline (same pattern as test_worker_auth_enabled) to exercise the §14 admin gate. The cross-impl interop test is @pytest.mark.skip pending a --cross-impl-adapter flag; the chapter-9 §5 group's wire-observable contract is exercised through the round-trip tests (same adapter, distinct experiment_id). test_error_vocabulary extended with the four checkpoint error types as IMPL-optional (only emit when the impl claims v1+checkpoints). 218/218 conformance scenarios pass (2 skipped — cross-impl + 1 pre-existing). The reference impl is now v1+roles+integrator+checkpoints conformant.

Codex review. Round 0 stalled at 8h+; substantive partial finding (wire export emitting empty experiment_config/repo.bundle + missing §12 validator) addressed in d1a4903. Round 1 surfaced 5 findings (4 blocking + 1 major): import status 201, branch-ref check in §12 validator, event-id counter reseed, chapter 10 §7 v0-defer (follow-up #102), StoreClient.import_checkpoint recovery-probe ladder — all addressed in 827bb90. Round 2 verified all 5 fixes and surfaced 1 medium (recovery probe doing full tar extract) — addressed in 6acbc87 (stream-read manifest only). Convergence after 3 substantive review iterations.

Deferred to follow-ups. Artifact substrate rewrite (checkpoint:sha256:<hex> URI rewriting): chapter 10 §7 softened to v0-deferred ("target shape (informative for v0; MUST for v1+checkpoints+artifacts)") with the future revision tracked at #102. Credential-reissue side-channel surfacing — wave 5 returns warnings on the import response but the reference deployment doesn't automatically drive reissue_credential from the receiver's wire layer yet. HTTP chunked transfer-encoding — chapter 10 §6 RECOMMENDS it for large checkpoints; the wave-4 binding materializes to an in-memory buffer; future revision. Compose smoke for checkpoint — no compose-smoke-checkpoint job yet; deferred until the deployment-substrate integration lands. Cross-impl interop conformance test — skipped pending a second IUT adapter + --cross-impl-adapter plumbing.

Plan + per-wave commit record at docs/plans/eden-phase-12b-portable-checkpoints.md. Codex-review artifacts at docs/plans/review/eden-phase-12b-portable-checkpoints/impl/.

Phase 12a-1c — Task transparency + lineage navigation — 2026-05-18

Read-only UI enrichments closing the transparency gap surfaced in the 2026-05-13 manual UI session: operators can now inspect upstream context (idea content, variant artifacts, attribution, lineage neighbors) without claiming first (which is irreversible during the TTL). Pure UI surface — zero spec changes, zero new wire endpoints, zero new storage operations. All data already on the wire post-12a-1; this chunk wires it into the views. New _lineage.py helper module with five per-page view models (IdeationTaskLineage / ExecutionTaskLineage / EvaluationTaskLineage / IdeaLineage / VariantLineage) + matching lineage_for_<kind> builders (one-hop only; collections cap at 20; transport-failure budget per page). Admin task / variant detail extensions: new attribution section (target, created_by, submitted_by, claim.worker_id — all worker-id slots hyperlinked to /admin/workers/<id>/) + new lineage section dispatched on task.kind; variant detail gains a lineage section + hyperlinks the parent-idea slug to /admin/ideas/<id>/. Executor + evaluator list previews surface per-task idea / variant context behind <details> blocks via the existing read_idea_content / read_variant_artifact trust-boundary helpers (1 MiB cap, file:// scheme, contained in --artifacts-dir). New /admin/ideas/ list + detail views (filterable by state; full record + inline content + lineage to originating ideation task + spawned variants). Plan + 5 codex-review rounds at docs/plans/eden-phase-12a-1c-task-transparency.md. Merged via PR #97.

Phase 12a-1i — spec amendment + server enforcement for executor no-op variant rejection — 2026-05-18

Promotes docs/design/executor-no-op-variant-rejection.md (landed in #89) to a normative spec amendment + reference-impl server-side enforcement + v1+roles conformance scenario. Closes #83. Spec amendments: spec/v0/03-roles.md §3.3 — new "Non-no-op variant" bullet on the Worker-branch invariants list (a VariantSubmission with status == "success" MUST have a git tree of commit_sha that differs from the git tree of at least one entry in idea.parent_commits); spec/v0/03-roles.md §3.4 — rejection rule (a no-op submission MUST be rejected; where the rejection surfaces is implementation-defined per chapter 9 §6 latitude; when surfaced via a wire error envelope the type MUST be eden://error/no-op-variant); spec/v0/04-task-protocol.md §4.2 — cross-reference clarifying that role-side success-contract invariants MUST be enforced server-side independently of the §4.1 atomic claim-match and §4.2 idempotency rules; spec/v0/07-wire-protocol.md §9 — eden://error/no-op-variant added to the closed v0 error vocabulary (HTTP 409); spec/v0/09-conformance.md §5 — Executor submission group scope extended. Reference impl: new NoOpVariant exception in eden_storage.errors + wire mapping in eden_wire.errors; _StoreBase gains an optional tree_resolver constructor parameter (forwarded by all three backends); two enforcement layers — a SHA-equality fast path (always on, no git dep) and a tree-identity check (fires when a tree_resolver is wired, catches empty-commit-on-parent); eden_task_store_server gains a build_tree_resolver backed by eden_git.GitRepo.commit_tree_sha + new --repo-path CLI flag. The eden-manual CLI's pre-existing client-side no-op check is retained as defense-in-depth.

Phase 12a-3 — Lifecycle policy — 2026-05-18

Phase 12a-3 complete (lifecycle policy). Closes the 12a-1 / 12a-2 lifecycle followups: termination is now a deployment-supplied policy, experiments carry an observed state lifecycle, ideas carry an intended_executor routing hint, and operator-driven create_task(kind=execution) is admin-OR-orchestrators-gated. Six waves shipped: c7fc68b (spec) / 58b0da9 (Pydantic + storage) / 8f91612 (wire endpoints + execution-task authority lift) / be3cf90 (dispatch driver + orchestrator service) / b9790db (web UI) / c062eb8 (conformance scenarios). What's covered: chapter 02 §2.1 / §2.3 drop the four legacy termination fields (max_variants / max_wall_time / convergence_window / target_condition); new §2.5 experiment-lifecycle state ("running" / "terminated"; one-way transition; observed-only field separate from declarative experiment-config); §2.4 dispatch_mode gains a fifth normative key termination defaulting to "manual" for backward compat with pre-12a-3 deployments; §5.1 idea gains optional intended_executor: TaskTarget | None (reuses the 12a-1 tagged-target shape). Chapter 03 §6.2 gains decision-type 0 (termination consulted first each iteration; returns Continue / Terminate(reason); policy-fault tolerance via experiment.policy_error; integration drain continues post-terminate); §6.4 multi-instance safety extended (terminate is exact-idempotent; the race-resolution §6.4.1 prose pins event ordering as not pinned between experiment.terminated and variant.integrated while cardinality + final state ARE pinned); §6.5 manual mode broadens kind=execution authority from orchestrators-only to admins-OR-orchestrators now that idea.intended_executor gives operators a non-fungible routing seed. Chapter 04 terminated-experiment guard on create_task + claim (§3.5 step 0); new §8 lifecycle ops (terminate_experiment is the public composite-commit; update_experiment_state is an internal Store primitive, NOT a wire endpoint); §9-§13 renumber. Chapter 05 §2 transactional invariant extended to experiments; §3.4 registers experiment.terminated (reason, terminated_by) + experiment.policy_error (policy_kind, error_type, error_message — exempt from §2 per its fault-vs-state-change shape). Chapter 07 §2.9 new POST /v0/experiments/{E}/terminate (admin-group-gated, body {reason} only — server stamps terminated_by) + companion GET /state. Chapter 08 §1.8 experiment persistence + lifecycle ops + Store-layer guard codification. Chapter 09 §5 three new index groups (Termination decision / Experiment lifecycle / Intended-executor flow-through) covered by 14 wave-6 conformance scenarios. New Store Protocol operations: terminate_experiment(reason, terminated_by) (composite commit; idempotent on terminated state — first reason wins), read_experiment_state() (returns "running" / "terminated"), update_experiment_state(state) (internal primitive), read_experiment() (returns the full Experiment runtime object), emit_policy_error(policy_kind, error_type, error_message) (single-event append; exempt from §2 invariant). create_execution_task gains an admin-override target: TaskTarget | None parameter; when omitted, the auto-dispatch flow-through copies idea.intended_executor to task.target. Pydantic bindings: ExperimentConfig drops the four legacy fields; new Experiment model (mirrors the new experiment.schema.json); Idea.intended_executor: TaskTarget | None; DispatchMode gains termination field; new ExperimentTerminatedEvent + ExperimentPolicyErrorEvent. SQLite + Postgres v5 migration: adds state text NOT NULL DEFAULT 'running' + created_at text NOT NULL columns; UPDATEs existing dispatch_mode rows to include the new termination: "manual" key. Wire schemas: new terminate-request.schema.json (body {reason}, additionalProperties: false) + experiment-state-response.schema.json + experiment.schema.json; existing dispatch-mode-{request,response}.schema.json gain the fifth termination key. New IndeterminateTermination error in reference/packages/eden-wire/src/eden_wire/client.py with a GET /state read-back ladder: idempotency means state="terminated" post-fail → confirmed success (synthetic Experiment); state="running" or read-back failure → indeterminate. Reference termination-policy library eden_dispatch.termination: Continue / Terminate(reason) discriminated union; TerminationPolicy = Callable[[ExperimentStateView], TerminationDecision]; five reference policies — never_terminate (default), max_variants_policy(target), max_wall_time_policy(duration), convergence_window_policy(metric, window, direction), target_condition_policy(metric, threshold, direction). Dispatch driver: run_orchestrator_iteration gains decision-type 0 (termination, consulted FIRST); a transient read_experiment_state failure fails closed (skips operational decisions for one iteration; integration drain still runs); policy fault → experiment.policy_error + treat-as-Continue (best-effort via Store.emit_policy_error; StoreClient stub falls back to structured log). ExperimentStateView gains four termination-slice fields (attempted_variant_count, experiment_created_at, recent_evaluations, latest_evaluation). Orchestrator service: new --termination-policy <module:callable> CLI flag defaulting to eden_dispatch.termination:default_termination_policy. _ALL_MANUAL fail-closed dispatch_mode also gains termination="manual" so a transient read can't trigger an unintended termination check. No special loop-exit branch for terminated: integration-drain is exact-finite, so quiescence-counter naturally fires once the drain completes. Web UI: intended_executor per-row dropdown on the ideator draft form (kind=none/worker/group + id input); new /admin/ideas/ list + /admin/ideas/{id}/ detail with admin-driven create-execution-task form (target=none inherits idea.intended_executor; explicit target overrides); new /admin/experiment/ lifecycle dashboard with terminate form on running state; idempotent-re-terminate UI distinguishes "we won" from "already-terminated" via event-log readback; admin dashboard banner when state==terminated. Conformance: 14 new scenarios resolve the wave-1 chapter 9 §5 index groups. The reference impl is now v1+roles+orchestrator-substrate+lifecycle conformant: 198/198 scenarios pass. The chapter-7 binding's pre-existing wave-3 wire tests in reference/packages/eden-wire/tests/test_lifecycle_wire.py cover the §6.5 authority ladder (admin / orchestrators / random worker matrix). 6-wave plan + per-wave commit record at docs/plans/eden-phase-12a-3-lifecycle-policy.md.

Phase 12a-1f — Substrate read access for ideator + evaluator agents — 2026-05-16

Phase 12a-1f complete (substrate read access for ideator + evaluator agents). Three read-side substrates (git, artifact server, Postgres event log) are now exposed to user-supplied *_command subprocesses spawned by the ideator + evaluator hosts so an agentic role implementation can explore experiment state without N+1 wire round-trips. What's covered: (a) the ideator host gains the same --repo-path / --gitea-url / --credential-helper triple executor + evaluator already use plus a new per-host ${EDEN_EXPERIMENT_DATA_ROOT}/ideator-repo bind-mount (subprocess-mode only — the scripted ideator doesn't read git; mirrors 12a-1g's per-host-repo bind-mount discipline); clone-on-startup mirrors Phase 10d follow-up B (_ensure_repo_clone at reference/services/ideator/src/eden_ideator_host/cli.py). (b) The task-store-server gains a reference-only GET /_reference/experiments/{experiment_id}/artifacts/{path:path} route (reference/packages/eden-wire/src/eden_wire/server.py) — always mounted; returns 503 eden://reference-error/artifact-serving-disabled when --artifacts-dir is unset. Route uses a descriptor-relative component walk (Linux openat2(RESOLVE_BENEATH) equivalent in stdlib): each path component opens with O_NOFOLLOW, anchored by the prior step's fd via dir_fd=, plus a pre-open lstat for OS-portable symlink rejection (macOS returns ENOTDIR rather than ELOOP for O_DIRECTORY|O_NOFOLLOW on a symlinked-dir). Closes intermediate-component TOCTOU. Fixed-bytes Response(content=…) (NOT FileResponse, which would re-open the path at body-write and re-open the TOCTOU window) with a 1 MiB cap; safe-delivery headers on every 200 (Content-Disposition: attachment + X-Content-Type-Options: nosniff + Content-Type: application/octet-stream — defeats stored-XSS via attacker-controlled .html / .svg artifacts). Auth-first (bearer-auth via eden_wire.auth.authenticate BEFORE any filesystem call) — admin OR worker bearer accepted. Three new WireReferenceError subclasses at reference/packages/eden-wire/src/eden_wire/errors.py (InvalidPath / ArtifactTooLarge / ArtifactServingDisabled) wired via a new @app.exception_handler(WireReferenceError) that delegates to the existing envelope_for_reference_error. (c) The task-store-server also provisions an eden_readonly Postgres role at startup when --readonly-password is set (provision_readonlyensure_readonly_role). Idempotent REVOKE-then-GRANT against current_database() / current_schema() (NOT hard-coded eden / public, so CI's EDEN_TEST_POSTGRES_DSN=postgresql://postgres@.../postgres + per-test schemas via search_path work transparently). Table-level SELECT on experiment, task, submission, idea, variant, event, worker_group, group_membership, schema_version; column-level SELECT on worker(worker_id, data) excluding credential_hashSELECT * FROM worker fails because the parser expands * to all columns. ALTER DEFAULT PRIVILEGES is intentionally OMITTED so future schema bumps that add a credential-bearing column don't silently re-expose it. (d) Subprocess env-var threading: EDEN_REPO_DIR / EDEN_ARTIFACT_URL / EDEN_ARTIFACT_PATH_ROOT / EDEN_READONLY_STORE_URL flow into spawned children via a new add_substrate_arguments(parser) helper + SubstrateArgs dataclass + substrate_args_for_exec_mode suppression at reference/services/_common/src/eden_service_common/cli.py. DooD-mode (--exec-mode docker) deliberately SUPPRESSES all four substrate keys because sibling containers can't resolve compose-internal hostnames (task-store-server:8080, postgres:5432) without --network plumbing in wrap_command; the host logs a WARN at startup. Re-enablement is deferred to a follow-up sub-chunk (12a-1f-followup-A). Spec posture: no normative chapter changed; only the informative spec/v0/reference-bindings/worker-host-subprocess.md is extended with the four new env-var rows in §1.1 + a new §9 substrate-access section. Chapter 7 §7 normative error vocabulary unchanged; chapter 7 §12's existing eden://reference-error/... namespace is the home for the three new types. Compose: compose.yaml bind-mounts ${EDEN_EXPERIMENT_DATA_ROOT}/artifacts:/var/lib/eden/artifacts:ro on task-store-server (12a-1g converted artifacts from a named volume) + passes --artifacts-dir /var/lib/eden/artifacts + --readonly-password ${EDEN_READONLY_PASSWORD}; the four substrate env vars are set on ideator + evaluator services ONLY in compose.subprocess.yaml (scripted-mode doesn't get them), and the subprocess overlay also adds the ${EDEN_EXPERIMENT_DATA_ROOT}/ideator-repo bind-mount on the ideator-host. compose.docker-exec.yaml takes no changes (DooD scope-out per §8.9). setup-experiment.sh creates the ideator-repo substrate dir, generates + preserves EDEN_READONLY_PASSWORD, and writes the rendered EDEN_READONLY_STORE_URL / EDEN_ARTIFACT_URL / EDEN_ARTIFACT_PATH_ROOT to .env. Tests: 32 artifact-route tests (test_artifact_route.py) covering auth-first via FS-entry-point sentinels, malformed-path 400, symlink-out-of-root 403, TOCTOU terminal-component swap, intermediate-component symlink swap race, safe-delivery headers on every 200 path, 1 MiB boundary + cap, 503-when-disabled, experiment-id-mismatch; 12 readonly-role tests (test_postgres_readonly.py) covering idempotency, password rotation, INSERT/UPDATE/DELETE/DDL denied, SELECT *-fails-on-worker, hardening-migration (legacy over-grant REVOKEd), default-privileges-NOT-installed; 7 task-store-server tests (test_artifacts_cli.py + test_readonly_provisioning.py); 9 substrate-CLI tests (test_substrate_arguments.py); 8 host-CLI env-threading + DooD-suppression tests (test_ideator_subprocess_env.py + test_evaluator_subprocess_env.py); 3 ideator-repo-init tests (test_ideator_repo_init.py). compose-smoke-subprocess gains three substrate-access assertions (git repo non-empty post-quiescence; artifact route returns 200 for an ideator-written content file; psql against EDEN_READONLY_STORE_URL confirms SELECT works, INSERT fails, SELECT *-on-worker fails, SELECT credential_hash fails). Out of scope (followups): executor-agent substrate access; DooD-mode env-var forwarding (needs --network plumbing); per-worker Gitea tokens with branch ACLs (Phase 13e); chart-managed managed-Postgres (Phase 13c); full Backend Protocol replacing the tactical artifact route (Phase 13d — the 12a-1f route is throwaway by design). Operator docs at docs/operations/agent-substrate-access.md + docs/operations/agent-readonly-db.md. Plan at docs/plans/eden-phase-12a-1f-substrate-access.md.

Phase 12a-2 — Orchestrator as role — 2026-05-16

Phase 12a-2 complete (orchestrator as role). The orchestrator is now a role (not a singleton process): zero, one, or many auto-orchestrator instances may run concurrently, each authenticating as a registered worker in the reserved orchestrators group. Decisions are gated per-experiment by a dispatch_mode field, and operators have first-class wire ops to reassign tasks and flip dispatch_mode on a live experiment. Eight waves shipped — ce85dd6 (spec) / 48c07c6 (pydantic + storage) / eaa36e4 (eden-wire + §3.7 authority) / 81ae6d7 (dispatch + orchestrator service) / abef67b (web UI) / 359d74a (conformance scenarios + harness) / 2dd74bd (compose + setup-experiment + smokes) / wave 8 (docs). What's covered: chapter 02 §2.5 (dispatch_mode — per-experiment object with four normative keys ideation_creation / execution_dispatch / evaluation_dispatch / integration, each "auto" (default) or "manual"; partial-merge semantics; unknown keys tolerated) + §7.5 (reserved-group carve-out for admins and orchestrators — names reserved against shadowing; deployments register them via register_group like any other group); chapter 03 §6 (new — orchestrator role contract: §6.1 dispatch_mode-gated invocation, §6.2 four decision types — ideation-task creation, execution-task dispatch, evaluation-task dispatch, integration, §6.3 authority boundary, §6.4 multi-instance safety with exact-idempotent vs bounded-overshoot split, §6.5 manual mode); chapter 04 §6 (new — reassign_task op: pending → field update + single task.reassigned event; claimed → composite-commit task.reclaimed(cause=operator) + task.reassigned; claimed-execution with in-flight starting variant → composite also emits variant.errored; submitted/terminal → 409 invalid-precondition with no partial state) + §7 (new — update_dispatch_mode op: atomic partial-merge; idempotent no-diff flip emits no event); chapter 05 §3.1 (new task.reassigned event with required-nullable new_target, reason, reassigned_by payload) + §3.4 (new experiment.dispatch_mode_changed event carrying post-update full state + changed diff + updated_by); chapter 07 §§2.7-2.8 (wire endpoints — POST /tasks/{T}/reassign body {new_target, reason}, PATCH /dispatch_mode partial object, GET /dispatch_mode companion read) + §13.3 (group-gated authority added to the bearer-classification ladder); chapter 09 §5 (four new index groups — Orchestrator role contract / Multi-instance safety / Reassignment / Dispatch mode). The §3.7 authority matrix is the canonical operator reference: create_task(kind=ideation|evaluation) requires worker in admins OR orchestrators; create_task(kind=execution) requires orchestrators only (operator-driven kind=execution deferred to 12a-3 / intended_executor); accept / reject / integrate_variant require orchestrators; reassign_task / update_dispatch_mode require admins. The deployment-admin bearer (literal "admin" principal) is bootstrap-only — it registers workers, registers groups, reissues credentials. It MUST NOT drive business-op routes; the server returns 403 forbidden for those attempts. New Store Protocol operations: reassign_task(task_id, new_target, *, reason, reassigned_by) / read_dispatch_mode() / update_dispatch_mode(updates, *, updated_by), plus tightened at-most-one-live invariants in _insert_execution_task / _insert_evaluation_task / create_execution_task / create_evaluation_task (§6.4 exact-idempotent: a second concurrent create observes the first's commit and either no-ops or raises). Pydantic bindings: DispatchMode model (in reference/packages/eden-contracts/src/eden_contracts/config.py); TaskReassignedEvent + ExperimentDispatchModeChangedEvent with a model_serializer(mode="wrap") that keeps new_target JSON-null on exclude_none=True round-trips (required-nullable field). SQLite + Postgres v3 migration adding dispatch_mode text NOT NULL to the experiment table; default value is the all-auto JSON literal. reassigned_by / updated_by attribution stamping: the binding stamps these server-side from the authenticated principal; wire body MUST NOT carry them (wire schemas use additionalProperties: false). Wire schemas: three new files under spec/v0/schemas/wire/reassign-request.schema.json (required-nullable new_target), dispatch-mode-request.schema.json (partial object), dispatch-mode-response.schema.json (full state, all four keys required). Error types: new IndeterminateReassign and IndeterminateDispatchModeUpdate in reference/packages/eden-wire/src/eden_wire/client.py — read-back ladders for transport-indeterminate POST/PATCH failures, parallel to the §5 integrate_variant ladder. Reference policy module: eden_dispatch.policies exposes maintain_pending(target, max_total=None) (bounded-overshoot per §6.4 — returns min(max(0, target - pending), max(0, max_total - total))), fixed_total(N) (one-shot equivalent of the retired --ideation-tasks N static seed), and default_policy() (the orchestrator-CLI default factory; reads EDEN_IDEATION_POLICY_TARGET_PENDING and EDEN_IDEATION_POLICY_MAX_TOTAL from the environment). The ExperimentStateView dataclass (in reference/packages/eden-dispatch/src/eden_dispatch/state_view.py) is a snapshot facade over five counters (pending_ideation_count / in_flight_ideation_count / total_ideation_count / running_variant_count / integrated_variant_count) the policy callable consults; built per-iteration with no caching. Orchestrator service: CLI drops --ideation-tasks; adds --ideation-policy <module:callable> (default eden_dispatch.policies:default_policy). The loop reads dispatch_mode at iteration start (all-auto fallback on transport blip) and forwards it to run_orchestrator_iteration, which gates each of the four §6.2 decision types; finalize (accept/reject of submitted tasks) is intentionally NOT gated — the spec lists creation / dispatch / integrate as the four gated decisions, and gating finalize would starve workers waiting for terminal transitions. A new _ensure_orchestrators_membership startup helper (in reference/services/orchestrator/src/eden_orchestrator/cli.py) defensively registers the orchestrators group (swallowing AlreadyExists) and adds the configured worker_id (idempotent on existing membership), with NotFound race recovery — defense-in-depth alongside setup-experiment's bootstrap. Web UI: new /admin/dispatch-mode/ page (4-toggle table driving Store.update_dispatch_mode) and /admin/tasks/{T}/reassign page (target-kind radios + per-kind selects populated from list_workers / list_groups + manual override + reason field). Both follow the chunk-9e auth-first POST + closed-allowlist banner discipline; namespace-disjoint from a parallel 12a-1b delegate's /admin/workers/ and /admin/groups/ routes (collision-guard tests pin this). Compose plumbing: compose.yaml drops --ideation-tasks ${EDEN_IDEATION_TASKS:-3} (the wave-4 CLI retirement made this stale and would have broken compose-smoke on first push) and threads EDEN_IDEATION_POLICY_TARGET_PENDING + EDEN_IDEATION_POLICY_MAX_TOTAL into the orchestrator's environment: block. New compose.multi-orchestrator.yaml overlay adds a second orchestrator container (worker_id=orchestrator-2) with per-replica repo + credentials volumes. New EDEN_ORCHESTRATOR_WORKER_ID (default orchestrator) and EDEN_ADMINS_INITIAL_MEMBER (default operator) env vars on .env.example. setup-experiment.sh gains a reserved-group bootstrap step after the seed: brings up task-store-server (with its postgres + blob-init deps), then makes four admin-token-authenticated wire calls inside the container, all idempotent on existing record per 12a-1 §D.1 / §D.2 — register_group("orchestrators") (accept 200 or 409), register_group("admins") (same), register_worker(${EDEN_ADMINS_INITIAL_MEMBER}), add_to_group("admins", ${EDEN_ADMINS_INITIAL_MEMBER}). The orchestrators group is created empty; auto-orchestrator instances populate it themselves at startup via the helper above. Smoke scripts: existing smoke.sh / smoke-subprocess.sh / smoke-subprocess-docker.sh / e2e.sh all pin EDEN_IDEATION_POLICY_MAX_TOTAL so the policy quiesces deterministically; smoke.sh bumps the registered-workers floor from ≥4 to ≥5 (initial admin worker now seeded) and adds assertions that /groups/admins contains the initial admin + /groups/orchestrators contains the auto-orchestrator. Two new smoke scripts: smoke-manual-mode.sh (flips execution_dispatch to manual, asserts the orchestrator skips the gated decision while an idea reaches ready, then flips back and asserts catch-up) and smoke-multi-orchestrator.sh (uses the overlay, asserts both worker_ids end up in orchestrators, chaos-kills the primary, asserts the secondary drives the experiment to quiescence with exactly 3 variant.integrated + 3 execution task.completed + 3 evaluation task.completed). The e2e_drive.py UI driver now discovers task IDs dynamically from the /ideator/ page (policy-driven UUID-suffixed ids replaced the fixed ideation-0001..0004 shape) and adds two new drills: a _dispatch_mode_toggle_drill (flips integration to manual via the web UI, verifies the form reflects state, flips back) and a _reassign_drill (reassigns one pending task to {kind: worker, id: ideator-1} — the targeted-claim path means the headless ideator-1 still completes the task so end-state variant count is unaffected). Conformance: four new scenario files / 26 tests at conformance/scenarios/test_orchestrator_role_contract.py + test_multi_instance_safety.py + test_reassignment.py + test_dispatch_mode_wire.py resolve the four wave-1 citation-check expected failures. Harness gains a PATCH verb + dispatch_mode_path() helper on WireClient plus three _seed.py helpers (reassign_task, update_dispatch_mode, read_dispatch_mode). The auth-disabled posture (reference adapter runs without --admin-token) means new scenarios don't exercise the §3.7 403-forbidden ladder — that surface is covered by wave-3 wire tests in reference/packages/eden-wire/tests/test_reassign_dispatch_wire.py. The reference impl is now v1+roles+orchestrator-substrate conformant: 184/184 scenarios pass. CI: two new jobs compose-smoke-manual-mode + compose-smoke-multi-orchestrator (not branch-protected yet; same posture chunks 10c / 10d / 10e took for newly-added jobs — bump to required-status after staying clean on main for ~2 weeks). Deferred to 12a-3: intended_executor on ideas (would enable operator-driven kind=execution task creation under §3.7's currently orchestrators-only gate) + an explicit termination-policy contract (today's EDEN_MAX_QUIESCENT_ITERATIONS env stays as the operational escape hatch). The bounded-overshoot §6.4 N * T math is exercised at the unit-test level in reference/packages/eden-dispatch/tests/test_dispatch_mode_gating.py; a wire-level conformance test for it would require an orchestrator-bearing IUT (chapter 9 §6 pins the suite to the chapter-7 binding) and is left for a future v1+roles+orchestrator-instance level if/when that becomes useful. 8-wave plan + per-wave commit record at docs/plans/eden-phase-12a-2-orchestrator-as-role.md. Operator playbooks under docs/operations/ cover dispatch-mode flipping, task reassignment, multi-orchestrator deployment, and initial-admin credential recovery.

Phase 12a-1g hotfix — container-side rm in smoke-script teardown — 2026-05-15

Hotfix for a CI regression introduced by 12a-1g: after switching substrates to host bind-mounts, all four compose smokes started failing CI on teardown with rm: cannot remove '/tmp/eden-smoke-XXXXXX/.../hooks/update.sample': Permission denied. Containers wrote into the bind-mounts as their own uid (postgres=70, gitea/eden=1000) and created subdirectories with their own umask; the smoke trap's rm -rf runs as the host CI runner (uid 1001 on GH Actions) and cannot delete files inside container-created subdirs. macOS Docker Desktop's VFS uid-translation layer masked the failure during local validation. Fix: before the host-side rm -rf, run a sibling alpine:3.20 container as root with the bind-mount mapped at /cleanup and find /cleanup -mindepth 1 -delete. Applied identically to all four smoke scripts (smoke.sh, smoke-subprocess.sh, smoke-subprocess-docker.sh, e2e.sh).

Phase 12a-1h — port eden-manual CLI to post-12a-1 auth + no-op variant design — 2026-05-14

Closes the manual-UI CLI auth gap flagged in PR #67's commit message. Phase 12a-1 (PR #78) shipped per-worker bearer auth and retired the per-claim opaque token; the rest of the stack was ported in that wave but reference/scripts/manual-ui/eden-manual was deliberately left as a known gap. This PR closes the auth half. Changes: reference/scripts/manual-ui/eden-manual — env-var rename (EDEN_SHARED_TOKENEDEN_ADMIN_TOKEN); _wire() gains bearer + swallow_401 kwargs; new _worker_bearer() ladder (whoami-verify → register-or-reissue); per-worker credentials at /tmp/eden-manual/.credentials.json mode 0600; claim/submit bodies drop the retired "token" field; cmd_push tolerates a clean working tree; new no-op variant guard in cmd_execution_submit. New design doc at docs/design/executor-no-op-variant-rejection.md (~215 lines): RFC for the spec amendment + reference-store enforcement + v1+roles conformance scenario (subsequently promoted in 12a-1i). Skill docs (3 files): eden-manual-{ideator,executor,evaluator}/SKILL.md updated for post-12a-1 auth shape + .env-divergence troubleshooting note.

Phase 12a-1d — rename rationale → content for the idea's markdown body — 2026-05-14

Cosmetic rename: the markdown body the ideator writes was previously called "rationale," which only conveys the WHY of an idea. The document actually carries both the WHAT-to-attempt and the WHY; "content" covers both dimensions and reads more naturally in UI labels. Pure file-name / helper-name / UI-label / spec-convention rename — Idea Pydantic record stays; artifacts_uri field stays; wire endpoints unchanged. Changed: filename rationale.mdcontent.md; helper read_idea_rationaleread_idea_content; identifiers rationale_path / _RATIONALE_MAX_BYTES / rationale_textcontent_*; UI labels "Rationale" → "Content"; form fields <textarea name="rationale"><textarea name="content">; CSS classes rationale-*content-*; subprocess-protocol JSON key "rationale""content"; glossary + spec conventions. Spec tightening: 03-roles §2.2 step 2 previously listed "plan text, rationale, supporting files" — with content subsuming both, this is now "idea content, supporting files". rationale added to the rename-discipline retired-vocab patterns (identifier-context only — bare English "rationale" meaning "reasoning" stays legitimate).

Phase 12a-1b — Worker + group admin UI — 2026-05-14

Phase 12a-1b complete (worker + group admin UI). Two new admin modules on the web-ui's existing /admin/* surface: /admin/workers/ (list + register + reissue-credential + detail with attribution view) and /admin/groups/ (list + register + detail with transitive-membership walk + add/remove member + delete). Zero spec changes, zero new wire endpoints — pure UI plumbing on top of the 12a-1 wire surface. Two StoreClient handles on the web-ui: the existing process-level worker-bearer app.state.store plus a new app.state.admin_store bearing admin:<admin_token> for admin-gated writes (register_worker, reissue_credential, register_group, add_to_group, remove_from_group, delete_group). When no admin token is configured (postures B/C from plan §D.3), the templates render with mutation controls disabled via a Jinja admin_enabled flag and mutating POSTs short-circuit with ?error=admin-disabled; read paths continue to work through the worker bearer. Posture-D startup guard: cli.py probes store.whoami() after bootstrap and exits non-zero on 401 to prevent a silently-broken auth-enabled-but-no-usable-bearer deployment. Token display: register / reissue return a 200 HTML page (NOT a 303 redirect) that displays the one-shot registration_token exactly once inside a <code class="token"> block, with Cache-Control: no-store to defeat browser caching; idempotent re-register on an existing worker renders the same template with a "no new token was issued" banner and no token. Worker filter on existing admin pages: /admin/tasks/?worker=<id> and /admin/variants/?worker=<id> filter post-fetch by claim / submitted_by / created_by (tasks) or executed_by / evaluated_by (variants), with chunk-9e _INVALID_FILTER empty-rowset discipline. Transitive-membership walk for group detail is client-side DFS over read_group with cycle-safe visited set + depth/breadth caps (GROUP_WALK_DEPTH_CAP=10, GROUP_WALK_BREADTH_CAP=1000). Closed banner-key allowlists per module (admin_workers._WORKER_OUTCOMES and admin_groups._GROUP_OUTCOMES) — unknown keys render no banner. Tests: 31 worker-admin tests (test_admin_workers_routes.py), 39 group-admin tests (test_admin_groups_routes.py), 10 worker-filter + landing-page tests (test_admin_worker_filter.py), plus 2 pytest.mark.e2e real-subprocess tests (test_admin_workers_e2e.py, test_admin_groups_e2e.py). The e2e tests spawn the task-store-server + web-ui with --admin-token enabled, drive register/reissue/group-mutations through real HTTP, extract the one-shot token from the rendered HTML, and verify the resulting credential authenticates via /whoami and a non-member's claim fails with WorkerNotEligible against a group-targeted task. Plan + 3-round codex-review record at docs/plans/eden-phase-12a-1b-worker-group-admin-ui.md + docs/plans/review/eden-phase-12a-1b-worker-group-admin-ui/.

Phase 12a-1g — Experiment durability — spec invariant + Compose bind-mounts — 2026-05-14

Phase 12a-1g complete (experiment durability — spec invariant + Compose bind-mounts). A normative aggregate-durability invariant landed at spec/v0/01-concepts.md §13 (anchored to chapter 01 §1's existing termination model, NOT replacing it) plus a spec/v0/08-storage.md §3 cross-reference and a placeholder row in spec/v0/09-conformance.md §5. The reference Compose deployment switches every durable substrate (postgres, gitea, artifacts, per-host bare clones, per-host worker credentials) from Docker named volumes to host bind-mounts under ${EDEN_EXPERIMENT_DATA_ROOT} — default $HOME/.eden/experiments/$EDEN_EXPERIMENT_ID/, operator-overridable via setup-experiment.sh --data-root <path>. Motivation: a manual experiment was wiped overnight when Docker Desktop rebuilt its embedded VM during an automatic update; substrates fsync'd correctly per chapter 08 §3.1 but their physical storage (the VM's Docker.raw) was less durable than the substrates assumed. Bind-mounts on the host filesystem are unaffected by VM rebuilds, factory resets, down -v, and engine restarts; only explicit rm -rf of the data root, disk failure, or OS reinstall destroys experiment state. Implementation: the base compose.yaml plus both overlays (compose.subprocess.yaml, compose.docker-exec.yaml) migrated together — the DooD wrap's --exec-volume <name>:<target> flags become --exec-bind ${EDEN_EXPERIMENT_DATA_ROOT}/<subdir>:<target> so sibling containers mount the same host-side bind-mounts the worker host sees, not silently-created fresh empty named volumes (the exact trap from the "Three load-bearing wiring traps" pitfall below). The dead eden-blob-data named volume and blob-init busybox service were removed entirely. Two named volumes survive as explicit-ephemeral: eden-repo-init-staging (bootstrap-only; docker volume rm on re-seed) and eden-worktrees (per-task scratch shared between executor and evaluator hosts). setup-experiment changes: new --data-root <path> flag with :-rejection at flag-parse time; idempotent preservation across re-runs; abort-on-incompatible-relocation guard when the existing data root has substrate data and a different --data-root is passed; substrate subdirectory tree created with chmod 0777 (same cross-uid posture as the cidfile dir). Smokes: all four smoke scripts (smoke.sh, smoke-subprocess.sh, smoke-subprocess-docker.sh, e2e.sh) use a per-script mktemp -d -t eden-smoke-XXXXXX data root with trap rm -rf EXIT so bind-mount trees are cleaned up on every exit path. Operator docs: new docs/operations/experiment-data-durability.md covers data layout, durability posture, custom data root, permissions, migration from the pre-12a-1g named-volume layout, and a self-contained manual kill-and-restart verification recipe. Deferred per operator decision: backup/restore tooling (substrate-durability is the substrate's job, not EDEN's), conformance scenarios for the invariant (the chapter 09 §5 placeholder row anchors a future "stop-stack / kill-mount / restart / replay" harness), and the Helm chart audit (Phase 13a uses PVCs — naturally durable under k8s). Plan + 5-round codex-review record at docs/plans/eden-phase-12a-1g-experiment-durability.md + docs/plans/review/eden-phase-12a-1g-experiment-durability/.

Phase 12a-1 — Worker identity foundation — 2026-05-13

Phase 12a-1 complete (worker identity foundation). Per-experiment worker registry, groups, identity-keyed claim ownership, and per-worker bearer authentication shipped across 6 waves. What's covered: chapter 02 §6 (worker registry — register_worker is idempotent on the existing record; ^[a-z0-9][a-z0-9_-]{0,63}$ grammar; admin / system / internal reserved) + §7 (groups — recursive set membership; cycle-detection at write time; reserved-identifier rejection) + §3.5 (Task.target: null/worker/group) + §9 (attribution fields submitted_by / executed_by / evaluated_by / created_by preserved across terminal transitions); chapter 04 §3.5 claim-time RBAC ladder (state→pending; worker_id registered or WorkerNotRegistered; target satisfied or WorkerNotEligible); chapter 07 §13 normative auth (Authorization: Bearer <principal>:<secret> where principal is admin or <worker_id>; argon2id-hashed credentials) + §7 closed error vocabulary gains worker-not-registered / worker-not-eligible / wrong-claimant / not-claimed / unauthorized / forbidden / cycle-detected / reserved-identifier. The per-claim opaque token was retired: claim ownership is now identity-keyed by task.claim.worker_id, and §4.1 submit performs an atomic claim-match against the authenticated worker_id. New Store Protocol operations: register_worker / read_worker / list_workers / reissue_credential / verify_worker_credential / register_group / add_to_group / remove_from_group / read_group / list_groups / delete_group / resolve_worker_in_group. New wire endpoints: POST /workers / POST /workers/<id>/reissue-credential / GET /workers / GET /workers/<id> / POST /groups / POST /groups/<id>/members / DELETE /groups/<id>/members/<mid> / GET /groups/<id> / GET /groups / DELETE /groups/<id> / GET /whoami. New reference helper eden_service_common.auth.bootstrap_worker_credential: each worker host registers itself at startup against the deployment admin token, persists the issued registration_token under --credentials-dir (default /var/lib/eden/credentials), and uses <worker_id>:<token> as its bearer for all subsequent calls. On restart, the persisted credential is verified via the /whoami probe; stale credentials escalate to reissue_credential per spec §6.3. Compose: EDEN_ADMIN_TOKEN replaces EDEN_SHARED_TOKEN in .env.example + compose.yaml + compose.subprocess.yaml + compose.docker-exec.yaml; per-host credential volumes (eden-orchestrator-credentials, eden-ideator-credentials, eden-executor-credentials, eden-evaluator-credentials, eden-web-ui-credentials) mounted at /var/lib/eden/credentials; smoke scripts (smoke.sh, smoke-subprocess.sh, smoke-subprocess-docker.sh) assert the worker registry has ≥4 rows before reading the event stream. Conformance: the v1 scenario index in spec/v0/09-conformance.md §5 gains six new groups (Worker registration, Group resolution, Claim ownership (replaces the retired Claim tokens), Claim eligibility, Worker auth, Attribution persistence); the closed error vocabulary check is split into a _CORE_VOCABULARY (must be observed at least once) and _AUTH_ONLY_TYPES (unauthorized / forbidden, only observable against an auth-enabled IUT). Deferred to a 12a-1b follow-up: retiring the X-Eden-Worker-Id test-fixture header that the reference adapter uses as the auth-disabled mode's worker-id source (full retirement requires migrating the conformance harness to per-call bearer tracking + ~25 scenario files of header drops; it is internal cleanup with no contract-coverage impact since wave 5's RBAC scenarios already exercise the spec ladder); and the §D.5b web-ui per-session retrofit (per-user session bearers via reissue_credential at sign-in is orthogonal to the spec resolution and is expected to land alongside the 12a-2 web-ui changes). 6-wave plan + per-wave commit record at docs/plans/eden-phase-12a-1-worker-identity.md.

Phase 11 chunk 11c — Role-contract conformance scenarios — 2026-05-02

Phase 11 chunk 11c complete (role-contract conformance scenarios). Three new scenario files under conformance/scenarios/test_ideator_submission.py, test_executor_submission.py, test_evaluator_submission.py — assert 20 chapter-03 §2.4 / §3.4 / §4.2 / §4.4 MUSTs grouped under three new chapter-9 §5 v1+roles index entries: Ideator submission, Executor submission, Evaluator submission. What's covered: ideator — drafting-idea → 409 illegal-transition, unknown-idea → 409, zero-idea success, status=error keeps drafts in drafting and emits no idea.dispatched; executor — unknown-variant → 409, cross-idea variant → 409, success-without-commit_sha MUST NOT terminalize variant as success (where the rejection surfaces — at /submit or /accept — is implementation-defined per chapter 9 latitude, so the assertion checks the observable end-state), accepted success writes commit_sha onto the variant, status=error round-trip terminalizes the variant as error AND blocks evaluation-task dispatch against the errored variant (cross-checks task.created events with kind="evaluation" against the errored variant_id), idempotent same-shape resubmit emits exactly one task.submitted, divergent commit_sha → 409 conflicting-resubmission; evaluator — mismatched variant_id → 409, undeclared metric key MUST NOT terminalize as success, type-violating metric (retries: 1.5 against the fixture's retries: integer) MUST NOT terminalize as success, accepted success writes metrics+artifacts_uri+completed_at atomically with variant.succeeded, status=error writes variant metrics+artifacts_uri (distinct from evaluation_error which discards them — pins the §4.4 per-status variant-side write rule), evaluation_error keeps variant in starting AND discards submission metrics+artifacts_uri, retry-exhausted declare-eval-error does NOT graft prior evaluation_error metrics, baseline idempotent role-rule resubmit + vary-only-artifacts_uri-resubmit also accepted (one task.submitted; first submission's artifacts_uri wins). Spec amendment: spec/v0/03-roles.md §4.4 evaluator-resubmit-equivalence was tightened to drop artifacts_uri from the formula and defer to chapter-04 §4.2 as canonical — codex-review round-0 on the plan surfaced an internal conflict where §4.4 had listed artifacts_uri while §4.2 (canonical) does not, and the reference impl follows §4.2. The §3.4 "success-without-commit_sha" and §4.2 "metrics-violate-schema" tests deliberately don't pin where the IUT rejects (submit vs accept) — only that the variant cannot end up success. This is faithful to chapter 9's "transport-neutral semantic tests" stance; reference-impl quirks (it rejects at accept-time via validate_terminal / _accept_* raising IllegalTransition) don't get codified as conformance assertions. Chapter 9 §3 prose extended to cite chapter 03 + 06 for the v1+roles / v1+roles+integrator levels; §5 gains a v1+roles index table with the three role groups. Helper extensions in conformance/src/conformance/harness/_seed.py: submit_plan always sends idea_ids (was omitted on status=error, which is non-conforming under the literal-spec reading of §2.4); submit_evaluation accepts explicit metrics for any status and a new artifacts_uri parameter (so the evaluation_error-with-metrics and vary-artifacts_uri tests drive real wire payloads instead of silently no-op'ing). Reference impl is now v1+roles conformant: 106/106 conformance scenarios pass. Plan + 4-round codex-review record at docs/archive/eden-phase-11c-role-contracts.md + docs/archive/review/eden-phase-11c-role-contracts/.

Phase 11 chunk 11d — v1+roles+integrator conformance — 2026-05-02

Phase 11 complete (chunk 11d shipped — v1+roles+integrator). Two new scenario files at conformance/scenarios/test_integrator_atomicity.py and conformance/scenarios/test_integration_preconditions.py assert 4 chapter-06 §2 / §3.4 / §5.3 MUSTs grouped under two new chapter-9 §5 v1+roles+integrator index entries: Integrator atomicity and Integration preconditions. What's covered: Integrator atomicity — cross-artifact-consistency-on-success (the field at read_variant.variant_commit_sha and the variant.integrated event payload's variant_commit_sha BOTH exist AND reference the SAME SHA — the wire-projection of §3.4's atomic-three invariant on the two wire-visible artifacts; git refs are off-wire), divergent-resubmit-leaves-no-second-event (a §5.3 no-overwrite test that strengthens the existing v1 test_different_sha_returns_409 by additionally pinning event-side cardinality + re-asserting the field via the chapter-06 citation rather than chapter-07); Integration preconditions — error variant → 409 invalid-precondition + no field + no event, evaluation_error variant → 409 + no field + no event (each test asserts the composite end-state per the chunk-11c "end-state, not endpoint" pattern, pinning §3.4's rollback-half "a failed write produces neither field nor event" without a separate §3.4 test). Spec amendment: spec/v0/09-conformance.md §4 v1+roles+integrator bullet was rewritten to scope the level to the wire-observable projection of chapter 06 (§2 + §3.4 + §5.3) and to explicitly defer the git-side artifacts (squash shape, eval-manifest, work/* discipline, reachability) to a future binding chapter — the chapter-7 binding is the only IUT contract a conformance harness can rely on (per chapter 9 §6), and git refs are not exposed through it. The codex-review round-0 surfaced a scope conflict between the original §4 prose ("squash shape, eval-manifest shape, work/* access discipline ...") and §6 ("the chapter-7 HTTP binding is the only IUT contract"); the amendment defers the role-side restatement to the canonical scope statement, applying the chunk-11c codified rule on inter-chapter restatement drift. §2(b) commit_sha is set is now classified as NOT independently wire-observable in v0 — the variant-status lifecycle (chapter 02 §7.1 / chapter 03 §3.2 / §4.4) enforces (b) transitively through (a), so a wire-only suite covers (b) only via (a)'s subsumption (a success variant without commit_sha is unreachable through normal wire endpoints; the reference impl's Store.integrate_variant does not inspect variant.commit_sha at integrate time anyway). §5 prose extended: the v1+roles+integrator level is now described in §4 + §5 as adding "the integrator groups further below" (was singular), and §5 gains a v1+roles+integrator index sub-table. Helper extensions in conformance/src/conformance/harness/_seed.py: two new composite drive-to-end-state helpers drive_to_error_variant (drives implement-status=error + reject — note: implement status=error is rejected by the orchestrator's reject path NOT the accept path; my round-0 implementation incorrectly used accept and tripped 409 IllegalTransition — the chunk-11c test_status_error_terminalizes_variant_and_blocks_evaluate_dispatch shows the canonical pattern) and drive_to_eval_error_variant (drives implement-success-accept + declare_variant_eval_error); both follow the chunk-11c hardening pattern with raise_for_status() on every wire call AND end-state assertions before returning. Reference impl is now v1+roles+integrator conformant: 110/110 conformance scenarios pass. Phase 11 EXIT criterion ("reference impl passes the full v1 suite; suite is documented as the conformance contract for eden-protocol/v0") is met. Plan + 3-round codex-review record at docs/archive/eden-phase-11d-integrator-conformance.md + docs/archive/review/eden-phase-11d-integrator-conformance/.

Phase 10d follow-up B — Gitea as the workers' git remote — 2026-04-30

Phase 10 chunk 10d follow-up B complete (Gitea as the workers' git remote). Workers stop sharing the eden-bare-repo named volume and instead clone a private bare copy from the in-network Gitea container over plain HTTP. setup-experiment.sh provisions Gitea (idempotent admin user, per-experiment HTTP Basic password, repo create via API, credential-helper script under reference/compose/.gitea-creds-<id>/, seed pushed via the new eden_service_common.repo_init --push-to flag). Each worker host's CLI gains --gitea-url and --credential-helper; startup logic clones bare on first run + git fetch --prune origin '+refs/heads/*:refs/heads/*' on subsequent starts (per-service volumes — eden-orchestrator-repo, eden-executor-repo, eden-evaluator-repo, eden-web-ui-repo). Integrator atomicity is now a four-step ladder per chapter 6 §3.4: local create_ref → remote push_refstore.integrate_variant → on store-fail run BOTH compensating delete_remote_ref (4a) AND delete_ref (4b). variant.integrated emits ONLY at step 3, never on local-only state. Step 2 distinguishes definite RefRefused (4b only) from transport-indeterminate GitTransportError (immediate ls_remote read-back disambiguates: remote absent → 4b only; remote == our SHA → 4a + 4b; remote == different SHA → 4b only; ls_remote also fails → 4b only with the §D.7c startup sweep as backstop). Remote-orphan reconciliation (Integrator.reconcile_remote_orphans() at startup) recovers variant_id from each remote variant/* commit's .eden/variants/<variant_id>/evaluation.json tree path (chapter 6 §3.2 — spec-authoritative; chapter 2 §1.3 keeps variant_id opaque so ref-name parsing is NOT used) and deletes refs whose Store.read_variant(variant_id) lacks a variant_commit_sha. Role-specific transport-failure mapping per chapter 3: executor maps GitTransportError to VariantSubmission(status="error"); evaluator maps to EvaluationSubmission(status="evaluation_error") (so the variant stays at success and can be re-evaluated); integrator follows §3.4 above. Compose volume name discipline: eden-executor-repo and eden-evaluator-repo get explicit name: so the docker-exec wrap's --mount source=eden-executor-repo resolves to the same volume the worker host is reading from (same trap that bit eden-bare-repo and eden-worktrees in sub-chunk A). New eden_git ops: clone_from, push_ref, fetch_ref, fetch_all_heads, delete_remote_ref, ls_remote, plus typed RefRefused / GitTransportError exceptions. 13 new test_remote_ops.py unit tests + 8 new test_remote_integrator.py tests drive Integrator.integrate() through every failure-mode branch via the public method; the compose-smoke job's existing git ls-remote assertion now confirms ≥3 refs/heads/variant/* on Gitea match the ≥3 variant.integrated events in the store. The full cutover removes the shared eden-bare-repo volume; eden-repo-init writes seed → ephemeral eden-repo-init-staging → push to Gitea, then exits.

Phase 10d follow-up A — Container-isolated *_command — 2026-04-29

Phase 10 chunk 10d follow-up A complete (container-isolated *_command). Each subprocess-mode worker host gains an opt-in --exec-mode docker that wraps every spawn of the user's *_command in a sibling container via DooD (Docker outside of Docker — host /var/run/docker.sock mounted into the worker). The wrap shape lives in reference/services/_common/src/eden_service_common/container_exec.py and is docker run --rm -i --init --cidfile <path> --label eden.{role,host,task_id} --mount type=volume,source=<name>,target=<path>[:readonly] --mount type=bind,source=<host-path>,target=<path>[:readonly] -w <cwd> -e <KEYS>... <image> bash -lc '<original-command>'. Mount targets inside the spawned child match the worker host's exact paths so worker-internal env vars (EDEN_TASK_JSON, EDEN_OUTPUT, EDEN_WORKTREE, EDEN_EXPERIMENT_DIR) resolve consistently in both views. The set of mounts is supplied by repeatable --exec-volume <name:target[:ro|rw]> and --exec-bind <host-path:target[:ro|rw]> flags driven by the compose overlay; eden-bare-repo, eden-worktrees, and eden-artifacts-data are pinned to literal name: in compose.yaml so docker daemon resolves the wrap's --mount source=<literal> against the same volume the worker sees. Container lifecycle: every spawn writes its container id to a unique <cidfile_dir>/<role>-<spawn-uuid>.cid; a cleanup_callback unlinks the cidfile on every terminal exit branch (graceful, fast-path, and SIGKILL escalation); a separate post_kill_callback runs docker kill && docker rm -f on the SIGKILL escalation branch only — necessary because killing the local docker run client does NOT kill the spawned container, the daemon parent keeps it alive. Both callbacks are stored on the Subprocess instance at spawn time (no per-call threading). At host startup, reap_orphaned_containers(role, host=gethostname()) removes any containers labeled eden.host=<this> from a prior crash, scoped by host so cross-host races are impossible by construction. Identity: eden-runtime:dev mirrors the eden:1000 user from eden-reference:dev — without this, worktree files written by the worker host (uid 1000) trip git's dubious ownership check inside the spawned child (running as root by default), and any commits the child produces would be uid-0-owned, breaking subsequent integrator reads. The wrap deliberately does not pass --user so experiment images that override USER (e.g. for system-level installs) are honored. DooD socket permissions: the worker host gets group_add: ["${EDEN_DOCKER_GID}"] from the compose overlay; setup-experiment probes the gid with docker run --rm -v /var/run/docker.sock:/var/run/docker.sock alpine stat -c '%g' /var/run/docker.sock (probe-from-inside, not host-side stat — Docker Desktop's VM has its own gid namespace). Image strategy: if the experiment dir contains a Dockerfile, setup-experiment builds it as eden-experiment-<id>:dev and writes that tag into .env; otherwise the default eden-runtime:dev (built from reference/compose/Dockerfile.runtime, python3 + git + bash + ca-certificates) is used. Security boundary: DooD is a soft boundary — secrets passed via -e KEY are visible to anyone with daemon access (which under DooD includes any other concurrent *_command); a malicious *_command has full daemon access. Documented as informative §7 in spec/v0/reference-bindings/worker-host-subprocess.md. The reference deployment trades that boundary for operational simplicity; the intent is bug isolation + dependency isolation, not hostile-code containment. Compose overlay split: compose.subprocess.yaml is the host-mode subprocess overlay (chunk-10d soft isolation, no docker socket access); a new compose.docker-exec.yaml layered ON TOP is the only place the docker socket gets mounted into worker hosts and group_add: ["${EDEN_DOCKER_GID}"] applied — so a layering bug can't accidentally grant DooD privilege to the host-mode path. Both smoke scripts assert this directly via docker inspect eden-executor-host: host mode requires the socket bind to be ABSENT; docker mode requires it to be PRESENT. New compose-smoke-subprocess-docker CI job mirrors compose-smoke-subprocess end-state assertions (≥3 variant.integrated, ≥9 task.completed, ≥3 ideation-task.completed) plus a per-task orphan-container assertion (no eden.role=executor|evaluator containers left after quiescence) and a clean-teardown check (after compose stop --timeout 15, no eden.role=ideator sibling remains — clean-SIGTERM or SIGKILL-escalation is acceptable for the smoke). The dedicated SIGKILL-escalation invariant (post_kill_callback actually docker kills a sibling whose user command ignored SIGTERM) is exercised by the pytest.mark.docker integration test test_terminate_sigkill_path_invokes_post_kill_callback in reference/services/_common/tests/test_container_exec_integration.py, which also drives the actual production worktree-with-gitlink mount shape (bare repo + worktree's .git gitlink, both mounted into the spawned child). Branch protection is not updated to require the new job in this chunk (same posture chunks 10a / 10b / 10c / 10d / 10e took for newly-added jobs).

Phase 10 chunk 10d — Subprocess-mode worker hosts — 2026-04-28

Phase 10 chunk 10d complete. Each of the ideator / executor / evaluator hosts gains a new --mode subprocess that invokes a user-supplied command from the experiment-config YAML (ideation_command / execution_command / evaluation_command) instead of the deterministic Phase-8b scripted profile. The subprocess invocation is via shell=True so user expressions like python3 ${EDEN_EXPERIMENT_DIR}/ideation.py expand correctly; the host supplies EDEN_EXPERIMENT_DIR, EDEN_TASK_JSON, EDEN_OUTPUT, and (for execution/evaluation) EDEN_WORKTREE. Ideator subprocess is long-running with a JSON-line ready / ideation / idea / ideation-done protocol over stdin/stdout (so user code can hold accumulating LLM context across ideation tasks); executor + evaluator subprocesses are per-task short-lived with cwd set to a per-task git worktree, and read/write outcome JSON via the EDEN_OUTPUT path. The executor flow honors spec/v0/03-roles.md §3.2 step 1 — Store.create_variant(status="starting") runs before any repository write, mirroring the chunk-9c web-ui executor ordering. Worktrees are host-private under <worktrees-dir>/<container_hostname>/<task_id>/; startup-time cleanup uses path-scoped git worktree remove --force (never repo-global git worktree prune), so cross-host races don't exist by construction. The chapter-3 submission shapes carry no free-form description field; worker-side failure context (subprocess timeout, malformed outcome JSON, etc.) goes to the host log only. The protocol is documented as a non-normative reference binding at spec/v0/reference-bindings/worker-host-subprocess.md — a conforming alternative host can either match the shape (drop-in compatibility with the fixture experiment) or pick a different mechanism. Three new fixture scripts (tests/fixtures/experiment/ideation.py, execution.py, evaluation.py) exercise the protocol deterministically without LLM secrets so the new compose-smoke-subprocess CI job can run end-to-end alongside the original compose-smoke. The fixture's *_command config keys gain ${EDEN_EXPERIMENT_DIR}/... prefixes (single source of truth: the experiment YAML, not a CLI flag). Compose plumbing is a non-invasive overlay reference/compose/compose.subprocess.yaml layered via compose -f compose.yaml -f compose.subprocess.yaml; setup-experiment grows --experiment-dir (defaults to the directory containing the config's .eden) and writes EDEN_EXPERIMENT_DIR_HOST into .env for the bind-mount. Container isolation of the user's command (docker-in-docker, experiment-specific image build, auth-secret mounting) is deferred to a 10d follow-up sub-chunk; the subprocess.Popen call is the wrappable layer, and the protocol stays unchanged when wrapped. Branch protection is not updated to require compose-smoke-subprocess in this chunk (same posture 10a / 10b+10c took for newly-added jobs).

Phase 10 chunk 10e — End-to-end Compose integration test — 2026-04-28

Phase 10 chunk 10e complete. A new compose-e2e CI job runs reference/compose/healthcheck/e2e.sh — the stack-shape coverage missing from compose-smoke / compose-smoke-subprocess: a Web UI ideator walkthrough (sign-in → claim → submit), an admin-reclaim drill (claim → admin-reclaim with a task.reclaimed cause=operator event in the log), and a termination drill (compose stop --timeout 10; every container in compose ps -a ends in exited with ExitCode != 137). The != 137 SIGKILL guard is deliberately weaker than == 0: external images don't always install graceful SIGTERM handlers, and 143 from a default-handler is also a clean termination — what we actually care about is that no container needed SIGKILL. Bring-up is staged to defeat the ideator-host-claim race that the round-0 codex review flagged: stage 1 omits both ideator-host AND gitea (gitea's 30s healthcheck start_period would race against the orchestrator's 30s quiescence budget; gitea is the deferred 10d follow-up sub-chunk and isn't consumed by 10e). The python driver in reference/compose/healthcheck/e2e_drive.py runs the UI walkthroughs against the seeded-but-unclaimed ideation tasks, then stage 2 brings ideator-host online so the experiment proceeds to quiescence and the orchestrator exits 0. EDEN_IDEATION_TASKS=4 is overridden in the staged .env (vs smoke's default 3) so ideation-0001 / ideation-0002 carry the UI walkthrough + admin-reclaim drill while ideation-0003 / ideation-0004 (and the reclaimed ideation-0002) flow through the headless host. The python driver polls /ideator/ for all 4 seeded IDs to land before claiming — the orchestrator has no compose healthcheck so compose up --wait returns as soon as the container is running, which can precede the seed completing. The poll happens after sign-in, not before: /ideator/ and /admin/tasks/ redirect unauthed sessions to /signin, so an unauthenticated poll loop would just see 303s. Driver assertions match the actual route shapes: ideator submit returns 200 with ideator_submitted.html (NOT 303); admin reclaim redirects to /admin/tasks/<id>/?reclaimed=ok with the trailing slash; orchestrator-seeded ideation tasks are ideation-{i:04d} not bare-int. Termination drill iterates the full compose ps -a output (not eden-* only) — covers postgres, blob-init, task-store-server, orchestrator, ideator-host, executor-host, evaluator-host, and web-ui (gitea was never started in stage 1 so it doesn't appear; eden-repo-init ran via compose run --rm and is gone). Local mac compatibility: the container loop uses portable while IFS= read -r … done < <(…) instead of bash 4+'s mapfile builtin so the script runs under macOS's default bash 3.2. New CI job is not required by branch protection in this chunk (same posture 10c / 10d took for newly-added compose jobs).

Phase 10 chunks 10b + 10c — Dockerized services + setup-experiment — 2026-04-26

Phase 10 chunks 10b + 10c complete. Six EDEN reference services are now dockerized and wired into the Compose stack at reference/compose/, and a new reference/scripts/setup-experiment/setup-experiment.sh bootstraps a runnable stack from an experiment-config YAML in one shot. After bash setup-experiment.sh <config.yaml> --experiment-id <id> && docker compose --env-file .env up -d --wait, the fixture experiment runs to completion in Compose: orchestrator dispatches three ideation tasks → ideator-host claims/submits → orchestrator integrates → 3 variants succeed and integrate, with the orchestrator exiting 0 on quiescence (restart: "on-failure" so exit-0 sticks; tuned to --poll-interval 1.0 --max-quiescent-iterations 30 for Compose worker-startup latency). All seven services (six EDEN services + setup-time eden-repo-init) share one eden-reference:dev image (multi-stage build with uv sync --frozen --no-dev --all-packages — the --all-packages flag is load-bearing because the workspace root pins tool.uv.package = false and lists the eden-* workspace members only in the dev group, which --no-dev would otherwise skip). The roadmap delta documenting the per-service-image deviation, the deferral of the LLM-executor experiment-specific image to 10d, the deferral of control-plane registration to Phase 12, and the deferral of Gitea-as-the-workers'-actual-git-remote to a follow-up sub-chunk after 10d is recorded in the plan and reflected in the chunk's roadmap update. New eden-storage.PostgresStore (reference/packages/eden-storage/src/eden_storage/postgres.py) is the third backend conforming to the same Store Protocol as InMemoryStore/SqliteStore; backed by psycopg[binary]>=3.2,<4 with autocommit=True + explicit per-op BEGIN ISOLATION LEVEL SERIALIZABLE READ WRITE (psycopg's auto-begin would error ActiveSqlTransaction on BEGIN ISOLATION LEVEL). Schema is byte-for-byte parallel to SQLite (data text NOT NULL, not jsonb — keeps the read path identical and defers indexing concerns). The new python-test-postgres CI job spins up postgres:16.6-alpine as a service container and runs the parametrized backend tests against it; the existing python-test job continues to run with EDEN_TEST_POSTGRES_DSN unset (so the postgres rows skip), keeping it fast. task-store-server gains a URL-based --store-url flag (:memory: / sqlite:///<path> / postgresql://… / bare-path-for-compat); the deprecated --db-path alias survives for one phase. Bare-repo bootstrap is owned synchronously by setup-experiment (one canonical command shape: docker compose build eden-repo-init && docker compose run --rm --no-deps eden-repo-init--no-deps keeps future dependency edits from accidentally starting Postgres/Gitea just to seed). The seed SHA is captured from stdout and replaced into a placeholder line in .env (zero-padded sentinel) — replaced, not appended, so grep -E '^EDEN_BASE_COMMIT_SHA=' returns one definition. The eden-repo-init service has profiles: ["setup"] so it's invisible to plain compose up. Volume permissions are pre-created in the Dockerfile via RUN mkdir -p /var/lib/eden/{repo,artifacts} && chown -R eden:eden /var/lib/eden — without this the named volume inherits root ownership from the volume driver and the eden user can't write. Healthchecks for task-store-server and web-ui use compose-load-time ${VAR} interpolation (single $), not container-shell $${VAR}, because the env-file vars aren't propagated into the container's shell environment unless declared under environment:; baking the values into the literal command at compose-load is simpler. The web-ui mount is read-write, not read-only, because passing --repo-path activates the entire executor module (chunk 9c) which writes refs on submit — read-only would 500. The redundancy with executor-host is intentional (operators pick which they prefer; both race for claim atomicity via Store.claim chapter 04 §3). New web-ui /healthz endpoint is unauthenticated by design (compose healthchecks must work before any user signs in; reveals only "process is up"). Extended reference/compose/healthcheck/smoke.sh (the compose-smoke CI job's invariant) now runs setup-experiment, asserts the seeded ref before compose up via a one-shot probe that reuses the already-built eden-reference:dev image (avoiding a floating latest tag), waits up to 180s for the orchestrator to exit 0 on quiescence, and verifies the final task-store state on the wire endpoint: ≥3 variant.integrated events (full pipeline), ≥9 task.completed events (3 plan + 3 implement + 3 evaluate), and ≥3 task.completed events whose task_id starts with plan- (each seeded ideation task reached a terminal state). Branch protection is not updated to require python-test-postgres in this chunk — that lands in a follow-up after the job has a few clean runs (same posture 10a took for compose-smoke).

Phase 9 chunk 1 — Web UI shell + ideator module — 2026-04-25

Phase 9 chunk 1 complete. A reference Web UI service (reference/services/web-ui/) ships the UI shell + ideator module. It is a backend-for-frontend over eden_wire.StoreClient: holds the shared bearer, talks to the task-store-server only via the wire binding, and renders server-side Jinja templates. HTMX 1.9.12 is vendored under static/ (SHA-256 pinned in the service README) as a progressive-enhancement layer — every mutating route works without JS via plain form-POST + 303 redirect / re-render, and HTMX-aware routes also serve a fragment when the browser sends HX-Request: true (the chunk-1 example is "add another idea row" inline). The session cookie is itsdangerous-signed (HttpOnly + SameSite=Lax + Path=/, plus opt-in --secure-cookies for TLS deployments) and carries a per-session CSRF token validated on every mutating route. The ideator page surfaces the experiment's objective and evaluation_schema (read-only) plus a recent-ideas/recent-variants context panel, then walks a 3-phase write — create_idea(state="drafting") for every row, then mark_idea_ready for each, then submit with retry-before-orphan (3 attempts, exponential backoff) leveraging chapter 07 §2.4 / §8.1 idempotent resubmit. The narrowest unsafe window (Phase 2 → Phase 3 ready-but-unreferenced ideas) is documented as a known limitation that applies equally to the existing scripted ideator host. Cross-cutting addition: eden_dispatch.sweep_expired_claims runs once per run_orchestrator_loop iteration so UI claims with expires_at are reclaimed automatically when an abandoned tab strands a task. Tests cover unit (route-by-route + CSRF), cross-request flow (claim → submit + validation recovery + stranded-claim recovery via the sweeper), security invariants (cookie attributes + bearer non-leak), partial-write recovery (Phase-2 mid-loop failure leaves drafting-only; Phase-3 retry-then-orphan), and a two-process pytest.mark.e2e test that drives the full ideator flow over real HTTP against a real task-store-server subprocess.

Phase 9 chunk 9c — Web UI executor module — 2026-04-25

Phase 9 chunk 9c complete. The Web UI service gains the executor module, gated on a new optional --repo-path <path> CLI flag. With the flag set, routes/executor.py walks claim → draft → submit for execution tasks: list pending tasks, claim with TTL, render a draft form that surfaces the idea's slug/priority/parent_commits/artifacts_uri (plus the content markdown rendered inline only when its file:// path resolves inside --artifacts-dir and is ≤ 1 MiB — the §A.1 trust-boundary helper read_idea_content lives in routes/_helpers.py), then submit. The submit handler enforces spec/v0/03-roles.md §3.3 reachability via eden_git.GitRepo.commit_exists + is_ancestor(parent, commit_sha) against every parent in idea.parent_commits, runs a Pre-Phase-1 ref-collision guard (ref_exists("refs/heads/work/<slug>-<variant_id>") short-circuits with a form error and no create_variant), then walks Phase 1 (store.create_variant(status="starting") — no commit_sha; _accept_execute writes commit_sha onto the variant later), Phase 2 (repo.create_ref on the canonical work branch — runs after create_variant so the §3.2 step-1 ordering "variant persisted before observable repo writes" holds), and Phase 3 (store.submit with retry-before-orphan: 3 attempts, exponential backoff, WrongToken/IllegalTransition/ConflictingResubmission short-circuit, transport-shaped exceptions retried). On retry exhaustion the route runs a committed-state read-back keyed off read_task + read_submission + submissions_equivalent (not worker_id, which is shared by default) — distinguishing "our submit committed; lost the response" (success page) from "never committed" (orphan page) from "different submission won the race" (orphan page, eden://error/conflicting-resubmission). Orphaned starting variants auto-recover via the chunk-1 expired-claim sweeper (_base.reclaim composite-commits the variant to error atomically with pending). variant_id is server-only — generated at claim time and stored in the in-process _CLAIMS dict alongside the claim token, never round-trips through the request surface; the rendered form has no name="variant_id" input and a forged form value is ignored. Tests cover per-route validation (test_executor_routes.py), the cross-request flow including reachability rejections and the status=error path (test_executor_flow.py), each branch of §C-recovery via monkeypatched failures (test_executor_partial_write.py), security invariants including the artifact-rendering trust boundary and the variant_id non-leak (test_executor_security.py), and a pytest.mark.e2e two-process test that drives the full executor flow against a real task-store-server + web-ui subprocess (test_executor_e2e.py).

Phase 9 chunk 9d — Web UI evaluator module — 2026-04-25

Phase 9 chunk 9d complete. The Web UI service gains the evaluator module, mounted unconditionally (no CLI flag — the evaluator never touches a repo through the UI). routes/evaluator.py walks claim → draft → submit for evaluation tasks: list pending tasks, claim with TTL (the variant_id is read from task.payload.variant_id and stashed in the in-process _CLAIMS dict, never round-tripping through the request surface), render a draft form that surfaces the variant under evaluation (variant_id, branch, commit_sha, parent_commits, plus the executor-set variant.description and variant.artifacts_uri per spec/v0/03-roles.md §3.2 step 3), the idea context, and one input per metric in experiment_config.evaluation_schema typed by declared MetricType (number step=1 for integer, number step=any for real, text otherwise). The submit handler validates the form (status ∈ {success, error, evaluation_error} per §4.4; metric values type-check against the schema using the same rules Store._validate_evaluation enforces, including the wire-legal integer form 1.0 per 02-data-model.md §1.3 and rejection of non-finite floats; status=success requires at least one metric value), then runs store.submit with retry-before-orphan + a committed-state read-back. Exception classification differs from chunk 9c by design: WrongToken and ConflictingResubmission short-circuit (definitive); InvalidPrecondition re-renders the form with the wire-error banner (fixable, not orphan); IllegalTransition falls through to read-back (so a "we won, response lost, orchestrator already terminalized" sequence correctly classifies as success rather than orphan — the chunk-9c short-circuit pattern is flagged for symmetric repair in §K-2 of the plan). The read-back keys off read_task + read_submission + submissions_equivalent; for EvaluationSubmission equivalence is status + variant_id + metrics per chapter 04 §4.2 (artifacts_uri is not part of equivalence — first submission's value wins). The chunk-9c trust-boundary helper was generalized to _read_inline_artifact(uri, artifacts_dir) in routes/_helpers.py; read_idea_content is now a thin wrapper, and a sibling read_variant_artifact covers the variant-side surface with the same envelope (file://, contained in --artifacts-dir, ≤ 1 MiB). Tests cover per-route validation including the integer-1.0 accept and nan-reject parity (test_evaluator_routes.py, 20 tests), the cross-request flow including validation-recovery and sweeper-driven stranded-claim recovery (test_evaluator_flow.py, 5 tests), every recovery branch including the new sub-cases I-a/I-b/I-c that exercise the IllegalTransition → read-back arm (test_evaluator_partial_write.py, 12 tests), security invariants including the variant-side _read_inline_artifact trust boundary, the variant_id non-leak, and variant.description Jinja autoescape (test_evaluator_security.py, 18 tests), and a pytest.mark.e2e two-process test that drives the full evaluator flow against a real task-store-server + web-ui subprocess (test_evaluator_e2e.py).

Phase 9 chunk 9e — Admin / observability surface — 2026-04-25

Phase 9 complete (chunk 9e shipped). The Web UI service gains an /admin/* surface that closes Phase 9: read-only dashboards over the experiment's tasks (filterable by kind/state, with claim-age + claim-expired badges), variants (filterable by status, with an orphaned-starting-variant badge for variants whose owning execution task went terminal without driving the variant atomically), and the event log (capped at a configurable limit; natural-index-before-slice discipline so an event's rendered position is stable across filter / limit / reverse operations); a per-task detail page that exposes operator reclaim via Store.reclaim(task_id, "operator") per spec/v0/04-task-protocol.md §5.1, with separate "reclaim" / "force-reclaim (replays work)" UI variants gating the claimed vs submitted cases and a closed allowlist of ?reclaimed=ok / ?error=… banner outcomes (no arbitrary text echo); a per-variant detail page with a related-events filter that unions event.data.variant_id == variant_id plus task-id matches for the execution task that produced this variant and any evaluation task that references it, ordered by replay-index position (the event_id factory is pluggable per reference/packages/eden-storage/src/eden_storage/_base.py and is not a reliable ordering contract); and a work/* ref garbage-collection page (when --repo-path is set) that classifies each ref by exact variant.branch equality — not by parsing the ref name — and offers CAS-guarded repo.delete_ref(ref_name, expected_old_sha=…) deletion only for terminal-and-handled variants whose commit_sha matches the live ref SHA, plus orphan-ref deletion when no variant owns the branch. The route layer never trusts the form's hidden expected_old_sha; it does its own fresh list_refs lookup at POST time and uses that SHA. The auth-first POST discipline matches existing modules (ideator / executor / evaluator): get_session(request) runs before csrf_ok, so an unauthenticated POST redirects to /signin (303) and an authenticated POST with a missing/wrong CSRF token returns 403. The make_app factory gains an @app.exception_handler(eden_storage.errors.NotFound) that translates raised storage/wire NotFound into the same _error.html 404 page the existing HTTP-404 handler renders (the prior handler only caught HTTP 404 responses, not raised NotFound exceptions). routes/admin.py is mounted unconditionally; the work-refs sub-page handles the repo=None case in-template. Tests cover per-route validation (test_admin_routes.py, 33 tests), cross-request flows including ideator-claim → admin-reclaim round trip + admin variant-detail rendering for an evaluator-seeded variant (test_admin_flow.py, 3 tests), security invariants including unauthenticated-POST-redirects-before-CSRF, error/reclaimed banner allowlist, ref-name regex rejection of every traversal/escape variant including refs/heads/variant/* / HEAD / shell-injection-shaped names with subprocess.run asserted uncalled, server-side expected_old_sha source (the form-hidden field is overridden by the live SHA), and the javascript: URI scheme allowlist on variant detail (test_admin_security.py, 21 tests), partial-write recovery for the four failure modes — IllegalTransition on terminal reclaim, transport-shaped Exception from the store with no auto-retry, GET-time/POST-time eligibility divergence, and CAS-miss → ?error=ref-changed (test_admin_partial_write.py, 5 tests), and a pytest.mark.e2e real-subprocess test that drives ideator-claim + admin-reclaim through real HTTP and verifies the resulting task.reclaimed event with cause=operator via a separate StoreClient (test_admin_e2e.py).

Phase 7a — eden-git subprocess wrapper — 2026-04-24

Phase 7a complete. eden-git's subprocess wrapper ships GitRepo covering ref/object inspection (rev_parse, resolve_ref, list_refs, is_ancestor, ls_tree), plumbing (write_blob, write_tree_from_entries, write_tree_with_file, commit_tree, create_ref, update_ref), worktree management, and branch management. Author identity and commit.gpgsign=false are pinned per-invocation so the user's ambient git config never leaks into integrator commits. See docs/roadmap.md for the full 13-phase plan.

Phase 7b — eden-git Integrator — 2026-04-24

Phase 7b complete. eden-git now also ships the Integrator that composes GitRepo with a Store to integrate success variants per chapter 6. Given a variant with a recorded commit_sha, Integrator.integrate builds the §3.2 single-commit squash (worker-tip tree plus the evaluation manifest at .eden/variants/<variant_id>/evaluation.json), writes the refs/heads/variant/<id>-<slug> ref via zero-oid CAS, and routes the store's atomic integrate_variant write for variant_commit_sha and the variant.integrated event. On store failure the ref is compensatingly deleted per §3.4, matching the post-integration reading recorded in spec/v0/design-notes/integrator-atomicity.md. Re-invocation on an already-integrated variant is a verified no-op (§5.3): ref SHA, squash tree shape, and manifest bytes are re-derived and compared. §2 preconditions (status == success, commit_sha reachable from branch tip), §1.4 reachability, and §2 metrics validity are all enforced up front; the new public Store.validate_evaluation closes the §2 MUST-NOT-integrate clause even if upstream orchestrator validation were bypassed. The spec itself was tightened at §3.4 to make the post-integration reading explicit. The dispatch driver accepted integrate_variant: Callable[[str], object] in place of the Phase 5 placeholder integrator_commit_factory parameter; in Phase 8c that hook moved to run_orchestrator_iteration. Eval-manifest bytes are deterministic (sorted keys, indent=2, trailing newline) to make §5.3 idempotency re-derivation stable.

Phase 6 — eden-storage Store Protocol + SQLite backend — 2026-04-24

Phase 6 complete. The eden-storage package ships the Store structural interface for the task store, event log, and idea/variant persistence sides of chapter 8 — collapsed into a single Protocol per §7 implementation latitude. Two conforming backends satisfy it: InMemoryStore (fast, non-durable; moved from eden-dispatch) and SqliteStore (durable across restarts via a WAL-mode SQLite database with synchronous=FULL, matching §3.1's crash-survival requirement). The Protocol covers the spec-literal create_task / replay / read_range operations alongside the typed convenience helpers. Both backends share the transition logic in _base.py, and the same conformance scenarios are parametrized across both — drift from the Protocol surfaces in tests, not in production. Restart-safety tests close and reopen a SQLite store mid-experiment to confirm state, event log, and claim tokens survive, and a monkey-patched _apply_commit failure verifies rollback. The artifact store (§5), subscribe streaming (§2.1), and Postgres remain non-goals for Phase 6; they land in Phase 10 / Phase 8 / later respectively.

Phase 8b — Standalone reference services — 2026-04-24

Phase 8b complete. Five reference services under reference/services/ extract the dispatch loop into independent OS processes that communicate only over the Phase 8a wire binding: task-store-server/ hosts the Store behind uvicorn; orchestrator/ runs the finalize/dispatch/integrate half of the loop against a StoreClient; ideator/, executor/, and evaluator/ are standalone worker hosts. A new shared package reference/services/_common/ holds the JSON-line logger, SIGTERM/SIGINT stop flag, readiness probe, common argparse flags, scripted ideation_fn/execution_fn/evaluation_fn profiles, and the seed_bare_repo helper. eden-dispatch exposes run_orchestrator_iteration as its orchestrator-side entry point so the standalone orchestrator drives only that half (the in-process run_experiment driver remained alongside it through 8b and was deleted in 8c). eden-wire.make_app(store, *, shared_token=…) and eden-wire.StoreClient(…, token=…) add reference-only bearer-token auth: requests without the right Authorization: Bearer <token> are rejected as eden://reference-error/unauthorized (HTTP 401), under a new namespace deliberately kept out of the normative eden://error/… vocabulary in spec/v0/07-wire-protocol.md §7. A new informative §12 in chapter 07 documents the scheme. The executor host writes real git commits (single-parent or merge, honoring idea.parent_commits) on work/* branches in the bare repo the orchestrator integrates from. A pytest.mark.e2e test in reference/services/orchestrator/tests/test_e2e.py forks all five processes on an ephemeral port, drives a 3-variant experiment to quiescence over real HTTP + real SQLite + real git, and tears down with SIGTERM → SIGKILL fallback.

Phase 8c — Cut over to wire-protocol-only dispatch — 2026-04-24

Phase 8c complete. The in-process dispatch driver is removed: eden-dispatch.run_experiment was deleted and its public-API surface narrows to run_orchestrator_iteration plus the scripted workers (ScriptedIdeator / ScriptedExecutor / ScriptedEvaluator). The Phase 5 / 7b in-process end-to-end tests in reference/packages/eden-dispatch/tests/test_end_to_end.py were deleted; the lifecycle-reconstruction invariant they covered moved into reference/services/orchestrator/tests/test_e2e.py as a _reconstruct_lifecycle fold over the post-teardown SqliteStore.read_range(). The malformed-success → validation_error routing test in eden-storage/tests/test_store_hardening.py was rewritten to drive run_orchestrator_iteration directly through the legitimate Store API. The pytest.mark.e2e real-subprocess test from Phase 8b is now the only end-to-end coverage path; components communicate exclusively through the Phase 8a wire binding.

Phase 8a — eden-wire HTTP binding — 2026-04-23

Phase 8a complete. eden-wire ships an HTTP binding for the chapter 4 / 5 / 6 §3.4 / 8 §§1.1–2.1 operations specified in the new spec/v0/07-wire-protocol.md chapter. The package exposes a FastAPI make_app(store) that routes every wire endpoint to a Store instance, and a StoreClient that satisfies the same Store Protocol against the HTTP surface — so existing callers (dispatch driver, integrator) work across the process boundary unchanged. Errors round-trip as RFC 7807 problem+json with a closed vocabulary of eden://error/<name> types (§7). Store.integrate_variant is now same-value idempotent (§5 of the chapter); the Integrator distinguishes different-SHA divergence (AtomicityViolation, no ref compensation) from other synchronous rejections (normal §3.4 compensating-delete flow). StoreClient.integrate_variant reconciles transport-indeterminate failures via read-back: the three outcomes are confirmed success, confirmed divergence (InvalidPrecondition), or IndeterminateIntegration when the server's outcome cannot be determined. Polling + long-poll-style subscribe are both bound (§6). Worker-process extraction, SSE/WebSocket push, and cut-over of in-process paths remain Phase 8b / 8c.