feat: Dual-Tree Secondary Index (#560)

* implement secondary index runtime access and recovery

Author: jiangzhe

docs/backlogs/000087-refactor-recovery-process-parallel-log-replay.md

@@ -0,0 +1,45 @@
+# Backlog: Refactor recovery process and parallelize log replay
+
+## Summary
+
+Recovery has accumulated timestamp-boundary logic, checkpoint bootstrap handling, hot MemTree rebuild behavior, and sequential redo replay in one coordinator. Add a follow-up to clean up the recovery code structure and evaluate or implement parallel log replay without regressing catalog or user-table replay correctness.
+
+## Reference
+
+Raised during task 000120 while wiring dual-tree secondary-index recovery. Discussion clarified LogRecovery state such as catalog_replay_start_ts, replay_floor, max_recovered_cts, table_states, and recovered_tables, and exposed that recovery needs clearer structure before adding more concurrency.
+
+## Deferred From (Optional)
+
+docs/tasks/000120-secondary-index-runtime-access-and-recovery.md
+
+## Deferral Context (Optional)
+
+- Defer Reason: Task 000120 is scoped to secondary-index runtime access and recovery cutover. Broad recovery cleanup and replay parallelization would widen the task beyond dual-tree integration and risk delaying the current feature.
+- Findings: Current recovery tracks multiple replay boundaries: catalog checkpoint replay start, per-table heap redo start, deletion cutoff, a global replay floor, and max recovered CTS. Checkpointed DiskTree secondary-index roots should remain cold state while recovery rebuilds only hot MemTree rows. DDL acts as a pipeline breaker today, and DML replay is still sequential with a todo for dispatching work to multiple threads.
+- Direction Hint: Start with documentation and small structural cleanup so timestamp boundary semantics remain explicit. Then plan parallel replay around DDL barriers, catalog-table serialization, user-table independence, and row-page conflict grouping. Preserve max recovered CTS as a global timestamp watermark even for skipped log records.
+
+## Scope Hint
+
+Audit the trx recovery flow, separate checkpoint bootstrap, DDL and catalog replay, user-table DML replay, index rebuild, and page refresh responsibilities where useful, then design parallel DML or log replay boundaries with deterministic ordering around DDL pipeline breakers and per-table or per-page conflicts.
+
+## Acceptance Hint
+
+Recovery code has clearer ownership boundaries and comments, tests cover catalog replay boundaries, per-table heap and delete cutoffs, hot secondary-index rebuilds, and parallel replay ordering, and full doradb-storage nextest passes with no timestamp reuse or index recovery regressions.
+
+## Notes (Optional)
+
+Consider whether a full RFC is needed before implementation because parallel replay touches recovery ordering, error propagation, and replay determinism.
+
+## Close Reason (Added When Closed)
+
+When a backlog item is moved to `docs/backlogs/closed/`, append:
+
+```md
+## Close Reason
+
+- Type: <implemented|stale|replaced|duplicate|wontfix|already-implemented|other>
+- Detail: <reason detail>
+- Closed By: <backlog close>
+- Reference: <task/issue/pr reference>
+- Closed At: <YYYY-MM-DD>
+```

docs/backlogs/000088-remove-recovery-skip-option.md

@@ -0,0 +1,44 @@
+# Backlog: Remove recovery skip option
+
+## Summary
+
+Remove the transaction-system `skip_recovery` configuration path. Skipping redo recovery is dangerous for a storage engine because it can admit runtime operations on stale or partially recovered state and corrupt data. The option was originally useful for tests, but the project should prioritize data safety as the storage engine matures.
+
+## Reference
+
+Deferred from `docs/tasks/000120-secondary-index-runtime-access-and-recovery.md` and parent RFC `docs/rfcs/0014-dual-tree-secondary-index.md` Phase 4 review. The issue surfaced while reviewing recovery startup behavior around `TrxSysConfig::skip_recovery`, `log_recover(skip=true)`, and `TransactionSystem::new` timestamp initialization.
+
+## Deferred From (Optional)
+
+docs/tasks/000120-secondary-index-runtime-access-and-recovery.md; docs/rfcs/0014-dual-tree-secondary-index.md Phase 4
+
+## Deferral Context (Optional)
+
+- Defer Reason: The active task is focused on secondary-index runtime/recovery cutover. Removing the recovery skip option affects broader engine configuration and many tests, so it should be planned separately instead of widening the current task during resolve.
+- Findings: The current code exposes `TrxSysConfig::skip_recovery` and many tests set `.skip_recovery(true)`. A reviewed startup path showed that skip mode can bypass recovery entirely and hand a transaction timestamp seed directly to `TransactionSystem::new`. Even if individual bugs are patched, the option itself is unsafe as a storage-engine configuration because it permits startup without replaying durable logs.
+- Direction Hint: Prefer deleting the production configuration surface and making recovery mandatory. For tests, first classify why each `.skip_recovery(true)` call exists: fresh temporary root convenience, log subsystem isolation, or explicit recovery bypass. Replace fresh-root cases with normal recovery, isolate log-only tests behind private test fixtures, and avoid any public API that can start a persisted engine while skipping recovery.
+
+## Scope Hint
+
+Audit and remove the production-facing recovery skip option from configuration and startup flow. Replace test uses with safer test-only construction helpers or fixtures that do not bypass recovery for persistent storage roots. Keep any unavoidable no-recovery behavior private to tests and clearly separated from engine configuration.
+
+## Acceptance Hint
+
+`TrxSysConfig` no longer exposes a general `skip_recovery` option for engine startup. Production startup always runs recovery. Existing tests are migrated to safe alternatives, and tests that intentionally need an empty log or fresh temporary root express that setup without disabling recovery. Documentation and comments no longer present recovery skipping as a supported mode.
+
+## Notes (Optional)
+
+
+## Close Reason (Added When Closed)
+
+When a backlog item is moved to `docs/backlogs/closed/`, append:
+
+```md
+## Close Reason
+
+- Type: <implemented|stale|replaced|duplicate|wontfix|already-implemented|other>
+- Detail: <reason detail>
+- Closed By: <backlog close>
+- Reference: <task/issue/pr reference>
+- Closed At: <YYYY-MM-DD>
+```

docs/backlogs/next-id

@@ -1 +1 @@
-000087
+000089

docs/rfcs/0014-dual-tree-secondary-index.md

@@ -502,10 +502,10 @@ SAFETY:` comments, and run the repository lint gate. [D10], [C4], [C7]
     visibility authority, rebuilding checkpointed cold DiskTree entries into
     MemTree, removing the single-tree catalog implementation, and generic
     B+Tree backend unification.
-  - Task Doc: `docs/tasks/TBD.md`
-  - Task Issue: `#0`
-  - Phase Status: `pending`
-  - Implementation Summary: `pending`
+  - Task Doc: `docs/tasks/000120-secondary-index-runtime-access-and-recovery.md`
+  - Task Issue: `#559`
+  - Phase Status: done
+  - Implementation Summary: Implemented Phase 4 user-table dual-tree runtime and recovery cutover, preserving catalog single-tree indexes, routing foreground/rollback access through composite indexes, and using checkpointed DiskTree roots as cold recovery state. [Task Resolve Sync: docs/tasks/000120-secondary-index-runtime-access-and-recovery.md @ 2026-04-16]
 
 - **Phase 5: Cleanup, Eviction, And Parity Hardening**
   - Scope: remove migration scaffolding only after composite behavior is