From a41f91a2371c9d35fcaeaf192b88ab51e0742fce Mon Sep 17 00:00:00 2001 From: Artur Shiriev Date: Thu, 25 Jun 2026 18:13:32 +0300 Subject: [PATCH 1/3] chore: adopt canonical planning convention + compact CLAUDE.md Fresh-adopt lesnik512/planning-convention v1.0.0 and compact the 32K CLAUDE.md onto the architecture/ truth home. Convention (fresh adopt): - vendor canonical planning/index.py + _templates/* (adds release.md) - migrate 16 plan.md spec links (slug -> actual spec filename) so the canonical validator passes - merge the convention prose into planning/README.md; refresh the stale Index/Other sections (flat newest-first; full capability list) - Justfile: add check-planning recipe, wire it into lint-ci - architecture/README.md: promotion rule + capability map - planning/.convention-version = 1.0.0 - pyproject: per-file RUF100 ignore for the vendored index.py (its canonical D212 noqa is unused under this repo's D213 choice) CLAUDE.md compaction (202 -> 45 lines): - promote the 8 orphan Architecture subsections (no architecture/ home) into 5 new capability pages: producer, schema, subscriber, integration, retry -- following the existing "implementation detail" convention - rewrite ## Architecture as terse invariant-summaries + a capability index over all 11 architecture/*.md (the file's stated role) - compact Commands + Workflow to pointers (Justfile / planning/README.md are the sources of truth) just lint-ci + check-planning green. --- CLAUDE.md | 195 ++---------------- Justfile | 7 +- architecture/README.md | 34 +++ architecture/integration.md | 30 +++ architecture/producer.md | 35 ++++ architecture/retry.md | 57 +++++ architecture/schema.md | 76 +++++++ architecture/subscriber.md | 72 +++++++ planning/.convention-version | 1 + planning/README.md | 77 +++++-- planning/_templates/change.md | 8 +- planning/_templates/decision.md | 1 - planning/_templates/design.md | 8 +- planning/_templates/plan.md | 4 +- planning/_templates/release.md | 39 ++++ .../plan.md | 2 +- .../plan.md | 2 +- .../plan.md | 2 +- .../plan.md | 2 +- .../2026-06-09.01-mkdocs-github-pages/plan.md | 2 +- .../plan.md | 2 +- .../plan.md | 2 +- .../2026-06-11.01-operator-pages/plan.md | 2 +- .../plan.md | 2 +- .../2026-06-12.01-docs-tutorials/plan.md | 2 +- .../plan.md | 2 +- .../plan.md | 2 +- .../plan.md | 2 +- .../2026-06-19.02-docs-diataxis-nav/plan.md | 2 +- .../plan.md | 2 +- .../2026-06-23.01-client-rules-kernel/plan.md | 2 +- planning/index.py | 141 ++++++++++--- pyproject.toml | 4 + 33 files changed, 566 insertions(+), 255 deletions(-) create mode 100644 architecture/README.md create mode 100644 architecture/integration.md create mode 100644 architecture/producer.md create mode 100644 architecture/retry.md create mode 100644 architecture/schema.md create mode 100644 architecture/subscriber.md create mode 100644 planning/.convention-version create mode 100644 planning/_templates/release.md diff --git a/CLAUDE.md b/CLAUDE.md index 22c4feb..7a308f6 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,190 +8,35 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## Commands -- `just test` — full suite in docker compose (Postgres 17). Forwards args: `just test tests/test_unit.py -k name`. Args are forwarded **unquoted**, so a spaced `-k` expression (`-k "a or b"`) word-splits and fails (`file or directory not found: or`) — run one keyword per invocation, or use a single substring matching all targets. -- `just lint` — `eof-fixer`, `ruff format`, `ruff check --fix`, `ty check`. `just lint-ci` is the non-mutating variant. -- `just install` — `uv lock --upgrade && uv sync --all-extras --all-groups --frozen`. -- `just build` / `just down` / `just sh` — image build, teardown, shell into the app container. -- `just docs-serve` / `just docs-build` — serve docs locally at `http://127.0.0.1:8000` with hot-reload, or one-shot `mkdocs build --strict`. `just docs-deploy` is reserved for CI (force-pushes to `gh-pages`). +`just` (task runner) + `uv` (package manager); the [`Justfile`](Justfile) is the source of truth for recipes — run `just --list` or read it. The non-obvious bits: -`tests/test_unit.py` and `tests/test_fake.py` need no Postgres — `uv run pytest tests/test_unit.py` works directly. `tests/test_integration.py` requires Postgres at `POSTGRES_DSN` (default `postgresql+asyncpg://outbox:outbox@localhost:5432/outbox`); `pg_engine` skips if unreachable. Coverage is on by default with `--cov-fail-under=100` — partial runs fail that gate; pass `--no-cov` or `--cov-fail-under=0` when iterating locally. +- `just test [args]` — full suite in docker compose (Postgres 17). Args forward **unquoted**, so a spaced `-k` expression (`-k "a or b"`) word-splits and fails (`file or directory not found: or`) — run one keyword per invocation, or a single substring matching all targets. `tests/test_unit.py` + `tests/test_fake.py` need no Postgres (`uv run pytest tests/test_unit.py` works directly); `tests/test_integration.py` needs Postgres at `POSTGRES_DSN` (default `postgresql+asyncpg://outbox:outbox@localhost:5432/outbox`; `pg_engine` skips if unreachable). Coverage is on with `--cov-fail-under=100` — partial runs fail that gate; pass `--no-cov` or `--cov-fail-under=0` when iterating. +- `just lint` / `just lint-ci` — autofix vs non-mutating; `lint-ci` also runs the planning-bundle validator. +- `just docs-serve` / `just docs-build` — local hot-reload at `http://127.0.0.1:8000` / one-shot strict `mkdocs build`. ## Workflow -Per-feature: brainstorming → spec in `planning/changes/YYYY-MM-DD.NN-/design.md` → writing-plans → plan in `planning/changes/YYYY-MM-DD.NN-/plan.md` → executing-plans / subagent-driven-development → requesting-code-review → finishing-a-development-branch. Each change is a folder bundle; `` is a kebab-case description, not a story ID; `.NN` is a zero-padded intra-day counter that breaks same-date ties so the timeline sorts stably. `summary:` is written when the change is created (it is the change's one-liner); the implementing PR then sets `status: shipped`, fills `pr:` and `outcome:` in-branch, and promotes its conclusions into the affected `architecture/.md` **in the same PR, alongside the code** — the only ship-time step (there is no folder move), reviewed in the implementing diff rather than applied after merge; that hand-edit is what keeps `architecture/` true. A design decision taken **without** a code change — especially a candidate **rejected** with a load-bearing reason (e.g. an architecture-review suggestion we declined) — is recorded as `planning/decisions/YYYY-MM-DD-.md` (a `decision.md` template, frontmatter `status: accepted|superseded`), each with a **Revisit trigger** so future reviews don't re-litigate it. See [`planning/README.md`](planning/README.md) for the conventions (run `just index` for the change + decision listing) and [`planning/_templates/`](planning/_templates/) for copy-and-fill starting points. - -**Spec** (`design.md`) captures the *thinking* — why we are doing this, what the design is, what trade-offs were considered, what is out of scope. Written before code; rarely revised after merge. **Plan** (`plan.md`) captures the *sequencing* — the ordered checklist of tasks an executor (human or agent) walks. References the spec for the "why"; never re-explains it. **`architecture/`** captures the *invariants* of shipped systems — the living truth, promoted in the change's implementing PR alongside the code. A plan paragraph that would still read correctly with all task numbers and checkboxes removed is design content and belongs in the spec. - -**Three lanes.** Scale the artifact to the change. **Full** — a `design.md` + `plan.md` bundle — for real design judgment, a new file/module, a public-API change, cross-cutting/multi-file work, or non-trivial test design. **Lightweight** — a single `change.md` — for small-but-real changes (≲30 LOC net, ≤2 files, no new file, no public-API change, a single straightforward test). **Tiny** — no bundle, just a conventional commit — for a typo fix, dep bump, linter/formatter/CI tweak, a mechanical rename to satisfy a just-landed convention, or a single-line config change. Heavier lane wins on ambiguity; a `change.md` that outgrows its lane splits into `design.md` + `plan.md`. +Planning uses a portable convention — `architecture/` (repo root) is the living **truth home** and promotion target; `planning/changes/` holds the per-change bundles. Start at the [Quick path](planning/README.md#quick-path-start-here) in `planning/README.md` (the authoritative spec) to pick a lane — **Full** (`design.md` + `plan.md`), **Lightweight** (single `change.md`), or **Tiny** (just a commit) — and ship. `just check-planning` validates bundles; `just index` prints the change + decision listing; `planning/_templates/` are copy-and-fill starting points. A design decision taken **without** a code change — especially a **rejected** option with a load-bearing reason — goes in `planning/decisions/YYYY-MM-DD-.md` (`status: accepted|superseded`) with a **Revisit trigger** so future reviews don't re-litigate it. ## Architecture -The package wires a FastStream `Broker`/`Registrator`/`Subscriber` trio whose transport is Postgres rows, not a message bus. - -Deep-dives live in `architecture/`; this file holds the invariants Claude must not break, plus pointers. - -### Producer side - -`broker.publish(body, *, queue, session, headers=None, correlation_id=None, activate_in=None, activate_at=None, timer_id=None)` and `broker.publish_batch(*bodies, queue, session, headers=None, activate_in=None, activate_at=None)` insert outbox rows through the caller's `AsyncSession`. **They do not flush, commit, or open their own transaction** — the row must commit with the caller's domain writes. Both reject non-`AsyncSession` with `TypeError`. `publish` returns the row id (or `None` on `timer_id` conflict); `publish_batch` returns nothing and rejects `timer_id` (per-row dedup is meaningless in a batch). `broker.request` raises `NotImplementedError` (outbox is fire-and-forget). - -`OutboxProducer` (`publisher/producer.py`) implements `ProducerProto[OutboxPublishCommand]` and is the canonical insert path. `broker.publish` / `publish_batch` / `OutboxPublisher.publish` all build an `OutboxPublishCommand` (`response.py`) and route through `_basic_publish(cmd, producer=self.config.producer)` — encode + insert + NOTIFY semantics live in one place. Session-type / queue / activate-args-mutex / tz validation lives in one shared `_validate_publish_args` (`response.py`), called by the `OutboxPublishCommand` constructor, `OutboxResponse.__init__`, and `broker.publish_batch`'s empty-batch branch — so every real publish entry point (including an empty batch) rejects the same misconfigurations identically and eagerly (order: activate-args → session → queue). `from_cmd` raises (relay chaining is unsupported here). - -`broker.publisher(queue, *, headers=None, title=None, description=None, schema=None, include_in_schema=True)` returns an `OutboxPublisher` — a typed wrapper around `broker.publish` with the same transactional contract. Static decorator headers merge with per-call (per-call wins). The publisher exists for AsyncAPI / per-queue config — **not** decorator-relay chaining: `OutboxPublisher.__call__` raises `NotImplementedError` at decoration time. A relay decorator can't reach an `AsyncSession` without breaking the transactional contract. - -For chained publishing, handlers can `return OutboxResponse(body=..., queue=..., session=session)`. `OutboxResponse.__init__` validates eagerly via the shared `_validate_publish_args` (so a misconfigured response raises at the `return` site, not at dispatch where it would masquerade as a handler failure); `as_publish_command()` re-runs the same validator, keeping `OutboxPublishCommand` the authoritative source. FastStream gates `_make_response_publisher` on truthy `message.reply_to`; `OutboxParser.parse_message` sets `reply_to=msg.queue` to trip it. The actual publisher is `OutboxFakePublisher` (`publisher/fake.py`), which gates on `isinstance(cmd, OutboxPublishCommand)` so plain returns (`None`, `dict`, …) become silent no-ops. `correlation_id` propagates via FastStream's `process_message` inheritance. - -`_encode_payload` (`envelope.py`) is the internal helper that turns `body` into `(payload_bytes, headers_dict)`. Used by both producers; not exported. - -### Relay to foreign broker - -`OutboxSubscriber` can source a FastStream-native cross-broker chain: `@kafka_pub @broker_outbox.subscriber("q")` (Kafka/Rabbit/NATS/Redis/Confluent). Upstream's `SubscriberUsecase.process_message` walks the publisher chain — no dispatch override is needed for the chain itself. Three guardrails on top: - -- **Bad chain composition is refused.** `OutboxResponse(...)` + a non-`OutboxFakePublisher` in the chain raises `_OutboxConfigError` (private `RuntimeError` subclass) via `process_message` / `consume()` / `dispatch_one` overrides; the worker loop catches it, logs it at ERROR, and leaves the row — the lease expires and another fetch reclaims it (retry via lease expiry, **not** the `retry_strategy`) until the config is fixed (P18). -- **WARNING for unstarted foreign brokers at `start()`** — one per broker, deduped via `_warned_foreign_config_ids: set[int]`. -- **`propagate_inbound_headers: bool = False`** — when True, inbound headers fill `Response.headers` only if the handler returned a `Response` with empty headers (user-set wins). Default False matches FastStream convention. - -Deep dive: `architecture/relay.md`. User-facing: `docs/usage/relay.md`. - -### Timers (delayed delivery) - -`activate_in: timedelta` / `activate_at: datetime` (mutually exclusive) set `next_attempt_at`; the fetch CTE's `next_attempt_at <= now()` gates eligibility. For `publish`, `activate_in` is computed server-side via `make_interval` (clock-skew-safe) while `activate_at` is bound as the caller's absolute literal; `publish_batch` is fully client-side. - -`timer_id` (single `publish` only) → partial unique index `(queue, timer_id) WHERE timer_id IS NOT NULL`. Producer uses `pg_insert(...).on_conflict_do_nothing(...)` — re-publishing the same id is a no-op (returns `None`). NOTIFY is skipped when future-dated OR the conflict suppressed the insert. The dedup window is **one *live* row per `(queue, timer_id)`** — it resets once the row is delivered (DELETEd) or terminally fails, so `timer_id` is "at most one in flight", not a global once-ever idempotency key (the DLQ keeps `timer_id` non-unique). - -`broker.cancel_timer(*, queue, timer_id, session)` issues `DELETE WHERE queue=? AND timer_id=? AND acquired_token IS NULL` — **the `acquired_token IS NULL` guard is load-bearing** (preserves the lease-token invariant; returns `False` if a handler is in flight). - -Deep dive: `architecture/timers.md`. User-facing: `docs/usage/timers.md`. - -### User-owned schema - -`make_outbox_table(metadata, table_name="outbox")` returns a `sqlalchemy.Table` on the user's `MetaData`. The package never creates or migrates — that's Alembic — but declares three partial indexes so autogenerate brings them up: - -- `(queue, next_attempt_at) WHERE acquired_token IS NULL` — fetch CTE Branch A (unleased). -- `(queue, acquired_at) WHERE acquired_token IS NOT NULL` — fetch CTE Branch B (expired-lease reclaim). -- unique `(queue, timer_id) WHERE timer_id IS NOT NULL` — `timer_id` dedup. - -Plus a `CHECK ((acquired_token IS NULL) = (acquired_at IS NULL))` (the `_lease_ck` constraint) so a half-set lease is unrepresentable. - -The fetch CTE's OR is written so each disjunct **explicitly carries its partial-index predicate as a conjunct** — Postgres only uses a partial index when the query implies its WHERE clause; the naive form falls back to seq-scan. Both fetch indexes pay write amplification on every claim. The index also satisfies the `ORDER BY next_attempt_at, id` **only for a single-queue subscriber** — a subscriber serving multiple queues (`queue = ANY(:queues)`), or the expired-lease branch (ordered by `next_attempt_at` while `_lease_idx` is keyed on `acquired_at`), adds a `LIMIT`-bounded sort node. Prefer one subscriber per queue when fetch ordering cost matters (same segregation pattern as lease TTLs). - -The `ORDER BY` lives on the inner CTE that **selects + LIMITs** the rows; the outer `UPDATE … RETURNING *` is unordered, so the order rows *dispatch* within a single fetch batch is unspecified (F2-09). The ordering governs *which* rows are claimed under contention (FIFO selection), not the per-row dispatch sequence — irrelevant with `max_workers > 1` anyway. Don't rely on within-batch FIFO delivery. - -There is **no `state` column**: a row is "available" iff `acquired_token IS NULL` or `acquired_at < now() - lease_ttl_seconds`. Terminal failures `DELETE` by default; opt in to audit via `dlq_table=make_dlq_table(metadata)`. - -`validate_schema()` is **opt-in** (call from `/health` or startup hook, not `broker.start()`) so migrations can run against the same DB without a loop. Beyond the alembic column/index diff it also probes the live partial-index **predicates** (alembic ignores `postgresql_where`), catching a drifted or non-partial `timer_id_uq` that would otherwise break `ON CONFLICT` at publish time (S2), **and probes `pg_constraint` for the `
_lease_ck` CHECK** (alembic has no check-constraint comparator), catching a missing or drifted lease pairing. Because these two probes (predicates + CHECK) catch drift `alembic revision --autogenerate` **cannot** remediate (no check-constraint comparator; index comparator ignores `postgresql_where`), the raised `RuntimeError` appends a pointer to `docs/operations/alembic.md#fixing-drift-autogenerate-cant-see` (the hand-written-migration recipe) — but **only** when one of those two probes fired; autogenerate-fixable drift (columns, plain indexes, DLQ) gets no pointer. Message composition lives in `_compose_schema_mismatch_message` (`client.py`), gated on `has_blind_drift`. Alembic is optional (`faststream-outbox[validate]`); without it `validate_schema()` raises `ImportError` but every other path works. - -### Opt-in DLQ on terminal failure - -`make_dlq_table(metadata, table_name="outbox_dlq")` + `OutboxBroker(..., dlq_table=...)` archives terminal failures. With `dlq_table=None` every existing code path is **bit-for-bit identical**. - -Atomicity: `delete_with_lease` switches to a single CTE `WITH deleted AS (DELETE … RETURNING …) INSERT INTO SELECT …` — preserves writer-connection autocommit + lease-token guard. INSERT failure rolls back DELETE, so DLQ misconfiguration surfaces as outbox growth + `lease_lost` spikes, not silent audit loss. - -**DLQ projection.** The outbox→DLQ column mapping is `_DLQ_PROJECTION` (+ `_DLQ_INJECTED_COLUMNS`) in `schema.py` — one declarative `(outbox_col, dlq_col)` list that the real CTE (`OutboxClient._build_dlq_cte_stmt`) and the fake (`FakeOutboxClient.delete_with_lease`) both build their column lists from. Adding a DLQ column is one edit there, not hand-kept parity across SQL and Python. - -`OutboxInnerMessage.terminal_failure_reason` is set on three paths: `allow_delivery` False → `"max_deliveries"`, `_nack` exhausted → `"retry_terminal"`, `_reject` → `"rejected"`. **Branch on `terminal_failure_reason` BEFORE `last_exception`** in `dispatch_one` so manual `await msg.reject()` (no exception raised) routes to `nacked_terminal(reason="rejected")`, not `acked`. - -**The `DLQFailureReason` `Literal` (`message.py`) is the public contract** — operator queries / dashboards key off these values; changes are API-breaking. - -`last_exception` defaults to `repr()` bounded by `_LAST_EXCEPTION_MAX_CHARS=8192` (`subscriber/usecase.py`); truncation appends `…[truncated]`. Because a `repr` can embed the payload / request body / credentials, `OutboxBroker(..., last_exception_renderer=...)` (F3-01) lets PII-sensitive deployments redact (e.g. `lambda exc: type(exc).__name__`) or drop it (return `None`); the rendered string is still length-capped. Rendering happens in `_render_last_exception` (`subscriber/usecase.py`), read from `OutboxBrokerConfig.last_exception_renderer`. DLQ `failure_reason` is `String(64)`. No built-in retention. - -Deep dive: `architecture/dlq.md`. User-facing: `docs/usage/dlq.md`. - -### Two-loop subscriber (`subscriber/usecase.py`) - -Per subscriber: - -1. **`_fetch_loop`** — long-lived `AsyncConnection` for the fetch CTE + separate raw asyncpg connection for `LISTEN outbox_
`. Single CTE: `SELECT … FOR UPDATE SKIP LOCKED → UPDATE acquired_token=:uuid, acquired_at=now() RETURNING *`. WHERE reclaims unleased rows **and** expired leases (`acquired_at < now() - make_interval(secs => :lease_ttl)`) — no separate reaper. NOTIFY shortcircuits idle sleep via `asyncio.Event` (idle latency from `max_fetch_interval` to ~10ms). LISTEN failures log once and fall back to polling. DB error → connections close, exponential backoff (`_BACKOFF_EXP_CAP=30`), reopen. -2. **`_worker_loop`** × `max_workers` — pulls from `asyncio.Queue(maxsize=fetch_batch_size)`, dispatches via `consume()`, flushes terminal state. Each worker owns a long-lived `AsyncConnection` (held across reconnect) and routes terminal writes through `delete_with_lease(conn, …)` / `mark_pending_with_lease(conn, …)` — drain of N rows costs O(workers) pool checkouts. Flush exceptions propagate (outer loop rebuilds the connection); inflight slot still releases in `finally`. Default `AckPolicy.NACK_ON_ERROR`; `REJECT_ON_ERROR` and `MANUAL` allowed. **`AckPolicy.ACK_FIRST` is rejected at registration with `ValueError`** — it would delete before the handler runs, defeating the outbox contract. `subscriber/factory.py` raises or warns on other footguns (`lease_ttl_seconds <= max_fetch_interval`, `max_deliveries` without retry, etc.). - -`OutboxSubscriber.get_one()` and `__aiter__()` are explicit `NotImplementedError`s — point operators at `broker.fetch_unprocessed(session=..., queue=...)`. A peek that acquires a lease has surprising `deliveries_count` semantics; lease-free reads belong on `fetch_unprocessed`. - -**Lease bound.** A subscriber can hold up to `fetch_batch_size + max_workers` leases at once, not `fetch_batch_size` (F1-01): `free = _inflight.maxsize - qsize()` counts only *queued* rows, so once `max_workers` rows are checked out for processing the loop can claim another full batch. Leases are bounded and self-expire via TTL, but size `lease_ttl_seconds` and reason about cross-replica contention against `fetch_batch_size + max_workers`, not `fetch_batch_size`. +> Quick orientation + the invariants Claude must not break. Each capability's full implementation detail lives in its `architecture/.md` (the truth home); user-facing docs live in `docs/`. **When a change alters a capability's behavior, update the matching `architecture/.md` in the same PR** — that promotion is what keeps `architecture/` true. -**Connection budget.** Each subscriber holds `max_workers + 1` SQLAlchemy pool connections steady-state + one raw asyncpg connection for LISTEN. Size the pool for `Σ subscribers × (max_workers + 1)` or startup blocks on checkout — the asyncpg LISTEN connection lives **outside** the pool, so it does not count toward pool sizing. **Per process** — Postgres `max_connections` must cover `replicas × Σ subscribers × (max_workers + 2)`: the `max_workers + 1` pool connections **plus** the out-of-pool asyncpg LISTEN connection. Undersize it and rolling deploys hit `FATAL: too many connections`. - -**NOTIFY semantics.** `broker.publish` / `publish_batch` emit `SELECT pg_notify('outbox_
', queue)` on the caller's session right after the INSERT, **except** when future-dated or `timer_id` conflict no-op'd the insert. NOTIFY is transactional — atomicity with the row is automatic; rolled-back transactions silently drop it. The future-dated decision is one shared `is_future_dated(activate_in, activate_at, now)` (in the stdlib-only leaf `_scheduling.py`, alongside `resolve_next_attempt_client_side` and `validate_activate_args` — the pure activate-args helpers the real and fake publish paths share); `activate_at`'s comparison and the `publish_batch` `next_attempt_at` are **worker-clock-relative** (unlike `activate_in`'s server-side `make_interval`), so under worker/DB clock skew NOTIFY may fire slightly early/late — polling backstops it (F2-04/F2-05). NOTIFYs emitted **during a fetch-loop reconnect/backoff window are lost** (LISTEN is not durable); latency degrades to the poll interval until the next tick — a latency, not a correctness, gap (F1-07). Channel naming is `outbox_`. Postgres limits identifiers to 63 bytes; `make_outbox_table` **raises `ValueError`** when the longest derived identifier — an index name like `
_pending_idx`, longer than the NOTIFY channel itself — would exceed it — so over-long table names (~>51 bytes) are rejected at construction, not silently degraded to polling. - -### Lease-token invariant — load-bearing - -Every terminal write (`delete_with_lease`, `mark_pending_with_lease`) filters on `acquired_token`. If a slow handler's lease expired and a newer fetch reclaimed the row, the slow handler's `DELETE`/`UPDATE` finds `rowcount == 0` and is silently dropped — preventing it from clobbering the new lease holder. **Any new fetch/terminal path must preserve this.** - -`lease_ttl_seconds` (default `60.0`, per-subscriber) **must exceed handler P99 with margin** or healthy handlers race their own expiry. The lease cutoff uses server-side `make_interval(secs => :lease_ttl)` — immune to clock skew. **Sizing tip**: route occasional slow work onto its own subscriber with a tall TTL; keep the fast subscriber's tight. TTL is per-subscriber, so segregation costs only an extra `@broker.subscriber(...)`. - -Lease-loss logs at WARNING with `extra={"event": "lease_lost", "phase": "terminal"|"retry", "row_id": …, "queue": …, "deliveries_count": …}`. Recurring `event=lease_lost` means `lease_ttl_seconds < handler P99`. - -`deliveries_count` counts **claims, not completed handler runs** — the fetch CTE increments it on every claim, including expired-lease reclaims (F2-07). Under lease churn (`lease_ttl < handler P99`) a row can cross `max_deliveries` after fewer than N successful handler invocations, so set `max_deliveries` with margin. `attempts_count` (via `_record_attempt`) is the handler-run-scoped counter. - -**Writer-connection autocommit.** `_open_worker_resources` sets the per-worker writer to `isolation_level="AUTOCOMMIT"`. Terminal writes are single statements; explicit BEGIN/COMMIT would add two round-trips per row. The `WHERE acquired_token = …` clause enforces the invariant, not the transaction wrapping. The fetch connection is **not** autocommit — it owns LISTEN/NOTIFY and amortizes BEGIN/COMMIT across the batch. - -**Shutdown race in `dispatch_one`.** If `stop()` flips `running=False` between a worker pulling a row from `_inflight` and entering `consume()`, base `SubscriberUsecase.consume()` early-exits without running the handler. `dispatch_one` detects this (`not row.state_set and not self.running` after `consume()` returns without raising) and returns before `assert_state_set → reject() → _safe_flush` would silently DELETE. Lease lives until expiry; another replica reclaims. No metric fires. **Without this guard, busy subscribers leak rows on every rolling deploy.** - -### Drain on stop (subscriber + broker) - -Both `OutboxSubscriber.stop()` and `OutboxBroker.stop()` override FastStream parents. Override comments carry `# Upstream equivalent (replaced): …`. - -- **Subscriber: two flags during drain.** `self.running` (FastStream's "actively dispatching") stays True for the duration of drain; `self._stopping` (new) signals "no new claims". `_fetch_inner` checks both; the worker loop only `running`. `stop()` flips `_stopping`, kicks `_notify_event`, waits up to `graceful_timeout` for `_inflight.join()`, then flips `running=False` and cancels tasks. `graceful_timeout=None` (unbounded for `ping()`) is **clamped to a finite fallback in the drain** so one wedged handler can't hang `stop()` forever. `super().stop()` is **not** called — its `MultiLock.wait_release` would re-wait stuck handlers for another full budget (2× shutdown regression). -- **Broker: parallel-gather subscriber stop** via `asyncio.gather(..., return_exceptions=True)` — sequential N × `graceful_timeout` exceeds K8s default `terminationGracePeriodSeconds=30s` once a service has 2+ subscribers. Exceptions logged via `_log_subscriber_stop_error`, never re-raised. -- **Phase interaction.** During drain `running` stays True so the `dispatch_one` guard is dormant; after drain `running=False` is set before `task.cancel()` so workers mid-`dispatch_one` benefit from the guard. The two changes are complementary. -- **Upstream divergence flag.** If FastStream adds cleanup to `BrokerUsecase.stop`, `SubscriberUsecase.stop`, or `TasksMixin.stop`, we silently miss it. **Re-check both overrides when touching shutdown.** Regression tests in `tests/test_fake.py` (`test_drain_finishes_inflight_rows_before_returning_in_fake_mode`, `test_broker_stop_cancels_wedged_handler_within_graceful_timeout_in_fake_mode`) and the Postgres-backed `tests/test_integration.py`. -- **Test-broker gotcha.** `_fake_close` sets `sub.running = False` and bypasses `subscriber.stop()` / `broker.stop()` entirely — drain tests must `await broker.stop()` explicitly inside the `async with` block. - -Deep dive: `architecture/drain.md`. - -### Test broker - -`TestOutboxBroker` (`testing.py`) swaps in `FakeOutboxClient` (in-memory `_FakeRow` dicts). Two modes: - -- **Sync (`run_loops=False`, default)** — `broker.publish` routes through `OutboxSubscriber.dispatch_one` synchronously; handler runs before `publish` returns. `producer` slot is swapped for `FakeOutboxProducer` so `broker.publisher("q").publish(...)` lands in the same store. Future-dated rows **fire immediately** in sync mode (sync dispatch ignores `next_attempt_at`). -- **Loop (`run_loops=True`)** — real `_fetch_loop` / `_worker_loop` against the fake client. Needed for retry rescheduling, lease-expiry reclaim, scheduled delivery firing. - -`OutboxSubscriber.dispatch_one(row)` is the public per-row entry point — worker loop and test broker both call it. Caller must hold the row's lease. - -`FakeOutboxClient.validate_schema()` raises `NotImplementedError` — a silent pass would let users ship broken schemas while tests stay green. Use a real `OutboxClient(real_engine, table)` for schema validation tests. - -**Client contract.** `OutboxClient` (SQL) and `FakeOutboxClient` (Python) implement the same rules in different substrates — they can't share code, so `tests/test_client_contract.py` pins them to one behavioural contract: a single parametrized scenario module run against **both** adapters (fake everywhere; real Postgres auto-skipped when unreachable) over the shared `AbstractOutboxClient` surface (`fetch` / `delete_with_lease` / `mark_pending_with_lease` + DLQ). Drift fails a test instead of shipping. It pins *structural* drift only — an in-process test can't manufacture cross-host DB-vs-worker clock skew, so the real client's server-side `make_interval` clock authority stays a documented invariant, not an assertion. `cancel_timer` and `timer_id` insert-dedup are broker/producer concerns, not on the client interface, so they live in `test_integration.py` / `test_fake.py`. - -**Session leniency.** The fake `publish` / `publish_batch` / `cancel_timer` / `fetch_unprocessed` all `del session` — any value (incl. `None`) is accepted, diverging from production's `isinstance(session, AsyncSession)` `TypeError` (F4-09). Tests that need to assert the session contract must use the real `OutboxClient` / a real `AsyncSession`, not the fake. `OutboxResponse` is **not** faked, so its eager session/queue/activate validation does fire under the test broker. - -**Gotcha:** subscribers registered via `OutboxRouter` (then `broker.include_router(router)`) live on the router, not `broker._subscribers`. Walk `broker.subscribers` (the property) for full introspection. - -Deep dive: `architecture/test-broker.md`. User-facing: `docs/usage/testing.md`. - -### Annotations module (`annotations.py`) - -Canonical home for `Annotated[..., Context(...)]` shortcuts — `OutboxMessage`, `OutboxBroker`, `OutboxProducer`, `OutboxClient`. Each shadows the underlying class via `from … import X as _X`. Producer path: `Context("broker._producer")` (via `BrokerUsecase._producer` property → `self.config.producer`). Client path: `Context("broker.config.broker_config.client")` (client lives only on the outbox-specific config layer). `faststream_outbox.fastapi` re-exports with FastAPI-aware `Context` (from `faststream._internal.fastapi.context`). - -### FastAPI router (`fastapi/router.py`) - -`OutboxRouter` subclasses FastStream's `StreamRouter` (which subclasses `APIRouter`). `app.include_router(router)` auto-starts the inner `OutboxBroker` via FastAPI lifespan. - -Critical for the transactional contract: `wrap_callable_to_fastapi_compatible` (FastStream internals) bridges FastAPI's dependency resolver into the consume pipeline, so `Depends(get_session)` inside a handler resolves the same `AsyncSession` it would in an HTTP route — and `OutboxResponse(session=...)` commits the follow-on row with the handler's domain writes. - -`subscriber()` and `publisher()` are overridden to pin defaults for FastAPI-specific kwargs (`response_model=Default(None)`, etc.) that the base declares keyword-only without defaults. Outbox kwargs flow through unchanged. `apply_types` and broker `dependencies` are intentionally **not exposed**: `StreamRouter` forces `apply_types=False` (FastDepends takes over), and the broker's `Dependant` list isn't useful in this flow. - -`fastapi` is an optional dependency (`faststream-outbox[fastapi]`). - -### Engine ownership - -Caller owns the `AsyncEngine` — the broker never disposes it. The engine lives on `OutboxBrokerConfig` (set by the broker constructor) and may be `None` until wired, so the broker can be constructed before the engine exists (used by the test broker). - -### Metrics + native middleware - -Two complementary seams — **don't collapse them.** - -- **Recorder seam** (`OutboxBroker(..., metrics_recorder=...)`): `Callable[[str, Mapping[str, Any]], None]`. Subscriber emits `fetched`, `dispatched`, `acked`, `nacked_retried`, `nacked_terminal`, `lease_lost`, `drain_timeout` (on a timed-out `stop()` drain), plus `dlq_written` when `dlq_table` is set. Producer emits `published`. The bundled Prometheus/OTel adapters translate every one of these (`drain_timeout` → `_outbox_drain_timeout_total` / `messaging.outbox.drain_timeout`). Default `_noop_recorder` lets sites fire unconditionally. Every call site is wrapped in `try/except` + DEBUG log. **Recorder must not block** (sync `Counter.inc()` fine; HTTP/StatsD not). `dlq_written` vs `nacked_terminal` divergence detects DLQ misconfiguration. -- **Native middleware** (`opentelemetry/`, `prometheus/`): thin subclasses of upstream's `TelemetryMiddleware[OutboxPublishCommand]` and `PrometheusMiddleware[OutboxInnerMessage, OutboxPublishCommand]`. Register via the public `OutboxBroker(..., middlewares=[...])` constructor kwarg (forwarded internally as `broker_middlewares`). Fire on `consume_scope` (via `dispatch_one → self.consume(row)`) and `publish_scope` (via `_basic_publish`). - -Why two: middleware owns `consume_scope` / `publish_scope` (spans, durations, status, size). Recorder owns events **outside** the bus — `fetched` (no `StreamMessage` at fetch time), `lease_lost` (after `consume_scope` exits), `nacked_terminal(reason="max_deliveries")` (before consume opens). Each fires for events the other physically cannot observe. - -Bundled adapters are optional extras (`[prometheus]` / `[opentelemetry]`). Canonical `messaging.system` / `broker` label is `"outbox"` (shared by both seams). Prometheus tags consume by `handler`, publish by `destination` (mirrors upstream). OTel adapter is meter-only; spans go via native middleware. - -Deep dive: `architecture/metrics.md`. User-facing: `docs/usage/observability.md`. - -### Retry strategies (`retry.py`) - -`get_next_attempt_delay(*, first_attempt_at, last_attempt_at, attempts_count, exception=None)` returns the **delay in seconds** before the next attempt (the DB computes `next_attempt_at` from it server-side, so timing is skew-immune), or `None` for terminal failure. It receives the raised exception so subclasses can retry only on transient errors. `_RetryStrategyTemplate` enforces `max_attempts` and `max_total_delay_seconds`. `ExponentialRetry` has optional jitter and `max_delay_seconds`. `max_total_delay_seconds` is a **lower bound** on the horizon: `elapsed` is measured `last_attempt_at − first_attempt_at` (both set equal on the first attempt), so the budget always permits roughly one more interval beyond the nominal cap (F2-01) — size it as "at least this long", not an exact ceiling. +The package wires a FastStream `Broker`/`Registrator`/`Subscriber` trio whose transport is Postgres rows, not a message bus. -**Default**: a subscriber with no explicit `retry_strategy` resolves to `ExponentialRetry(initial_delay_seconds=1.0, multiplier=2.0, max_delay_seconds=300.0, max_attempts=10, jitter_factor=0.2)` (`_default_retry_strategy()` in `registrator.py`). "Delete on first error" is the wrong default for an outbox; opt in with `NoRetry()`. +**Invariants — do not break (detail in the linked capability file):** + +- **Producer / transactional contract** — `broker.publish`/`publish_batch` insert through the caller's `AsyncSession` and **never flush/commit/open their own transaction**; the row commits with the caller's domain writes. Non-`AsyncSession` → `TypeError`; `broker.request` → `NotImplementedError`. → [producer.md](architecture/producer.md) +- **Relay to foreign broker** — native FastStream publisher-chain (`@kafka_pub @broker_outbox.subscriber("q")`); bad chain composition raises `_OutboxConfigError` and retries via **lease expiry**, not the retry strategy. → [relay.md](architecture/relay.md) +- **Timers** — `activate_in`/`activate_at` are mutually exclusive and gate eligibility; `timer_id` is "at most one *live* row per `(queue, timer_id)`", not a global idempotency key; `cancel_timer`'s `acquired_token IS NULL` guard is load-bearing. → [timers.md](architecture/timers.md) +- **User-owned schema** — caller owns the table; the three partial indexes + the `
_lease_ck` CHECK are load-bearing; **no `state` column**; `validate_schema()` is opt-in. → [schema.md](architecture/schema.md) +- **Opt-in DLQ** — with `dlq_table=None` every path is **bit-for-bit identical**; the `DLQFailureReason` `Literal` is the **public contract** (changes are API-breaking); branch on `terminal_failure_reason` **before** `last_exception`. → [dlq.md](architecture/dlq.md) +- **Two-loop subscriber + lease-token invariant** — fetch loop + N worker loops; **every terminal write filters on `acquired_token`** (a stale write finds `rowcount==0` and is dropped) — any new fetch/terminal path must preserve this; `AckPolicy.ACK_FIRST` is rejected at registration; `lease_ttl_seconds` must exceed handler P99. → [subscriber.md](architecture/subscriber.md) +- **Drain on stop** — custom `stop()` overrides + the `dispatch_one` shutdown-race guard prevent row leaks on rolling deploys; **re-check both overrides when touching shutdown**. → [drain.md](architecture/drain.md) +- **Test broker** — `TestOutboxBroker` swaps in `FakeOutboxClient`; sync mode dispatches the handler before `publish` returns; `test_client_contract.py` pins real-vs-fake parity. → [test-broker.md](architecture/test-broker.md) +- **Integration** — annotations `Context` shortcuts; `OutboxRouter` bridges FastAPI deps so `OutboxResponse(session=...)` commits with the handler's writes; caller owns the `AsyncEngine` (broker never disposes it). → [integration.md](architecture/integration.md) +- **Metrics + native middleware** — two complementary seams, **don't collapse them**; each fires for events the other physically cannot observe; canonical `messaging.system` label is `"outbox"`. → [metrics.md](architecture/metrics.md) +- **Retry strategies** — `get_next_attempt_delay` returns delay-seconds or `None` (terminal); the default is `ExponentialRetry(...)`, **not** delete-on-error (`NoRetry()` to opt out); `max_total_delay_seconds` is a lower bound. → [retry.md](architecture/retry.md) ## Conventions diff --git a/Justfile b/Justfile index 8e24fd8..446852e 100644 --- a/Justfile +++ b/Justfile @@ -27,11 +27,16 @@ lint-ci: uv run ruff format --check uv run ruff check --no-fix uv run ty check + uv run python planning/index.py --check -# Print the planning change index (grouped by status) to stdout. +# Print the planning change index (flat, newest-first) to stdout. index: uv run python planning/index.py +# Validate planning bundles + decisions (frontmatter, lanes, spec links); CI runs this. +check-planning: + uv run python planning/index.py --check + publish: rm -rf dist uv version $GITHUB_REF_NAME diff --git a/architecture/README.md b/architecture/README.md new file mode 100644 index 0000000..3de782e --- /dev/null +++ b/architecture/README.md @@ -0,0 +1,34 @@ +# Architecture + +The living truth about what `faststream-outbox` does **now** — one file per +capability, updated by hand whenever a change ships. The *why* and *how it got +here* live in [`../planning/changes/`](../planning/changes/), and decisions +deliberately taken (including options rejected) in +[`../planning/decisions/`](../planning/decisions/); this directory is the present. + +Each capability file is an **implementation-detail** page. Its terse +**invariant summary** ("what Claude must not break") lives in +[`../CLAUDE.md`](../CLAUDE.md) § Architecture; the **user-facing** account lives +under `../docs/`. + +These files carry **no frontmatter** — they are prose, dated by git. + +## Capabilities + +- [producer.md](producer.md) — the publish path and transactional contract. +- [relay.md](relay.md) — foreign-broker relay chain and guardrails. +- [timers.md](timers.md) — delayed delivery, `timer_id` dedup, `cancel_timer`. +- [schema.md](schema.md) — the user-owned table, partial indexes, `validate_schema()`. +- [dlq.md](dlq.md) — opt-in dead-letter on terminal failure. +- [subscriber.md](subscriber.md) — the two-loop subscriber and lease-token invariant. +- [drain.md](drain.md) — graceful drain on stop. +- [test-broker.md](test-broker.md) — `TestOutboxBroker`, the fake client, the client contract. +- [integration.md](integration.md) — annotations, FastAPI router, engine ownership. +- [metrics.md](metrics.md) — the recorder and native-middleware seams. +- [retry.md](retry.md) — retry strategies. + +## Promotion rule + +Shipping a change hand-edits the affected capability file(s) here to match the +new reality, in the same PR as the code. The change bundle stays in place under +[`../planning/changes/`](../planning/changes/) — no folder move. diff --git a/architecture/integration.md b/architecture/integration.md new file mode 100644 index 0000000..56148a8 --- /dev/null +++ b/architecture/integration.md @@ -0,0 +1,30 @@ +# Integration: annotations, FastAPI router, engine ownership — implementation detail + +User-facing: `docs/` (FastAPI integration). Invariant summary: `CLAUDE.md` § Integration. + +## Annotations module (`annotations.py`) + +This module is the canonical home for the `Annotated[..., Context(...)]` shortcuts — `OutboxMessage`, `OutboxBroker`, `OutboxProducer`, and `OutboxClient`. Each shortcut shadows its underlying class, which is imported via `from … import X as _X` so the public name can be re-bound to the `Annotated` form while the plain class stays available under its `_`-prefixed alias. + +Two of the shortcuts resolve through non-obvious attribute paths: + +- **Producer path**: `Context("broker._producer")`. This resolves via the `BrokerUsecase._producer` property, which returns `self.config.producer`. +- **Client path**: `Context("broker.config.broker_config.client")`. The client lives only on the outbox-specific config layer, so the shortcut points at it directly rather than through a broker property. + +`faststream_outbox.fastapi` re-exports these shortcuts with a FastAPI-aware `Context` (sourced from `faststream._internal.fastapi.context`). + +## FastAPI router (`fastapi/router.py`) + +`OutboxRouter` subclasses FastStream's `StreamRouter`, which itself subclasses `APIRouter`. Calling `app.include_router(router)` auto-starts the inner `OutboxBroker` via the FastAPI lifespan. + +This bridge is critical for the transactional contract. `wrap_callable_to_fastapi_compatible` (a FastStream internal) bridges FastAPI's dependency resolver into the consume pipeline, so a `Depends(get_session)` inside a handler resolves the same `AsyncSession` it would in an HTTP route — and `OutboxResponse(session=...)` commits the follow-on row together with the handler's domain writes. + +`subscriber()` and `publisher()` are overridden to pin defaults for FastAPI-specific kwargs (such as `response_model=Default(None)`) that the base declares keyword-only without defaults. The outbox kwargs flow through unchanged. + +`apply_types` and the broker `dependencies` are intentionally not exposed: `StreamRouter` forces `apply_types=False` (FastDepends takes over), and the broker's `Dependant` list isn't useful in this flow. + +`fastapi` is an optional dependency (`faststream-outbox[fastapi]`). + +## Engine ownership + +The caller owns the `AsyncEngine` — the broker never disposes it. The engine lives on `OutboxBrokerConfig` (set by the broker constructor) and may be `None` until wired, so the broker can be constructed before the engine exists (used by the test broker). diff --git a/architecture/producer.md b/architecture/producer.md new file mode 100644 index 0000000..105292f --- /dev/null +++ b/architecture/producer.md @@ -0,0 +1,35 @@ +# Producer / publish path — implementation detail + +User-facing: `docs/usage/` (publishing). Invariant summary: `CLAUDE.md` § Producer. + +## The transactional contract + +`broker.publish(body, *, queue, session, headers=None, correlation_id=None, activate_in=None, activate_at=None, timer_id=None)` and `broker.publish_batch(*bodies, queue, session, headers=None, activate_in=None, activate_at=None)` insert outbox rows through the caller's `AsyncSession`. They do not flush, commit, or open their own transaction — the row must commit with the caller's domain writes. + +Both reject a non-`AsyncSession` with `TypeError`. `publish` returns the row id (or `None` on a `timer_id` conflict); `publish_batch` returns nothing and rejects `timer_id` (per-row dedup is meaningless in a batch). `broker.request` raises `NotImplementedError` (the outbox is fire-and-forget). + +## OutboxProducer + the single insert path + +`OutboxProducer` (`publisher/producer.py`) implements `ProducerProto[OutboxPublishCommand]` and is the canonical insert path. `broker.publish`, `publish_batch`, and `OutboxPublisher.publish` all build an `OutboxPublishCommand` (`response.py`) and route through `_basic_publish(cmd, producer=self.config.producer)` — encode + insert + NOTIFY semantics live in one place. + +Session-type / queue / activate-args-mutex / tz validation lives in one shared `_validate_publish_args` (`response.py`), called by the `OutboxPublishCommand` constructor, `OutboxResponse.__init__`, and `broker.publish_batch`'s empty-batch branch — so every real publish entry point (including an empty batch) rejects the same misconfigurations identically and eagerly. The checks run in a fixed order: activate-args → session → queue. + +`from_cmd` raises (relay chaining is unsupported here). + +## Publisher wrapper + +`broker.publisher(queue, *, headers=None, title=None, description=None, schema=None, include_in_schema=True)` returns an `OutboxPublisher` — a typed wrapper around `broker.publish` with the same transactional contract. Static decorator headers merge with per-call headers (per-call wins). + +The publisher exists for AsyncAPI / per-queue config — not decorator-relay chaining: `OutboxPublisher.__call__` raises `NotImplementedError` at decoration time. A relay decorator can't reach an `AsyncSession` without breaking the transactional contract. + +## Chained publishing via OutboxResponse + +For chained publishing, handlers can `return OutboxResponse(body=..., queue=..., session=session)`. + +`OutboxResponse.__init__` validates eagerly via the shared `_validate_publish_args` (so a misconfigured response raises at the `return` site, not at dispatch where it would masquerade as a handler failure); `as_publish_command()` re-runs the same validator, keeping `OutboxPublishCommand` the authoritative source. + +FastStream gates `_make_response_publisher` on a truthy `message.reply_to`; `OutboxParser.parse_message` sets `reply_to=msg.queue` to trip it. The actual publisher is `OutboxFakePublisher` (`publisher/fake.py`), which gates on `isinstance(cmd, OutboxPublishCommand)` so plain returns (`None`, `dict`, …) become silent no-ops. `correlation_id` propagates via FastStream's `process_message` inheritance. + +## Payload encoding + +`_encode_payload` (`envelope.py`) is the internal helper that turns `body` into `(payload_bytes, headers_dict)`. It is used by both producers and is not exported. diff --git a/architecture/retry.md b/architecture/retry.md new file mode 100644 index 0000000..ab072ca --- /dev/null +++ b/architecture/retry.md @@ -0,0 +1,57 @@ +# Retry strategies — implementation detail + +User-facing: `docs/usage/` (retries). Invariant summary: `CLAUDE.md` § Retry. + +## get_next_attempt_delay + +Retry strategies live in `retry.py`. The core method, +`get_next_attempt_delay(*, first_attempt_at, last_attempt_at, attempts_count, exception=None)`, +returns the delay in seconds before the next attempt, or `None` to signal +terminal failure. + +The returned value is a delay, not an absolute timestamp: the DB computes +`next_attempt_at` from it server-side, so the timing is immune to clock skew +between the worker and the DB host. + +The method receives the raised `exception` so subclasses can retry only on +transient errors. + +## Template enforcement + +`_RetryStrategyTemplate` enforces the two cross-cutting limits shared by the +concrete strategies: `max_attempts` and `max_total_delay_seconds`. A concrete +strategy that derives from the template gets both of these caps applied on top +of its own per-attempt delay computation. + +## ExponentialRetry + +`ExponentialRetry` adds two optional knobs on top of the template: jitter and +`max_delay_seconds`. + +## max_total_delay_seconds is a lower bound + +`max_total_delay_seconds` is a lower bound on the horizon, not an exact ceiling. +`elapsed` is measured as `last_attempt_at − first_attempt_at`, and both +timestamps are set equal on the first attempt. Because of that, the budget +always permits roughly one more interval beyond the nominal cap (F2-01). + +Size `max_total_delay_seconds` as "at least this long", not as an exact ceiling. + +## Default strategy + +A subscriber with no explicit `retry_strategy` resolves to: + +```python +ExponentialRetry( + initial_delay_seconds=1.0, + multiplier=2.0, + max_delay_seconds=300.0, + max_attempts=10, + jitter_factor=0.2, +) +``` + +This comes from `_default_retry_strategy()` in `registrator.py`. + +"Delete on first error" is the wrong default for an outbox, so it is not the +default; opt in to that behavior explicitly with `NoRetry()`. diff --git a/architecture/schema.md b/architecture/schema.md new file mode 100644 index 0000000..55c6c10 --- /dev/null +++ b/architecture/schema.md @@ -0,0 +1,76 @@ +# User-owned schema — implementation detail + +User-facing: `docs/operations/alembic.md`. Invariant summary: `CLAUDE.md` § Schema. + +## make_outbox_table + partial indexes + +`make_outbox_table(metadata, table_name="outbox")` returns a `sqlalchemy.Table` on +the user's `MetaData`. The package never creates or migrates the table — that's +Alembic's job — but it declares three partial indexes on the table so that +autogenerate brings them up: + +- `(queue, next_attempt_at) WHERE acquired_token IS NULL` — fetch CTE Branch A + (unleased rows). +- `(queue, acquired_at) WHERE acquired_token IS NOT NULL` — fetch CTE Branch B + (expired-lease reclaim). +- unique `(queue, timer_id) WHERE timer_id IS NOT NULL` — `timer_id` dedup. + +## The lease CHECK constraint + +In addition to the indexes, `make_outbox_table` declares a +`CHECK ((acquired_token IS NULL) = (acquired_at IS NULL))` — the `
_lease_ck` +constraint. It makes a half-set lease unrepresentable: the two lease columns must +either both be set or both be unset. + +## Why the fetch CTE carries partial-index predicates + +The fetch CTE's `OR` is written so that each disjunct explicitly carries its +partial-index predicate as a conjunct. Postgres only uses a partial index when the +query implies the index's `WHERE` clause; the naive form of the query (without the +predicate spelled out per disjunct) falls back to a seq-scan. Both fetch indexes +pay write amplification on every claim. + +## ORDER BY and sort nodes + +The fetch index also satisfies the `ORDER BY next_attempt_at, id`, but only for a +single-queue subscriber. A subscriber serving multiple queues +(`queue = ANY(:queues)`), or the expired-lease branch (which is ordered by +`next_attempt_at` while `_lease_idx` is keyed on `acquired_at`), adds a +`LIMIT`-bounded sort node. Prefer one subscriber per queue when fetch ordering cost +matters — the same segregation pattern as lease TTLs. + +The `ORDER BY` lives on the inner CTE that selects and `LIMIT`s the rows; the outer +`UPDATE … RETURNING *` is unordered, so the order in which rows dispatch within a +single fetch batch is unspecified (F2-09). The ordering governs which rows are +claimed under contention (FIFO selection), not the per-row dispatch sequence — +which is irrelevant with `max_workers > 1` anyway. Don't rely on within-batch FIFO +delivery. + +## No state column + +There is no `state` column. A row is "available" iff `acquired_token IS NULL` or +`acquired_at < now() - lease_ttl_seconds`. Terminal failures `DELETE` by default; +opt in to audit via `dlq_table=make_dlq_table(metadata)`. + +## validate_schema() — opt-in drift detection + +`validate_schema()` is opt-in — call it from `/health` or a startup hook, not from +`broker.start()` — so that migrations can run against the same DB without a loop. + +Beyond the alembic column/index diff it also probes the live partial-index +predicates (alembic ignores `postgresql_where`), catching a drifted or non-partial +`timer_id_uq` that would otherwise break `ON CONFLICT` at publish time (S2). It also +probes `pg_constraint` for the `
_lease_ck` CHECK (alembic has no +check-constraint comparator), catching a missing or drifted lease pairing. + +Because these two probes (predicates + CHECK) catch drift that +`alembic revision --autogenerate` cannot remediate, the raised `RuntimeError` +appends a pointer to +`docs/operations/alembic.md#fixing-drift-autogenerate-cant-see` (the +hand-written-migration recipe) — but only when one of those two probes fired. +Autogenerate-fixable drift (columns, plain indexes, DLQ) gets no pointer. Message +composition lives in `_compose_schema_mismatch_message` (`client.py`), gated on +`has_blind_drift`. + +Alembic is optional (`faststream-outbox[validate]`); without it `validate_schema()` +raises `ImportError`, but every other path works. diff --git a/architecture/subscriber.md b/architecture/subscriber.md new file mode 100644 index 0000000..763bc9c --- /dev/null +++ b/architecture/subscriber.md @@ -0,0 +1,72 @@ +# Two-loop subscriber + lease-token invariant — implementation detail + +User-facing docs do not cover this directly. Invariant summary: `CLAUDE.md` § Subscriber. + +## The two loops + +The subscriber runtime lives in `subscriber/usecase.py`. Per subscriber, two kinds of async task run: + +**1. `_fetch_loop`** — owns a long-lived `AsyncConnection` for the fetch CTE plus a separate raw asyncpg connection for `LISTEN outbox_
`. It runs a single CTE: + +``` +SELECT … FOR UPDATE SKIP LOCKED → UPDATE acquired_token=:uuid, acquired_at=now() RETURNING * +``` + +The `WHERE` clause reclaims both unleased rows and expired leases (`acquired_at < now() - make_interval(secs => :lease_ttl)`) — so there is no separate reaper. A `NOTIFY` short-circuits the idle sleep via an `asyncio.Event`, dropping idle latency from `max_fetch_interval` to ~10ms. `LISTEN` failures log once and fall back to polling. On any DB error the connections close, the loop backs off exponentially (`_BACKOFF_EXP_CAP=30`), and reopens. + +**2. `_worker_loop`** — one per `max_workers`. Each pulls from an `asyncio.Queue(maxsize=fetch_batch_size)`, dispatches via `consume()`, and flushes the terminal state. Each worker owns a long-lived `AsyncConnection` (held across reconnect) and routes terminal writes through `delete_with_lease(conn, …)` / `mark_pending_with_lease(conn, …)`, so a drain of N rows costs O(workers) pool checkouts. Flush exceptions propagate (the outer loop rebuilds the connection); the inflight slot still releases in `finally`. + +The default ack policy is `AckPolicy.NACK_ON_ERROR`; `REJECT_ON_ERROR` and `MANUAL` are allowed. `AckPolicy.ACK_FIRST` is rejected at registration with `ValueError` — it would delete before the handler runs, defeating the outbox contract. `subscriber/factory.py` raises or warns on other footguns (`lease_ttl_seconds <= max_fetch_interval`, `max_deliveries` without retry, etc.). + +`OutboxSubscriber.get_one()` and `__aiter__()` are explicit `NotImplementedError`s — they point operators at `broker.fetch_unprocessed(session=..., queue=...)`. A peek that acquires a lease has surprising `deliveries_count` semantics; lease-free reads belong on `fetch_unprocessed`. + +## Lease bound + +A subscriber can hold up to `fetch_batch_size + max_workers` leases at once, not `fetch_batch_size` (F1-01). The free-slot computation `free = _inflight.maxsize - qsize()` counts only queued rows, so once `max_workers` rows are checked out for processing, the loop can claim another full batch. Leases are bounded and self-expire via TTL, but when sizing `lease_ttl_seconds` and reasoning about cross-replica contention, reason against `fetch_batch_size + max_workers`, not `fetch_batch_size`. + +## Connection budget + +Each subscriber holds `max_workers + 1` SQLAlchemy pool connections steady-state, plus one raw asyncpg connection for `LISTEN`. + +- Size the pool for `Σ subscribers × (max_workers + 1)` or startup blocks on checkout. The asyncpg `LISTEN` connection lives outside the pool, so it does not count toward pool sizing. +- Per process, Postgres `max_connections` must cover `replicas × Σ subscribers × (max_workers + 2)`: the `max_workers + 1` pool connections plus the out-of-pool asyncpg `LISTEN` connection. Undersize it and rolling deploys hit `FATAL: too many connections`. + +## NOTIFY semantics + +`broker.publish` / `publish_batch` emit `SELECT pg_notify('outbox_
', queue)` on the caller's session right after the `INSERT` — except when a future-dated insert or a `timer_id` conflict no-op'd the insert. + +`NOTIFY` is transactional, so atomicity with the row is automatic; rolled-back transactions silently drop it. The future-dated decision is one shared `is_future_dated(activate_in, activate_at, now)` in the stdlib-only leaf `_scheduling.py`, alongside `resolve_next_attempt_client_side` and `validate_activate_args`. `activate_at`'s comparison and the `publish_batch` `next_attempt_at` are worker-clock-relative (unlike `activate_in`'s server-side `make_interval`), so under worker/DB clock skew `NOTIFY` may fire slightly early or late — polling backstops it (F2-04 / F2-05). + +`NOTIFY`s emitted during a fetch-loop reconnect/backoff window are lost (`LISTEN` is not durable); latency degrades to the poll interval until the next tick — a latency, not a correctness, gap (F1-07). + +Channel naming is `outbox_`. Postgres limits identifiers to 63 bytes; `make_outbox_table` raises `ValueError` when the longest derived identifier — an index name like `
_pending_idx` — would exceed it, so over-long table names (~>51 bytes) are rejected at construction, not silently degraded to polling. + +## The lease-token invariant + +Load-bearing. Every terminal write (`delete_with_lease`, `mark_pending_with_lease`) filters on `acquired_token`. If a slow handler's lease expired and a newer fetch reclaimed the row, the slow handler's `DELETE`/`UPDATE` finds `rowcount == 0` and is silently dropped — preventing it from clobbering the new lease holder. Any new fetch or terminal path must preserve this. + +Lease-loss logs at `WARNING` with `extra={"event": "lease_lost", "phase": "terminal"|"retry", "row_id": …, "queue": …, "deliveries_count": …}`. Recurring `event=lease_lost` means `lease_ttl_seconds < handler P99`. + +## lease_ttl sizing + +`lease_ttl_seconds` (default `60.0`, per-subscriber) must exceed handler P99 with margin or healthy handlers race their own expiry. The lease cutoff uses server-side `make_interval(secs => :lease_ttl)`, so it is immune to clock skew. + +Sizing tip: route occasional slow work onto its own subscriber with a tall TTL; keep the fast subscriber's tight. TTL is per-subscriber. + +## deliveries_count vs attempts_count + +`deliveries_count` counts claims, not completed handler runs — the fetch CTE increments it on every claim, including expired-lease reclaims (F2-07). Under lease churn a row can cross `max_deliveries` after fewer than N successful handler invocations, so set `max_deliveries` with margin. + +`attempts_count` (via `_record_attempt`) is the handler-run-scoped counter. + +## Writer-connection autocommit + +`_open_worker_resources` sets the per-worker writer to `isolation_level="AUTOCOMMIT"`. Terminal writes are single statements; an explicit `BEGIN`/`COMMIT` would add two round-trips per row. The `WHERE acquired_token = …` clause enforces the invariant, not the transaction wrapping. + +The fetch connection is not autocommit — it owns `LISTEN`/`NOTIFY` and amortizes `BEGIN`/`COMMIT` across the batch. + +## Shutdown race in dispatch_one + +If `stop()` flips `running=False` between a worker pulling a row from `_inflight` and entering `consume()`, base `SubscriberUsecase.consume()` early-exits without running the handler. `dispatch_one` detects this (`not row.state_set and not self.running` after `consume()` returns without raising) and returns before `assert_state_set → reject() → _safe_flush` would silently `DELETE`. The lease lives until expiry; another replica reclaims the row. No metric fires. Without this guard, busy subscribers leak rows on every rolling deploy. + +See also [`architecture/drain.md`](drain.md) for the drain-phase interaction. diff --git a/planning/.convention-version b/planning/.convention-version new file mode 100644 index 0000000..3eefcb9 --- /dev/null +++ b/planning/.convention-version @@ -0,0 +1 @@ +1.0.0 diff --git a/planning/README.md b/planning/README.md index d2958c3..5f76ba0 100644 --- a/planning/README.md +++ b/planning/README.md @@ -4,12 +4,42 @@ Specs, plans, and change history for `faststream-outbox`. The living truth about *what the system does now* lives in [`architecture/`](../architecture/) at the repo root; this directory records *how it got there*. +## Quick path (start here) + +> The fast lane for making a change. The full reference is in +> [Conventions](#conventions) below — read it only when this isn't enough. + +**1. Choose a lane — first matching rule wins:** + +1. Any of: needs design judgment · new file/module · public-API change · + cross-cutting or multi-file · non-trivial test design → **Full** + (`design.md` + `plan.md`) +2. Purely mechanical: typo · dep bump · linter/formatter/CI tweak · + mechanical rename · single-line config → **Tiny** (no bundle, conventional + commit) +3. Small-but-real, none of the above: ≲30 LOC net · ≤2 files · no new file · + no public-API change · one straightforward test → **Lightweight** + (`change.md`) + +Ambiguous between two? Take the heavier. A `change.md` that outgrows its lane +splits into `design.md` + `plan.md`. + +**2. Create the bundle** (Full / Lightweight only): +`planning/changes/YYYY-MM-DD.NN-/`, where `.NN` is a zero-padded +intra-day counter. Copy the matching template from +[`_templates/`](_templates/). + +**3. Ship in the implementing PR:** hand-edit the affected +`architecture/.md`, fill `outcome:` in +the bundle frontmatter, and run `just check-planning` before pushing. + ## Conventions -> This section is the portable convention — identical across the -> modern-python repos. The generated change listing (`just index`) and the `## Other` pointers below are repo-local. To adopt elsewhere, -> copy this section plus [`_templates/`](_templates/) and point that repo's -> `CLAUDE.md` Workflow + truth home at it. +> This is the portable convention, sourced from the canonical repo +> [`lesnik512/planning-convention`](https://github.com/lesnik512/planning-convention) +> (applied version in [`.convention-version`](.convention-version)). To update +> it, run that repo's `APPLY.md` flow. The generated change index (`just index`) +> and the `## Other` pointers below are repo-local. ### Two axes, never mixed @@ -33,8 +63,8 @@ A change is a folder `changes/YYYY-MM-DD.NN-/`: - `` — kebab-case description, not a story ID. `summary` is written when the change is created (it is the change's -one-liner). The implementing PR then sets `status: shipped` and fills `pr` -and `outcome` **in the branch**, alongside the code and the `architecture/` +one-liner). The implementing PR fills `outcome` +**in the branch**, alongside the code and the `architecture/` promotion — no post-merge bookkeeping, no folder move. ### Three lanes @@ -53,38 +83,43 @@ into `design.md` + `plan.md`. - **`design.md`** — the spec: the *thinking* (why, design, trade-offs, scope). - **`plan.md`** — the plan: the *sequencing* (the executor's task checklist). - **`change.md`** — both, condensed, for the lightweight lane. -- **`decisions/-.md`** — one file per design decision taken - (especially options *rejected*), each with a revisit trigger, so reviews don't - re-litigate them; listed by `just index`. - **`releases/.md`** — per-release user-facing notes. - **`audits/-.md`** — findings from a code/docs/bug-hunt sweep; spawns fix changes. - **`retros/-.md`** — what we learned after a body of work. - **`deferred.md`** — real-but-unscheduled items, each with a revisit trigger. +- **`decisions/-.md`** — one file per design decision taken + (especially options *rejected*), each with a revisit trigger; listed by + `just index`. Templates live in [`_templates/`](_templates/). ### Frontmatter -`design.md` / `change.md`: `status` (draft|approved|shipped|superseded), -`date`, `slug`, `summary` (single line), `supersedes`, `superseded_by`, `pr`, -`outcome`. `plan.md`: `status`, `date`, `slug`, `spec`, `pr`. `decisions/*.md`: -`status` (accepted|superseded), `date`, `slug`, `summary`, `supersedes`, -`superseded_by`, `pr`. Files in `architecture/` carry **no** frontmatter — -living prose, dated by git. +`design.md` / `change.md`: `date`, `slug`, `summary` (single line), `outcome`. +`plan.md`: `date`, `slug`, `spec`. `decisions/*.md`: `status` +(accepted|superseded), `date`, `slug`, `summary`, `supersedes`, `superseded_by`. +Files in `architecture/` carry **no** frontmatter — living prose, dated by git. + +**`outcome`** is filled at ship time: one line, ~1–3 sentences (≤ ~300 chars), +stating the realized result — what shipped and its effect (deviations from the +plan included), written so a future reader grasps the consequence without +opening the diff. It is distinct from `summary`, which is the pre-ship intent +one-liner. ## Index -The listing is **generated**, not maintained — run `just index` to print it: -changes grouped by `status` (In progress / Shipped / Superseded), then -decisions newest-first. The frontmatter in each bundle / decision file is the -single source of truth; there is no committed copy to drift. +The listing is **generated**, not maintained — run `just index` to print it: a +flat, newest-first list of changes, then decisions newest-first. The frontmatter +in each bundle / decision file is the single source of truth; there is no +committed copy to drift. ## Other - **[`architecture/`](../architecture/)** at the repo root — the living - capability truth (relay, timers, dlq, drain, metrics, test broker). This is - the promotion target on every ship. + capability truth (producer, relay, timers, schema, dlq, subscriber, drain, + test-broker, integration, metrics, retry). This is the promotion target on + every ship. - **[decisions/](decisions/)** — design decisions taken (and alternatives rejected), each with a revisit trigger, so reviews don't re-litigate them; indexed by `just index`. diff --git a/planning/_templates/change.md b/planning/_templates/change.md index d3085ed..ab6bc1f 100644 --- a/planning/_templates/change.md +++ b/planning/_templates/change.md @@ -1,12 +1,8 @@ --- -status: draft date: YYYY-MM-DD slug: my-change -summary: One line — shown in the generated index. Fill when creating the change. -supersedes: null -superseded_by: null -pr: null -outcome: null +summary: One line — shown in the generated index. Fill at ship time. +outcome: Realized result — filled at ship time (~1–3 sentences); see README "Frontmatter". --- # Change: One-line capitalized title diff --git a/planning/_templates/decision.md b/planning/_templates/decision.md index 940fb37..e3801b8 100644 --- a/planning/_templates/decision.md +++ b/planning/_templates/decision.md @@ -5,7 +5,6 @@ slug: my-decision summary: One line — shown in `just index`. supersedes: null superseded_by: null -pr: null # PR/commit where the decision was made or recorded --- # One-line capitalized title diff --git a/planning/_templates/design.md b/planning/_templates/design.md index 9dfe115..861ebd2 100644 --- a/planning/_templates/design.md +++ b/planning/_templates/design.md @@ -1,12 +1,8 @@ --- -status: draft date: YYYY-MM-DD slug: my-change -summary: One line — shown in the generated index. Fill when creating the change. -supersedes: null -superseded_by: null -pr: null -outcome: null +summary: One line — shown in the generated index. Fill at ship time. +outcome: Realized result — filled at ship time (~1–3 sentences); see README "Frontmatter". --- # Design: One-line capitalized title diff --git a/planning/_templates/plan.md b/planning/_templates/plan.md index f2b90e8..202ff6a 100644 --- a/planning/_templates/plan.md +++ b/planning/_templates/plan.md @@ -1,9 +1,7 @@ --- -status: draft date: YYYY-MM-DD slug: my-change -spec: my-change -pr: null +spec: design.md --- # — implementation plan diff --git a/planning/_templates/release.md b/planning/_templates/release.md new file mode 100644 index 0000000..7372d7e --- /dev/null +++ b/planning/_templates/release.md @@ -0,0 +1,39 @@ +# modern-di — + + + + + +## Feature + +- **.** What it adds and how to use it. + +## Fix + +- **.** What was broken, now fixed (reference the issue/regression). + +## Internal refactors + +- **.** What changed under the hood, stated as no behavior change. + +## Packaging + +- Metadata / build / dependency changes visible to installers. + +## Why + +Context a reader needs for the headline change. Omit for small releases. + +## Downstream + +What integrations (FastAPI, Litestar, FastStream, Typer, `modern-di-pytest`) +must do — e.g. bump their `modern-di` floor — or "No action needed" when there +is no API change. + +## Internals + +- Coverage / tooling notes (e.g. 100% line coverage across Python 3.10–3.14). diff --git a/planning/changes/2026-06-03.01-all-extra-and-planning-dir/plan.md b/planning/changes/2026-06-03.01-all-extra-and-planning-dir/plan.md index 1ec3afa..305b52b 100644 --- a/planning/changes/2026-06-03.01-all-extra-and-planning-dir/plan.md +++ b/planning/changes/2026-06-03.01-all-extra-and-planning-dir/plan.md @@ -2,7 +2,7 @@ status: shipped date: 2026-06-03 slug: all-extra-and-planning-dir -spec: all-extra-and-planning-dir +spec: design.md pr: "41" --- diff --git a/planning/changes/2026-06-03.02-faststream-0.7-migration/plan.md b/planning/changes/2026-06-03.02-faststream-0.7-migration/plan.md index 431f59c..f15106d 100644 --- a/planning/changes/2026-06-03.02-faststream-0.7-migration/plan.md +++ b/planning/changes/2026-06-03.02-faststream-0.7-migration/plan.md @@ -2,7 +2,7 @@ status: shipped date: 2026-06-03 slug: faststream-0.7-migration -spec: faststream-0.7-migration +spec: design.md pr: "42" --- diff --git a/planning/changes/2026-06-04.01-faststream-0.7.1-testbroker-typing/plan.md b/planning/changes/2026-06-04.01-faststream-0.7.1-testbroker-typing/plan.md index f2a75df..97d29da 100644 --- a/planning/changes/2026-06-04.01-faststream-0.7.1-testbroker-typing/plan.md +++ b/planning/changes/2026-06-04.01-faststream-0.7.1-testbroker-typing/plan.md @@ -2,7 +2,7 @@ status: shipped date: 2026-06-04 slug: faststream-0.7.1-testbroker-typing -spec: faststream-0.7.1-testbroker-typing +spec: design.md pr: "43" --- diff --git a/planning/changes/2026-06-04.02-foreign-broker-relay/plan.md b/planning/changes/2026-06-04.02-foreign-broker-relay/plan.md index 15cb51e..1603bad 100644 --- a/planning/changes/2026-06-04.02-foreign-broker-relay/plan.md +++ b/planning/changes/2026-06-04.02-foreign-broker-relay/plan.md @@ -2,7 +2,7 @@ status: shipped date: 2026-06-04 slug: foreign-broker-relay -spec: foreign-broker-relay +spec: design.md pr: "44" --- diff --git a/planning/changes/2026-06-09.01-mkdocs-github-pages/plan.md b/planning/changes/2026-06-09.01-mkdocs-github-pages/plan.md index 88e1b11..8602a9b 100644 --- a/planning/changes/2026-06-09.01-mkdocs-github-pages/plan.md +++ b/planning/changes/2026-06-09.01-mkdocs-github-pages/plan.md @@ -2,7 +2,7 @@ status: shipped date: 2026-06-09 slug: mkdocs-github-pages -spec: mkdocs-github-pages +spec: design.md pr: "45" --- diff --git a/planning/changes/2026-06-09.02-drain-test-flaky-fetch-observation/plan.md b/planning/changes/2026-06-09.02-drain-test-flaky-fetch-observation/plan.md index 74b2fd5..97884cd 100644 --- a/planning/changes/2026-06-09.02-drain-test-flaky-fetch-observation/plan.md +++ b/planning/changes/2026-06-09.02-drain-test-flaky-fetch-observation/plan.md @@ -2,7 +2,7 @@ status: shipped date: 2026-06-09 slug: drain-test-flaky-fetch-observation -spec: drain-test-flaky-fetch-observation +spec: design.md pr: "48" --- diff --git a/planning/changes/2026-06-10.02-docs-landing-and-comparison/plan.md b/planning/changes/2026-06-10.02-docs-landing-and-comparison/plan.md index 88843ce..24a7f8d 100644 --- a/planning/changes/2026-06-10.02-docs-landing-and-comparison/plan.md +++ b/planning/changes/2026-06-10.02-docs-landing-and-comparison/plan.md @@ -2,7 +2,7 @@ status: shipped date: 2026-06-10 slug: docs-landing-and-comparison -spec: docs-landing-and-comparison +spec: design.md pr: "50" --- diff --git a/planning/changes/2026-06-11.01-operator-pages/plan.md b/planning/changes/2026-06-11.01-operator-pages/plan.md index dde6f32..a37cd67 100644 --- a/planning/changes/2026-06-11.01-operator-pages/plan.md +++ b/planning/changes/2026-06-11.01-operator-pages/plan.md @@ -2,7 +2,7 @@ status: shipped date: 2026-06-11 slug: operator-pages -spec: operator-pages +spec: design.md pr: "53" --- diff --git a/planning/changes/2026-06-11.02-docs-tutorials-and-observability-split/plan.md b/planning/changes/2026-06-11.02-docs-tutorials-and-observability-split/plan.md index ec21f73..fbaea63 100644 --- a/planning/changes/2026-06-11.02-docs-tutorials-and-observability-split/plan.md +++ b/planning/changes/2026-06-11.02-docs-tutorials-and-observability-split/plan.md @@ -2,7 +2,7 @@ status: shipped date: 2026-06-11 slug: docs-tutorials-and-observability-split -spec: docs-tutorials-and-observability-split +spec: design.md pr: "56" --- diff --git a/planning/changes/2026-06-12.01-docs-tutorials/plan.md b/planning/changes/2026-06-12.01-docs-tutorials/plan.md index f975138..738ff39 100644 --- a/planning/changes/2026-06-12.01-docs-tutorials/plan.md +++ b/planning/changes/2026-06-12.01-docs-tutorials/plan.md @@ -2,7 +2,7 @@ status: shipped date: 2026-06-12 slug: docs-tutorials -spec: docs-tutorials +spec: design.md pr: "58" --- diff --git a/planning/changes/2026-06-13.01-portable-planning-convention/plan.md b/planning/changes/2026-06-13.01-portable-planning-convention/plan.md index 9db6277..91f2797 100644 --- a/planning/changes/2026-06-13.01-portable-planning-convention/plan.md +++ b/planning/changes/2026-06-13.01-portable-planning-convention/plan.md @@ -2,7 +2,7 @@ status: shipped date: 2026-06-13 slug: portable-planning-convention -spec: portable-planning-convention +spec: design.md pr: "77" --- diff --git a/planning/changes/2026-06-16.01-actionable-schema-drift-error/plan.md b/planning/changes/2026-06-16.01-actionable-schema-drift-error/plan.md index c2a640b..13cc121 100644 --- a/planning/changes/2026-06-16.01-actionable-schema-drift-error/plan.md +++ b/planning/changes/2026-06-16.01-actionable-schema-drift-error/plan.md @@ -2,7 +2,7 @@ status: shipped date: 2026-06-16 slug: actionable-schema-drift-error -spec: actionable-schema-drift-error +spec: design.md pr: "99" --- diff --git a/planning/changes/2026-06-19.01-messaging-service-patterns-doc/plan.md b/planning/changes/2026-06-19.01-messaging-service-patterns-doc/plan.md index e8c9f49..f1a13d6 100644 --- a/planning/changes/2026-06-19.01-messaging-service-patterns-doc/plan.md +++ b/planning/changes/2026-06-19.01-messaging-service-patterns-doc/plan.md @@ -2,7 +2,7 @@ status: shipped date: 2026-06-19 slug: messaging-service-patterns-doc -spec: messaging-service-patterns-doc +spec: design.md pr: 103 --- diff --git a/planning/changes/2026-06-19.02-docs-diataxis-nav/plan.md b/planning/changes/2026-06-19.02-docs-diataxis-nav/plan.md index 87d4083..8b1c670 100644 --- a/planning/changes/2026-06-19.02-docs-diataxis-nav/plan.md +++ b/planning/changes/2026-06-19.02-docs-diataxis-nav/plan.md @@ -2,7 +2,7 @@ status: shipped date: 2026-06-19 slug: docs-diataxis-nav -spec: docs-diataxis-nav +spec: design.md pr: 104 --- diff --git a/planning/changes/2026-06-20.01-flat-changes-generated-index/plan.md b/planning/changes/2026-06-20.01-flat-changes-generated-index/plan.md index 5c221bd..06c0fc5 100644 --- a/planning/changes/2026-06-20.01-flat-changes-generated-index/plan.md +++ b/planning/changes/2026-06-20.01-flat-changes-generated-index/plan.md @@ -2,7 +2,7 @@ status: shipped date: 2026-06-20 slug: flat-changes-generated-index -spec: flat-changes-generated-index +spec: design.md pr: 105 --- diff --git a/planning/changes/2026-06-23.01-client-rules-kernel/plan.md b/planning/changes/2026-06-23.01-client-rules-kernel/plan.md index 25f8787..3bd1374 100644 --- a/planning/changes/2026-06-23.01-client-rules-kernel/plan.md +++ b/planning/changes/2026-06-23.01-client-rules-kernel/plan.md @@ -2,7 +2,7 @@ status: shipped date: 2026-06-23 slug: client-rules-kernel -spec: client-rules-kernel +spec: design.md pr: 109 --- diff --git a/planning/index.py b/planning/index.py index d38dffd..ec6c2dc 100644 --- a/planning/index.py +++ b/planning/index.py @@ -1,25 +1,28 @@ -# ruff: noqa: INP001 # planning/ is not a Python package; this is a standalone script +# ruff: noqa: INP001, D212 # planning/ is not a Python package; D212/D213 conflict differs from faststream-outbox """ Generate the planning index from frontmatter. -Run via ``just index``. Globs ``planning/changes/*/`` (each bundle's ``design.md``, -falling back to ``change.md``) and ``planning/decisions/*.md``, reads their -frontmatter, and prints a Markdown listing to stdout — changes grouped by lifecycle -status, then decisions newest-first. Never writes a file: the listing is a query over -the files, not a committed artifact. +Run via ``just index``. Globs ``planning/changes/*/`` (each bundle's +``design.md``, falling back to ``change.md``) and ``planning/decisions/*.md``, +reads their frontmatter, and prints a Markdown listing to stdout — changes +then decisions, newest-first. Never writes a file: +the listing is a query over the files, not a committed artifact. """ import pathlib +import re import sys CHANGES_DIR = pathlib.Path(__file__).parent / "changes" DECISIONS_DIR = pathlib.Path(__file__).parent / "decisions" -GROUPS: tuple[tuple[str, tuple[str, ...]], ...] = ( - ("In progress", ("draft", "approved")), - ("Shipped", ("shipped",)), - ("Superseded", ("superseded",)), -) +VALID_DECISION_STATUS = {"accepted", "superseded"} +DATE_RE = re.compile(r"^\d{4}-\d{2}-\d{2}$") +BUNDLE_RE = re.compile(r"^\d{4}-\d{2}-\d{2}\.\d{2}-(?P.+)$") +ALLOWED_BUNDLE_FILES = {"design.md", "plan.md", "change.md"} +SPEC_REQUIRED = ("date", "slug", "summary", "outcome") +PLAN_REQUIRED = ("date", "slug", "spec") +DECISION_REQUIRED = ("status", "date", "slug", "summary") def parse_frontmatter(text: str) -> dict[str, str]: @@ -31,6 +34,8 @@ def parse_frontmatter(text: str) -> dict[str, str]: for line in lines[1:]: if line.strip() == "---": break + if line[:1] in (" ", "\t"): + continue key, sep, value = line.partition(": ") if not sep: continue @@ -73,13 +78,12 @@ def load_decisions() -> list[dict[str, str]]: def format_row(bundle: dict[str, str]) -> str: - """Render one bundle or decision as a Markdown list item.""" + """Render one bundle as a Markdown list item.""" slug = bundle.get("slug", "?") path = bundle.get("path", "") - pr = bundle.get("pr") or "—" date = bundle.get("date", "") summary = bundle.get("summary") or "(no summary)" - line = f"- **[{slug}]({path})** (#{pr}, {date}) — {summary}" + line = f"- **[{slug}]({path})** ({date}) — {summary}" if bundle.get("supersedes"): line += f" _(supersedes {bundle['supersedes']})_" if bundle.get("superseded_by"): @@ -88,26 +92,111 @@ def format_row(bundle: dict[str, str]) -> str: def render(bundles: list[dict[str, str]], decisions: list[dict[str, str]]) -> str: - """Render the full Markdown listing: changes by status, then decisions.""" + """Render the full Markdown listing: changes then decisions, newest-first.""" out = ["# Planning index", "", "_Generated by `just index` — do not edit._", "", "## Changes", ""] - for title, statuses in GROUPS: - out += [f"### {title}", ""] - rows = sorted( - (b for b in bundles if b.get("status") in statuses), - key=lambda b: b.get("name", ""), - reverse=True, - ) - out += [format_row(b) for b in rows] if rows else ["_None._"] - out.append("") - out += ["## Decisions", ""] + change_rows = sorted(bundles, key=lambda b: b.get("name", ""), reverse=True) + out += [format_row(b) for b in change_rows] if change_rows else ["_None._"] + out += ["", "## Decisions", ""] decision_rows = sorted(decisions, key=lambda d: d.get("name", ""), reverse=True) out += [format_row(d) for d in decision_rows] if decision_rows else ["_None._"] out.append("") return "\n".join(out).rstrip() + "\n" +def _require(fields: dict[str, str], keys: tuple[str, ...], rel: str, violations: list[str]) -> None: + """Append a violation for each required key that is absent or empty.""" + violations.extend(f"{rel}: missing or empty frontmatter key '{key}'" for key in keys if not fields.get(key)) + + +def _check_common(fields: dict[str, str], dir_slug: str | None, rel: str, violations: list[str]) -> None: + """Validate date/slug fields shared by every artifact type.""" + date = fields.get("date", "") + if date and not DATE_RE.match(date): + violations.append(f"{rel}: date '{date}' is not YYYY-MM-DD") + slug = fields.get("slug", "") + if dir_slug and slug and slug != dir_slug: + violations.append(f"{rel}: slug '{slug}' does not match directory slug '{dir_slug}'") + + +def _check_spec_file(path: pathlib.Path, rel: str, dir_slug: str | None, violations: list[str]) -> None: + """Validate a design.md / change.md spec file.""" + fields = parse_frontmatter(path.read_text(encoding="utf-8")) + _require(fields, SPEC_REQUIRED, rel, violations) + _check_common(fields, dir_slug, rel, violations) + + +def _check_plan_file( + path: pathlib.Path, bundle: pathlib.Path, rel: str, dir_slug: str | None, violations: list[str] +) -> None: + """Validate a plan.md file, including that its spec: link resolves.""" + fields = parse_frontmatter(path.read_text(encoding="utf-8")) + _require(fields, PLAN_REQUIRED, rel, violations) + _check_common(fields, dir_slug, rel, violations) + spec = fields.get("spec", "") + if spec and not (bundle / spec).resolve().exists(): + violations.append(f"{rel}: spec link '{spec}' does not resolve to a file") + + +def _check_bundle(bundle: pathlib.Path, violations: list[str]) -> None: + """Validate one change bundle directory.""" + rel = f"changes/{bundle.name}" + match = BUNDLE_RE.match(bundle.name) + dir_slug = match.group("slug") if match else None + if match is None: + violations.append(f"{rel}: directory name is not 'YYYY-MM-DD.NN-slug'") + violations.extend( + f"{rel}/{child.name}: unexpected file in bundle (allowed: {', '.join(sorted(ALLOWED_BUNDLE_FILES))})" + for child in sorted(bundle.iterdir()) + if child.name not in ALLOWED_BUNDLE_FILES + ) + design = bundle / "design.md" + change = bundle / "change.md" + plan = bundle / "plan.md" + if not design.exists() and not change.exists(): + violations.append(f"{rel}: bundle has neither design.md nor change.md") + for spec_file in (design, change): + if spec_file.exists(): + _check_spec_file(spec_file, f"{rel}/{spec_file.name}", dir_slug, violations) + if plan.exists(): + _check_plan_file(plan, bundle, f"{rel}/plan.md", dir_slug, violations) + + +def _check_decision(path: pathlib.Path, violations: list[str]) -> None: + """Validate one decision file.""" + rel = f"decisions/{path.name}" + fields = parse_frontmatter(path.read_text(encoding="utf-8")) + _require(fields, DECISION_REQUIRED, rel, violations) + _check_common(fields, None, rel, violations) + status = fields.get("status", "") + if status and status not in VALID_DECISION_STATUS: + violations.append(f"{rel}: invalid status '{status}' (allowed: {', '.join(sorted(VALID_DECISION_STATUS))})") + + +def check() -> list[str]: + """Validate every bundle and decision; return the list of violation strings.""" + violations: list[str] = [] + for bundle in sorted(CHANGES_DIR.iterdir()): + if bundle.is_dir(): + _check_bundle(bundle, violations) + if DECISIONS_DIR.is_dir(): + for path in sorted(DECISIONS_DIR.glob("*.md")): + if path.name == "README.md" or path.name.startswith("_"): + continue + _check_decision(path, violations) + return violations + + def main() -> int: - """Print the listing to stdout.""" + """Print the listing to stdout, or validate bundles with --check.""" + if "--check" in sys.argv[1:]: + violations = check() + if violations: + sys.stderr.write(f"planning: {len(violations)} violation(s)\n") + for violation in violations: + sys.stderr.write(f" - {violation}\n") + return 1 + sys.stdout.write("planning: OK\n") + return 0 sys.stdout.write(render(load_bundles(), load_decisions())) return 0 diff --git a/pyproject.toml b/pyproject.toml index 05224e1..24ad9d8 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -93,6 +93,10 @@ max-args = 15 "S101", # allow asserts "PLR2004", # allow magic values ] +# planning/index.py is vendored verbatim from lesnik512/planning-convention and +# linted in that repo's CI; its file-level noqa is tuned for the canonical D212 +# docstring choice, which this repo inverts (D213), so tolerate the unused noqa. +"planning/index.py" = ["RUF100"] [tool.pytest.ini_options] addopts = "--cov=. --cov-report term-missing --cov-fail-under=100" From 21a9574dae70d8f0deef1f604b313bb0bb2df6e1 Mon Sep 17 00:00:00 2001 From: Artur Shiriev Date: Thu, 25 Jun 2026 18:53:35 +0300 Subject: [PATCH 2/3] Slim planning frontmatter to match updated convention The canonical convention now derives date/slug from names and drops spec/outcome/pr/status. Re-copy canonical index.py + templates and migrate bundles accordingly (supersedes the earlier spec-link migration): - design.md/change.md: summary only (date/slug/outcome + legacy pr/status/supersedes/superseded_by/spec stripped) - plan.md: no frontmatter (16 files) - decisions: status/summary (+ supersedes/superseded_by); date/slug/pr stripped - README frontmatter prose updated to the lean form just lint-ci + check-planning green. --- planning/README.md | 33 +++++---- planning/_templates/change.md | 5 +- planning/_templates/decision.md | 2 - planning/_templates/design.md | 5 +- planning/_templates/plan.md | 10 +-- .../design.md | 7 -- .../plan.md | 8 --- .../design.md | 7 -- .../plan.md | 8 --- .../design.md | 7 -- .../plan.md | 8 --- .../design.md | 7 -- .../plan.md | 8 --- .../design.md | 7 -- .../2026-06-09.01-mkdocs-github-pages/plan.md | 8 --- .../design.md | 7 -- .../plan.md | 8 --- .../design.md | 7 -- .../design.md | 7 -- .../plan.md | 8 --- .../2026-06-11.01-operator-pages/design.md | 7 -- .../2026-06-11.01-operator-pages/plan.md | 8 --- .../design.md | 7 -- .../plan.md | 8 --- .../2026-06-12.01-docs-tutorials/design.md | 7 -- .../2026-06-12.01-docs-tutorials/plan.md | 8 --- .../design.md | 7 -- .../plan.md | 8 --- .../design.md | 7 -- .../plan.md | 8 --- .../design.md | 7 -- .../plan.md | 8 --- .../2026-06-19.02-docs-diataxis-nav/design.md | 7 -- .../2026-06-19.02-docs-diataxis-nav/plan.md | 8 --- .../design.md | 7 -- .../plan.md | 8 --- .../design.md | 16 ----- .../2026-06-23.01-client-rules-kernel/plan.md | 8 --- .../change.md | 12 ---- .../change.md | 15 ---- ...026-06-23-metrics-recorders-not-unified.md | 3 - planning/index.py | 71 ++++++++----------- 42 files changed, 49 insertions(+), 363 deletions(-) diff --git a/planning/README.md b/planning/README.md index 5f76ba0..34d24bd 100644 --- a/planning/README.md +++ b/planning/README.md @@ -30,8 +30,8 @@ intra-day counter. Copy the matching template from [`_templates/`](_templates/). **3. Ship in the implementing PR:** hand-edit the affected -`architecture/.md`, fill `outcome:` in -the bundle frontmatter, and run `just check-planning` before pushing. +`architecture/.md`, finalize the bundle's `summary:` to the +realized result, and run `just check-planning` before pushing. ## Conventions @@ -62,10 +62,11 @@ A change is a folder `changes/YYYY-MM-DD.NN-/`: (`.01`, `.02`, …) that breaks same-date ties so the timeline sorts stably. - `` — kebab-case description, not a story ID. -`summary` is written when the change is created (it is the change's -one-liner). The implementing PR fills `outcome` -**in the branch**, alongside the code and the `architecture/` -promotion — no post-merge bookkeeping, no folder move. +`summary` is written when the change is created (the intent one-liner) and +**finalized at ship** to state the realized result — set in the implementing +PR, alongside the code and the `architecture/` promotion. No post-merge +bookkeeping, no folder move. `date` and `slug` are never written — they are +read from the bundle's directory name. ### Three lanes @@ -96,16 +97,18 @@ Templates live in [`_templates/`](_templates/). ### Frontmatter -`design.md` / `change.md`: `date`, `slug`, `summary` (single line), `outcome`. -`plan.md`: `date`, `slug`, `spec`. `decisions/*.md`: `status` -(accepted|superseded), `date`, `slug`, `summary`, `supersedes`, `superseded_by`. -Files in `architecture/` carry **no** frontmatter — living prose, dated by git. +`date` and `slug` are **derived from the directory / file name** — never +repeated in frontmatter. So: -**`outcome`** is filled at ship time: one line, ~1–3 sentences (≤ ~300 chars), -stating the realized result — what shipped and its effect (deviations from the -plan included), written so a future reader grasps the consequence without -opening the diff. It is distinct from `summary`, which is the pre-ship intent -one-liner. +- `design.md` / `change.md`: `summary` (single line) only. +- `plan.md`: **no frontmatter** — its identity is the bundle directory. +- `decisions/*.md`: `status` (accepted|superseded), `summary`, and optional + `supersedes` / `superseded_by`. +- Files in `architecture/` carry **no** frontmatter — living prose, dated by git. + +**`summary`** is one line: written at creation as the intent, then **finalized +at ship** to state the realized result — what shipped and its effect. It is the +only field the index renders. ## Index diff --git a/planning/_templates/change.md b/planning/_templates/change.md index ab6bc1f..d4c8962 100644 --- a/planning/_templates/change.md +++ b/planning/_templates/change.md @@ -1,8 +1,5 @@ --- -date: YYYY-MM-DD -slug: my-change -summary: One line — shown in the generated index. Fill at ship time. -outcome: Realized result — filled at ship time (~1–3 sentences); see README "Frontmatter". +summary: One line — shown in the generated index. Written at creation; finalize at ship to state the realized result. --- # Change: One-line capitalized title diff --git a/planning/_templates/decision.md b/planning/_templates/decision.md index e3801b8..45ccaf0 100644 --- a/planning/_templates/decision.md +++ b/planning/_templates/decision.md @@ -1,7 +1,5 @@ --- status: accepted # accepted | superseded -date: YYYY-MM-DD -slug: my-decision summary: One line — shown in `just index`. supersedes: null superseded_by: null diff --git a/planning/_templates/design.md b/planning/_templates/design.md index 861ebd2..d63e22d 100644 --- a/planning/_templates/design.md +++ b/planning/_templates/design.md @@ -1,8 +1,5 @@ --- -date: YYYY-MM-DD -slug: my-change -summary: One line — shown in the generated index. Fill at ship time. -outcome: Realized result — filled at ship time (~1–3 sentences); see README "Frontmatter". +summary: One line — shown in the generated index. Written at creation; finalize at ship to state the realized result. --- # Design: One-line capitalized title diff --git a/planning/_templates/plan.md b/planning/_templates/plan.md index 202ff6a..132d720 100644 --- a/planning/_templates/plan.md +++ b/planning/_templates/plan.md @@ -1,9 +1,3 @@ ---- -date: YYYY-MM-DD -slug: my-change -spec: design.md ---- - # — implementation plan > **For agentic workers:** REQUIRED SUB-SKILL: Use @@ -44,9 +38,7 @@ in the spec. ```bash git add path/to/file.py - git commit -m ": - - Co-Authored-By: Claude Opus 4.7 (1M context) " + git commit -m ": " ``` --- diff --git a/planning/changes/2026-06-03.01-all-extra-and-planning-dir/design.md b/planning/changes/2026-06-03.01-all-extra-and-planning-dir/design.md index 07e1435..a082d11 100644 --- a/planning/changes/2026-06-03.01-all-extra-and-planning-dir/design.md +++ b/planning/changes/2026-06-03.01-all-extra-and-planning-dir/design.md @@ -1,12 +1,5 @@ --- -status: shipped -date: 2026-06-03 -slug: all-extra-and-planning-dir summary: Add faststream-outbox[all] aggregate extra; bootstrap the planning/ directory itself. -supersedes: null -superseded_by: null -pr: "41" -outcome: merged 2026-06-03 as #41 --- # Design: `all` aggregate extra + `planning/` workflow directory diff --git a/planning/changes/2026-06-03.01-all-extra-and-planning-dir/plan.md b/planning/changes/2026-06-03.01-all-extra-and-planning-dir/plan.md index 305b52b..5d81ed6 100644 --- a/planning/changes/2026-06-03.01-all-extra-and-planning-dir/plan.md +++ b/planning/changes/2026-06-03.01-all-extra-and-planning-dir/plan.md @@ -1,11 +1,3 @@ ---- -status: shipped -date: 2026-06-03 -slug: all-extra-and-planning-dir -spec: design.md -pr: "41" ---- - # `all` extra and `planning/` workflow dir — Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. diff --git a/planning/changes/2026-06-03.02-faststream-0.7-migration/design.md b/planning/changes/2026-06-03.02-faststream-0.7-migration/design.md index e40bf79..b4c056a 100644 --- a/planning/changes/2026-06-03.02-faststream-0.7-migration/design.md +++ b/planning/changes/2026-06-03.02-faststream-0.7-migration/design.md @@ -1,12 +1,5 @@ --- -status: shipped -date: 2026-06-03 -slug: faststream-0.7-migration summary: Migrate to faststream>=0.7,<0.8; fix mechanical break points; drop per-call middlewares= kwarg. -supersedes: null -superseded_by: null -pr: "42" -outcome: merged 2026-06-03 as #42 --- # Design: FastStream 0.7 migration diff --git a/planning/changes/2026-06-03.02-faststream-0.7-migration/plan.md b/planning/changes/2026-06-03.02-faststream-0.7-migration/plan.md index f15106d..448b1c5 100644 --- a/planning/changes/2026-06-03.02-faststream-0.7-migration/plan.md +++ b/planning/changes/2026-06-03.02-faststream-0.7-migration/plan.md @@ -1,11 +1,3 @@ ---- -status: shipped -date: 2026-06-03 -slug: faststream-0.7-migration -spec: design.md -pr: "42" ---- - # FastStream 0.7 Migration — Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. diff --git a/planning/changes/2026-06-04.01-faststream-0.7.1-testbroker-typing/design.md b/planning/changes/2026-06-04.01-faststream-0.7.1-testbroker-typing/design.md index 4680710..c14e66f 100644 --- a/planning/changes/2026-06-04.01-faststream-0.7.1-testbroker-typing/design.md +++ b/planning/changes/2026-06-04.01-faststream-0.7.1-testbroker-typing/design.md @@ -1,12 +1,5 @@ --- -status: shipped -date: 2026-06-04 -slug: faststream-0.7.1-testbroker-typing summary: Adopt FastStream 0.7.1 TestBroker[Broker, EnterType] typing fix; drop two ty:ignore directives. -supersedes: null -superseded_by: null -pr: "43" -outcome: merged 2026-06-04 as #43 --- # FastStream 0.7.1 TestBroker typing alignment — design diff --git a/planning/changes/2026-06-04.01-faststream-0.7.1-testbroker-typing/plan.md b/planning/changes/2026-06-04.01-faststream-0.7.1-testbroker-typing/plan.md index 97d29da..a4622a0 100644 --- a/planning/changes/2026-06-04.01-faststream-0.7.1-testbroker-typing/plan.md +++ b/planning/changes/2026-06-04.01-faststream-0.7.1-testbroker-typing/plan.md @@ -1,11 +1,3 @@ ---- -status: shipped -date: 2026-06-04 -slug: faststream-0.7.1-testbroker-typing -spec: design.md -pr: "43" ---- - # FastStream 0.7.1 TestBroker typing alignment — implementation plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. diff --git a/planning/changes/2026-06-04.02-foreign-broker-relay/design.md b/planning/changes/2026-06-04.02-foreign-broker-relay/design.md index d24edcc..6a74ad0 100644 --- a/planning/changes/2026-06-04.02-foreign-broker-relay/design.md +++ b/planning/changes/2026-06-04.02-foreign-broker-relay/design.md @@ -1,12 +1,5 @@ --- -status: shipped -date: 2026-06-04 -slug: foreign-broker-relay summary: OutboxSubscriber officially supports the FastStream-native decorator relay to Kafka/Rabbit/NATS/Redis with three guardrails. -supersedes: null -superseded_by: null -pr: "44" -outcome: merged 2026-06-05 as #44 --- # Foreign-broker relay from `OutboxSubscriber` — design diff --git a/planning/changes/2026-06-04.02-foreign-broker-relay/plan.md b/planning/changes/2026-06-04.02-foreign-broker-relay/plan.md index 1603bad..a50fcd1 100644 --- a/planning/changes/2026-06-04.02-foreign-broker-relay/plan.md +++ b/planning/changes/2026-06-04.02-foreign-broker-relay/plan.md @@ -1,11 +1,3 @@ ---- -status: shipped -date: 2026-06-04 -slug: foreign-broker-relay -spec: design.md -pr: "44" ---- - # Foreign-broker relay — implementation plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. diff --git a/planning/changes/2026-06-09.01-mkdocs-github-pages/design.md b/planning/changes/2026-06-09.01-mkdocs-github-pages/design.md index 0ea6481..6e72529 100644 --- a/planning/changes/2026-06-09.01-mkdocs-github-pages/design.md +++ b/planning/changes/2026-06-09.01-mkdocs-github-pages/design.md @@ -1,12 +1,5 @@ --- -status: shipped -date: 2026-06-09 -slug: mkdocs-github-pages summary: Docs hosting moves from Read the Docs to GitHub Pages on faststream-outbox.modern-python.org. -supersedes: null -superseded_by: null -pr: "45" -outcome: merged 2026-06-09 as #45 --- # Design: Migrate docs hosting from Read the Docs to GitHub Pages diff --git a/planning/changes/2026-06-09.01-mkdocs-github-pages/plan.md b/planning/changes/2026-06-09.01-mkdocs-github-pages/plan.md index 8602a9b..7ebd128 100644 --- a/planning/changes/2026-06-09.01-mkdocs-github-pages/plan.md +++ b/planning/changes/2026-06-09.01-mkdocs-github-pages/plan.md @@ -1,11 +1,3 @@ ---- -status: shipped -date: 2026-06-09 -slug: mkdocs-github-pages -spec: design.md -pr: "45" ---- - # Migrate Docs Hosting from Read the Docs to GitHub Pages — Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. diff --git a/planning/changes/2026-06-09.02-drain-test-flaky-fetch-observation/design.md b/planning/changes/2026-06-09.02-drain-test-flaky-fetch-observation/design.md index 40b735c..2d9f29f 100644 --- a/planning/changes/2026-06-09.02-drain-test-flaky-fetch-observation/design.md +++ b/planning/changes/2026-06-09.02-drain-test-flaky-fetch-observation/design.md @@ -1,12 +1,5 @@ --- -status: shipped -date: 2026-06-09 -slug: drain-test-flaky-fetch-observation summary: Drain test waits via the fetched recorder instead of an SQL poll, killing a 3.14 coverage flake. -supersedes: null -superseded_by: null -pr: "48" -outcome: merged 2026-06-10 as #48 --- # Drain test: replace SQL-poll flake with recorder observation diff --git a/planning/changes/2026-06-09.02-drain-test-flaky-fetch-observation/plan.md b/planning/changes/2026-06-09.02-drain-test-flaky-fetch-observation/plan.md index 97884cd..981a79e 100644 --- a/planning/changes/2026-06-09.02-drain-test-flaky-fetch-observation/plan.md +++ b/planning/changes/2026-06-09.02-drain-test-flaky-fetch-observation/plan.md @@ -1,11 +1,3 @@ ---- -status: shipped -date: 2026-06-09 -slug: drain-test-flaky-fetch-observation -spec: design.md -pr: "48" ---- - # Drain Test Flake Fix Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. diff --git a/planning/changes/2026-06-10.01-planning-conventions/design.md b/planning/changes/2026-06-10.01-planning-conventions/design.md index a006675..d683b05 100644 --- a/planning/changes/2026-06-10.01-planning-conventions/design.md +++ b/planning/changes/2026-06-10.01-planning-conventions/design.md @@ -1,12 +1,5 @@ --- -status: superseded -date: 2026-06-10 -slug: planning-conventions summary: Spec/plan boundary, active/archived/_templates layout, frontmatter, migration of the existing pairs. Superseded by portable-planning-convention. -supersedes: null -superseded_by: portable-planning-convention -pr: "49" -outcome: merged 2026-06-10 as #49 --- # Design: Rework planning conventions + migrate existing artifacts diff --git a/planning/changes/2026-06-10.02-docs-landing-and-comparison/design.md b/planning/changes/2026-06-10.02-docs-landing-and-comparison/design.md index 10bc45d..d7f452e 100644 --- a/planning/changes/2026-06-10.02-docs-landing-and-comparison/design.md +++ b/planning/changes/2026-06-10.02-docs-landing-and-comparison/design.md @@ -1,12 +1,5 @@ --- -status: shipped -date: 2026-06-10 -slug: docs-landing-and-comparison summary: Docs landing rewrite, four-section nav reshape, new Comparison page. -supersedes: null -superseded_by: null -pr: "50" -outcome: merged 2026-06-10 as #50 --- # Design: Rework the docs landing + nav, add a comparison page diff --git a/planning/changes/2026-06-10.02-docs-landing-and-comparison/plan.md b/planning/changes/2026-06-10.02-docs-landing-and-comparison/plan.md index 24a7f8d..b26d1cf 100644 --- a/planning/changes/2026-06-10.02-docs-landing-and-comparison/plan.md +++ b/planning/changes/2026-06-10.02-docs-landing-and-comparison/plan.md @@ -1,11 +1,3 @@ ---- -status: shipped -date: 2026-06-10 -slug: docs-landing-and-comparison -spec: design.md -pr: "50" ---- - # docs-landing-and-comparison — implementation plan > **For agentic workers:** REQUIRED SUB-SKILL: Use diff --git a/planning/changes/2026-06-11.01-operator-pages/design.md b/planning/changes/2026-06-11.01-operator-pages/design.md index 327b147..843686b 100644 --- a/planning/changes/2026-06-11.01-operator-pages/design.md +++ b/planning/changes/2026-06-11.01-operator-pages/design.md @@ -1,12 +1,5 @@ --- -status: shipped -date: 2026-06-11 -slug: operator-pages summary: docs/operations/: Production checklist, Troubleshooting playbook, Alembic migrations. The B follow-on from #50. -supersedes: null -superseded_by: null -pr: "53" -outcome: merged 2026-06-11 as #53 --- # Design: Operator pages — Production checklist, Troubleshooting, Alembic migrations diff --git a/planning/changes/2026-06-11.01-operator-pages/plan.md b/planning/changes/2026-06-11.01-operator-pages/plan.md index a37cd67..e0e1e6d 100644 --- a/planning/changes/2026-06-11.01-operator-pages/plan.md +++ b/planning/changes/2026-06-11.01-operator-pages/plan.md @@ -1,11 +1,3 @@ ---- -status: shipped -date: 2026-06-11 -slug: operator-pages -spec: design.md -pr: "53" ---- - # operator-pages — implementation plan > **For agentic workers:** REQUIRED SUB-SKILL: Use diff --git a/planning/changes/2026-06-11.02-docs-tutorials-and-observability-split/design.md b/planning/changes/2026-06-11.02-docs-tutorials-and-observability-split/design.md index 3ac234f..0b48a12 100644 --- a/planning/changes/2026-06-11.02-docs-tutorials-and-observability-split/design.md +++ b/planning/changes/2026-06-11.02-docs-tutorials-and-observability-split/design.md @@ -1,12 +1,5 @@ --- -status: shipped -date: 2026-06-11 -slug: docs-tutorials-and-observability-split summary: Three-way split of usage/observability.md into Reference + How-to + Explanation; tutorials deferred to #58. -supersedes: null -superseded_by: null -pr: "56" -outcome: merged 2026-06-12 as #56 (observability split only; tutorials §1 §2 deferred to a follow-on spec) --- # Design: Add two tutorials and split observability.md diff --git a/planning/changes/2026-06-11.02-docs-tutorials-and-observability-split/plan.md b/planning/changes/2026-06-11.02-docs-tutorials-and-observability-split/plan.md index fbaea63..7cfdd3e 100644 --- a/planning/changes/2026-06-11.02-docs-tutorials-and-observability-split/plan.md +++ b/planning/changes/2026-06-11.02-docs-tutorials-and-observability-split/plan.md @@ -1,11 +1,3 @@ ---- -status: shipped -date: 2026-06-11 -slug: docs-tutorials-and-observability-split -spec: design.md -pr: "56" ---- - # docs-tutorials-and-observability-split — implementation plan > **For agentic workers:** REQUIRED SUB-SKILL: Use diff --git a/planning/changes/2026-06-12.01-docs-tutorials/design.md b/planning/changes/2026-06-12.01-docs-tutorials/design.md index 84f4bad..7b5cd39 100644 --- a/planning/changes/2026-06-12.01-docs-tutorials/design.md +++ b/planning/changes/2026-06-12.01-docs-tutorials/design.md @@ -1,12 +1,5 @@ --- -status: shipped -date: 2026-06-12 -slug: docs-tutorials summary: The two tutorials deferred from #56: Your first outbox app and Add a Kafka relay. Kill-Kafka step folded into an at-least-once callout after aiokafka absorbed the outage. -supersedes: null -superseded_by: null -pr: "58" -outcome: merged 2026-06-12 as #58 (both tutorials, kill-Kafka step replaced by an at-least-once contract callout per spec authorization) --- # Design: Two new tutorials (Diátaxis F-min, part 2) diff --git a/planning/changes/2026-06-12.01-docs-tutorials/plan.md b/planning/changes/2026-06-12.01-docs-tutorials/plan.md index 738ff39..a88d1af 100644 --- a/planning/changes/2026-06-12.01-docs-tutorials/plan.md +++ b/planning/changes/2026-06-12.01-docs-tutorials/plan.md @@ -1,11 +1,3 @@ ---- -status: shipped -date: 2026-06-12 -slug: docs-tutorials -spec: design.md -pr: "58" ---- - # docs-tutorials — implementation plan > **For agentic workers:** REQUIRED SUB-SKILL: Use diff --git a/planning/changes/2026-06-13.01-portable-planning-convention/design.md b/planning/changes/2026-06-13.01-portable-planning-convention/design.md index f2ef6d4..99188b1 100644 --- a/planning/changes/2026-06-13.01-portable-planning-convention/design.md +++ b/planning/changes/2026-06-13.01-portable-planning-convention/design.md @@ -1,12 +1,5 @@ --- -status: shipped -date: 2026-06-13 -slug: portable-planning-convention summary: Two-axis OpenSpec-shaped convention: architecture/ truth + changes/ folder bundles, .NN intra-day tiebreak, three lanes, dedicated audits/+retros/, portable README. Supersedes planning-conventions. -supersedes: planning-conventions -superseded_by: null -pr: "77" -outcome: merged 2026-06-13 as #77 --- # Design: Portable OpenSpec-shaped planning convention diff --git a/planning/changes/2026-06-13.01-portable-planning-convention/plan.md b/planning/changes/2026-06-13.01-portable-planning-convention/plan.md index 91f2797..a14801c 100644 --- a/planning/changes/2026-06-13.01-portable-planning-convention/plan.md +++ b/planning/changes/2026-06-13.01-portable-planning-convention/plan.md @@ -1,11 +1,3 @@ ---- -status: shipped -date: 2026-06-13 -slug: portable-planning-convention -spec: design.md -pr: "77" ---- - # Portable planning-convention — implementation plan > **For agentic workers:** REQUIRED SUB-SKILL: Use diff --git a/planning/changes/2026-06-16.01-actionable-schema-drift-error/design.md b/planning/changes/2026-06-16.01-actionable-schema-drift-error/design.md index 571e4bd..9f64a4e 100644 --- a/planning/changes/2026-06-16.01-actionable-schema-drift-error/design.md +++ b/planning/changes/2026-06-16.01-actionable-schema-drift-error/design.md @@ -1,12 +1,5 @@ --- -status: shipped -date: 2026-06-16 -slug: actionable-schema-drift-error summary: validate_schema() appends a hand-written-migration pointer to its RuntimeError for Alembic-blind drift (the outbox_lease_ck CHECK and partial-index predicates autogenerate cannot remediate). -supersedes: null -superseded_by: null -pr: "99" -outcome: shipped in #99 --- # Design: Actionable error for Alembic-blind schema drift diff --git a/planning/changes/2026-06-16.01-actionable-schema-drift-error/plan.md b/planning/changes/2026-06-16.01-actionable-schema-drift-error/plan.md index 13cc121..19e156a 100644 --- a/planning/changes/2026-06-16.01-actionable-schema-drift-error/plan.md +++ b/planning/changes/2026-06-16.01-actionable-schema-drift-error/plan.md @@ -1,11 +1,3 @@ ---- -status: shipped -date: 2026-06-16 -slug: actionable-schema-drift-error -spec: design.md -pr: "99" ---- - # actionable-schema-drift-error — implementation plan > **For agentic workers:** REQUIRED SUB-SKILL: Use diff --git a/planning/changes/2026-06-19.01-messaging-service-patterns-doc/design.md b/planning/changes/2026-06-19.01-messaging-service-patterns-doc/design.md index 3e0c95e..44b4a89 100644 --- a/planning/changes/2026-06-19.01-messaging-service-patterns-doc/design.md +++ b/planning/changes/2026-06-19.01-messaging-service-patterns-doc/design.md @@ -1,12 +1,5 @@ --- -status: shipped -date: 2026-06-19 -slug: messaging-service-patterns-doc summary: New docs/patterns/ section with one page composing the outbox in an anonymized chat/notifications service: transactional event relay, fire-unless-cancelled timer, nested test brokers. -supersedes: null -superseded_by: null -pr: 103 -outcome: Shipped docs/patterns/messaging-service.md + a Patterns nav section — one anonymized service composing transactional relay, fire-unless-cancelled timer, and nested test brokers. No architecture/ change (docs only, no new invariant). --- # Design: A "Patterns" docs page composing the outbox in a real service diff --git a/planning/changes/2026-06-19.01-messaging-service-patterns-doc/plan.md b/planning/changes/2026-06-19.01-messaging-service-patterns-doc/plan.md index f1a13d6..d62ea17 100644 --- a/planning/changes/2026-06-19.01-messaging-service-patterns-doc/plan.md +++ b/planning/changes/2026-06-19.01-messaging-service-patterns-doc/plan.md @@ -1,11 +1,3 @@ ---- -status: shipped -date: 2026-06-19 -slug: messaging-service-patterns-doc -spec: design.md -pr: 103 ---- - # messaging-service-patterns-doc — implementation plan > **For agentic workers:** REQUIRED SUB-SKILL: Use diff --git a/planning/changes/2026-06-19.02-docs-diataxis-nav/design.md b/planning/changes/2026-06-19.02-docs-diataxis-nav/design.md index 0eeb406..d526e00 100644 --- a/planning/changes/2026-06-19.02-docs-diataxis-nav/design.md +++ b/planning/changes/2026-06-19.02-docs-diataxis-nav/design.md @@ -1,12 +1,5 @@ --- -status: shipped -date: 2026-06-19 -slug: docs-diataxis-nav summary: Dissolve the standalone Patterns nav section: fold the messaging-service case study into Guides and move the file to docs/usage/ (no section renames). Seven top-level sections to six. -supersedes: null -superseded_by: null -pr: 104 -outcome: Dropped the Patterns nav section; messaging-service page moved to docs/usage/ under Guides (no renames). 7 sections → 6. No architecture/ change (docs only). --- # Design: Dissolve the "Patterns" nav section into Guides diff --git a/planning/changes/2026-06-19.02-docs-diataxis-nav/plan.md b/planning/changes/2026-06-19.02-docs-diataxis-nav/plan.md index 8b1c670..c1af294 100644 --- a/planning/changes/2026-06-19.02-docs-diataxis-nav/plan.md +++ b/planning/changes/2026-06-19.02-docs-diataxis-nav/plan.md @@ -1,11 +1,3 @@ ---- -status: shipped -date: 2026-06-19 -slug: docs-diataxis-nav -spec: design.md -pr: 104 ---- - # docs-diataxis-nav — implementation plan > **For agentic workers:** REQUIRED SUB-SKILL: Use diff --git a/planning/changes/2026-06-20.01-flat-changes-generated-index/design.md b/planning/changes/2026-06-20.01-flat-changes-generated-index/design.md index 72ce791..25ed524 100644 --- a/planning/changes/2026-06-20.01-flat-changes-generated-index/design.md +++ b/planning/changes/2026-06-20.01-flat-changes-generated-index/design.md @@ -1,12 +1,5 @@ --- -status: shipped -date: 2026-06-20 -slug: flat-changes-generated-index summary: Flatten changes/ (drop active/archive), make status frontmatter the sole lifecycle state, add a summary field, and replace the hand-maintained README Index with a stdlib generator (just index). -supersedes: null -superseded_by: null -pr: 105 -outcome: Flattened planning/changes/ (active/archive removed); added single-line summary: to every bundle; new stdlib planning/index.py + just index generator (grouped In progress/Shipped/Superseded, stdout-only); slimmed README to conventions + generator note; single-step in-branch lifecycle in README + CLAUDE.md. --- # Design: Flatten changes/ and generate the index from frontmatter diff --git a/planning/changes/2026-06-20.01-flat-changes-generated-index/plan.md b/planning/changes/2026-06-20.01-flat-changes-generated-index/plan.md index 06c0fc5..ec874a0 100644 --- a/planning/changes/2026-06-20.01-flat-changes-generated-index/plan.md +++ b/planning/changes/2026-06-20.01-flat-changes-generated-index/plan.md @@ -1,11 +1,3 @@ ---- -status: shipped -date: 2026-06-20 -slug: flat-changes-generated-index -spec: design.md -pr: 105 ---- - # flat-changes-generated-index — implementation plan > **For agentic workers:** REQUIRED SUB-SKILL: Use diff --git a/planning/changes/2026-06-23.01-client-rules-kernel/design.md b/planning/changes/2026-06-23.01-client-rules-kernel/design.md index 575a54f..3402b3e 100644 --- a/planning/changes/2026-06-23.01-client-rules-kernel/design.md +++ b/planning/changes/2026-06-23.01-client-rules-kernel/design.md @@ -1,21 +1,5 @@ --- -status: shipped -date: 2026-06-23 -slug: client-rules-kernel summary: Deduplicate the outbox rules between the real and fake clients — extract the genuinely-pure bits (DLQ projection, scheduling resolution) and co-verify the irreducibly-SQL bits with one contract suite run against both adapters. -supersedes: null -superseded_by: null -pr: 109 -outcome: | - Landed as planned. Pure bits extracted: `_scheduling.py` (activate-args resolution + - validation, shared by real and fake publish paths) and `_DLQ_PROJECTION` in `schema.py` - (single source for the outbox→DLQ column mapping). Irreducibly-SQL rules co-verified by - `tests/test_client_contract.py` — one parametrized module over both adapters (fake - everywhere, real Postgres auto-skipped). Scope correction during execution: `cancel_timer` - and `timer_id` insert-dedup are broker/producer concerns, not on `AbstractOutboxClient`, so - they were excluded from the contract suite; within-batch fetch *return order* is unspecified - (F2-09), so the suite asserts FIFO *selection* under LIMIT, not return order. Replace-don't-layer - removed ~220 lines of subsumed per-adapter tests; full suite 543 passed at 100% coverage. --- # Design: Dedupe the outbox rules between the real and fake clients diff --git a/planning/changes/2026-06-23.01-client-rules-kernel/plan.md b/planning/changes/2026-06-23.01-client-rules-kernel/plan.md index 3bd1374..8fdbb6a 100644 --- a/planning/changes/2026-06-23.01-client-rules-kernel/plan.md +++ b/planning/changes/2026-06-23.01-client-rules-kernel/plan.md @@ -1,11 +1,3 @@ ---- -status: shipped -date: 2026-06-23 -slug: client-rules-kernel -spec: design.md -pr: 109 ---- - # client-rules-kernel — implementation plan > **For agentic workers:** REQUIRED SUB-SKILL: Use diff --git a/planning/changes/2026-06-23.02-consolidate-lease-lost/change.md b/planning/changes/2026-06-23.02-consolidate-lease-lost/change.md index 7f553d1..3714af0 100644 --- a/planning/changes/2026-06-23.02-consolidate-lease-lost/change.md +++ b/planning/changes/2026-06-23.02-consolidate-lease-lost/change.md @@ -1,17 +1,5 @@ --- -status: shipped -date: 2026-06-23 -slug: consolidate-lease-lost summary: Consolidate the duplicated lease-lost detect→log→emit block shared by _flush_terminal and _flush_retry into one _emit_lease_lost helper. -supersedes: null -superseded_by: null -pr: 110 -outcome: | - Landed as the minimal (A) form: `_emit_lease_lost(row, *, phase)` in - subscriber/usecase.py; both flush methods call it. ~36 duplicated lines removed. - Existing lease-lost unit tests (which invoke the flush methods directly) passed - unchanged as the regression guard; full suite 543 passed at 100% coverage. (B) the - Lease value object and (C) the Lease module were evaluated and rejected — see Approach. --- # Change: Give lease-lost telemetry one home diff --git a/planning/changes/2026-06-23.03-self-validating-subscriber-config/change.md b/planning/changes/2026-06-23.03-self-validating-subscriber-config/change.md index 5686b40..ffc141a 100644 --- a/planning/changes/2026-06-23.03-self-validating-subscriber-config/change.md +++ b/planning/changes/2026-06-23.03-self-validating-subscriber-config/change.md @@ -1,20 +1,5 @@ --- -status: shipped -date: 2026-06-23 -slug: self-validating-subscriber-config summary: Move subscriber-knob validation from the factory into OutboxSubscriberConfig.__post_init__ so every construction path is validated, not just the factory's. -supersedes: null -superseded_by: null -pr: 111 -outcome: | - Landed. Validation moved to OutboxSubscriberConfig.__post_init__ (guarded super-call + - self._validate()); factory.py dropped _validate_subscriber_config and now just wires. - Behavior preserved exactly (EMPTY→None ack_policy mapping). One wrinkle surfaced under - CI: moving validation under the dataclass-generated __init__ added a "" frame - that the 3.13 C warnings.warn(skip_file_prefixes=...) refuses to skip (works on 3.14), - so the FastAPI-router attribution test failed in docker. Replaced skip_file_prefixes - with a manual stacklevel walk (_subscriber_warn) that's version- and call-path-robust. - Existing validation tests passed as the regression guard; full suite 543 passed at 100%. --- # Change: Make the subscriber config validate itself diff --git a/planning/decisions/2026-06-23-metrics-recorders-not-unified.md b/planning/decisions/2026-06-23-metrics-recorders-not-unified.md index 7b3e2ae..91f4b66 100644 --- a/planning/decisions/2026-06-23-metrics-recorders-not-unified.md +++ b/planning/decisions/2026-06-23-metrics-recorders-not-unified.md @@ -1,11 +1,8 @@ --- status: accepted -date: 2026-06-23 -slug: metrics-recorders-not-unified summary: Keep PrometheusRecorder and OpenTelemetryRecorder as separate hand-written event switches — do not factor them behind a shared data-driven event→metric table. supersedes: null superseded_by: null -pr: 112 --- # Metrics recorders stay separate; no shared event→metric table diff --git a/planning/index.py b/planning/index.py index ec6c2dc..8916661 100644 --- a/planning/index.py +++ b/planning/index.py @@ -7,6 +7,9 @@ reads their frontmatter, and prints a Markdown listing to stdout — changes then decisions, newest-first. Never writes a file: the listing is a query over the files, not a committed artifact. + +``date`` and ``slug`` are derived from the directory / file name, not +frontmatter — the name is the single source of truth for both. """ import pathlib @@ -17,12 +20,11 @@ CHANGES_DIR = pathlib.Path(__file__).parent / "changes" DECISIONS_DIR = pathlib.Path(__file__).parent / "decisions" VALID_DECISION_STATUS = {"accepted", "superseded"} -DATE_RE = re.compile(r"^\d{4}-\d{2}-\d{2}$") -BUNDLE_RE = re.compile(r"^\d{4}-\d{2}-\d{2}\.\d{2}-(?P.+)$") +BUNDLE_RE = re.compile(r"^(?P\d{4}-\d{2}-\d{2})\.\d{2}-(?P.+)$") +DECISION_RE = re.compile(r"^(?P\d{4}-\d{2}-\d{2})-(?P.+)$") ALLOWED_BUNDLE_FILES = {"design.md", "plan.md", "change.md"} -SPEC_REQUIRED = ("date", "slug", "summary", "outcome") -PLAN_REQUIRED = ("date", "slug", "spec") -DECISION_REQUIRED = ("status", "date", "slug", "summary") +SPEC_REQUIRED = ("summary",) +DECISION_REQUIRED = ("status", "summary") def parse_frontmatter(text: str) -> dict[str, str]: @@ -44,8 +46,17 @@ def parse_frontmatter(text: str) -> dict[str, str]: return fields +def _named(fields: dict[str, str], name: str, pattern: re.Pattern[str]) -> dict[str, str]: + """Inject ``date``/``slug`` derived from a dir/file name into ``fields``.""" + match = pattern.match(name) + if match: + fields["date"] = match.group("date") + fields["slug"] = match.group("slug") + return fields + + def load_bundles() -> list[dict[str, str]]: - """Read every bundle's spec frontmatter under ``CHANGES_DIR``.""" + """Read each bundle's summary; derive date/slug from the directory name.""" bundles: list[dict[str, str]] = [] for bundle in sorted(CHANGES_DIR.iterdir()): if not bundle.is_dir(): @@ -55,7 +66,7 @@ def load_bundles() -> list[dict[str, str]]: spec = bundle / "change.md" if not spec.exists(): continue - fields = parse_frontmatter(spec.read_text(encoding="utf-8")) + fields = _named(parse_frontmatter(spec.read_text(encoding="utf-8")), bundle.name, BUNDLE_RE) fields["path"] = f"changes/{bundle.name}/{spec.name}" fields["name"] = bundle.name bundles.append(fields) @@ -63,14 +74,14 @@ def load_bundles() -> list[dict[str, str]]: def load_decisions() -> list[dict[str, str]]: - """Read frontmatter from every decision file under ``DECISIONS_DIR``.""" + """Read each decision's frontmatter; derive date/slug from the file name.""" decisions: list[dict[str, str]] = [] if not DECISIONS_DIR.is_dir(): return decisions for path in sorted(DECISIONS_DIR.glob("*.md")): if path.name == "README.md" or path.name.startswith("_"): continue - fields = parse_frontmatter(path.read_text(encoding="utf-8")) + fields = _named(parse_frontmatter(path.read_text(encoding="utf-8")), path.stem, DECISION_RE) fields["path"] = f"decisions/{path.name}" fields["name"] = path.stem decisions.append(fields) @@ -108,41 +119,16 @@ def _require(fields: dict[str, str], keys: tuple[str, ...], rel: str, violations violations.extend(f"{rel}: missing or empty frontmatter key '{key}'" for key in keys if not fields.get(key)) -def _check_common(fields: dict[str, str], dir_slug: str | None, rel: str, violations: list[str]) -> None: - """Validate date/slug fields shared by every artifact type.""" - date = fields.get("date", "") - if date and not DATE_RE.match(date): - violations.append(f"{rel}: date '{date}' is not YYYY-MM-DD") - slug = fields.get("slug", "") - if dir_slug and slug and slug != dir_slug: - violations.append(f"{rel}: slug '{slug}' does not match directory slug '{dir_slug}'") - - -def _check_spec_file(path: pathlib.Path, rel: str, dir_slug: str | None, violations: list[str]) -> None: - """Validate a design.md / change.md spec file.""" +def _check_spec_file(path: pathlib.Path, rel: str, violations: list[str]) -> None: + """Validate a design.md / change.md spec file (requires `summary`).""" fields = parse_frontmatter(path.read_text(encoding="utf-8")) _require(fields, SPEC_REQUIRED, rel, violations) - _check_common(fields, dir_slug, rel, violations) - - -def _check_plan_file( - path: pathlib.Path, bundle: pathlib.Path, rel: str, dir_slug: str | None, violations: list[str] -) -> None: - """Validate a plan.md file, including that its spec: link resolves.""" - fields = parse_frontmatter(path.read_text(encoding="utf-8")) - _require(fields, PLAN_REQUIRED, rel, violations) - _check_common(fields, dir_slug, rel, violations) - spec = fields.get("spec", "") - if spec and not (bundle / spec).resolve().exists(): - violations.append(f"{rel}: spec link '{spec}' does not resolve to a file") def _check_bundle(bundle: pathlib.Path, violations: list[str]) -> None: """Validate one change bundle directory.""" rel = f"changes/{bundle.name}" - match = BUNDLE_RE.match(bundle.name) - dir_slug = match.group("slug") if match else None - if match is None: + if BUNDLE_RE.match(bundle.name) is None: violations.append(f"{rel}: directory name is not 'YYYY-MM-DD.NN-slug'") violations.extend( f"{rel}/{child.name}: unexpected file in bundle (allowed: {', '.join(sorted(ALLOWED_BUNDLE_FILES))})" @@ -151,22 +137,21 @@ def _check_bundle(bundle: pathlib.Path, violations: list[str]) -> None: ) design = bundle / "design.md" change = bundle / "change.md" - plan = bundle / "plan.md" if not design.exists() and not change.exists(): violations.append(f"{rel}: bundle has neither design.md nor change.md") for spec_file in (design, change): if spec_file.exists(): - _check_spec_file(spec_file, f"{rel}/{spec_file.name}", dir_slug, violations) - if plan.exists(): - _check_plan_file(plan, bundle, f"{rel}/plan.md", dir_slug, violations) + _check_spec_file(spec_file, f"{rel}/{spec_file.name}", violations) + # plan.md carries no frontmatter — its identity comes from the bundle dir. def _check_decision(path: pathlib.Path, violations: list[str]) -> None: - """Validate one decision file.""" + """Validate one decision file (requires `status` + `summary`).""" rel = f"decisions/{path.name}" + if DECISION_RE.match(path.stem) is None: + violations.append(f"{rel}: file name is not 'YYYY-MM-DD-slug.md'") fields = parse_frontmatter(path.read_text(encoding="utf-8")) _require(fields, DECISION_REQUIRED, rel, violations) - _check_common(fields, None, rel, violations) status = fields.get("status", "") if status and status not in VALID_DECISION_STATUS: violations.append(f"{rel}: invalid status '{status}' (allowed: {', '.join(sorted(VALID_DECISION_STATUS))})") From dafcca0d214227640da1307e26cb358e7359b330 Mon Sep 17 00:00:00 2001 From: Artur Shiriev Date: Thu, 25 Jun 2026 19:27:06 +0300 Subject: [PATCH 3/3] style: align docstring convention to D212 (org standard), drop RUF100 hack faststream-outbox was the only repo inverting the docstring rule (ignoring D212 / enforcing D213), which left the vendored planning/index.py's file-level `noqa: D212` unused and forced a `RUF100` per-file ignore. Flip to the modern-python standard (enforce D212, ignore D213) so the vendored file is naturally clean and no special-casing is needed. ruff auto-applied the D213->D212 summary-line move across 39 files (mechanical; no logic change). just lint-ci green; no per-file RUF100 ignore. --- faststream_outbox/_import_checker.py | 6 +- faststream_outbox/_scheduling.py | 3 +- faststream_outbox/annotations.py | 3 +- faststream_outbox/broker.py | 24 +++----- faststream_outbox/client.py | 39 +++++-------- faststream_outbox/configs.py | 3 +- faststream_outbox/envelope.py | 3 +- faststream_outbox/fastapi/__init__.py | 3 +- faststream_outbox/fastapi/router.py | 3 +- faststream_outbox/message.py | 9 +-- faststream_outbox/metrics/__init__.py | 6 +- faststream_outbox/metrics/opentelemetry.py | 6 +- faststream_outbox/metrics/prometheus.py | 6 +- faststream_outbox/opentelemetry/__init__.py | 3 +- faststream_outbox/opentelemetry/middleware.py | 3 +- faststream_outbox/prometheus/__init__.py | 3 +- faststream_outbox/prometheus/middleware.py | 3 +- faststream_outbox/publisher/fake.py | 3 +- faststream_outbox/publisher/producer.py | 3 +- faststream_outbox/publisher/usecase.py | 6 +- faststream_outbox/registrator.py | 6 +- faststream_outbox/response.py | 9 +-- faststream_outbox/retry.py | 3 +- faststream_outbox/router.py | 3 +- faststream_outbox/schema.py | 12 ++-- faststream_outbox/subscriber/config.py | 6 +- faststream_outbox/subscriber/usecase.py | 57 +++++++------------ faststream_outbox/testing.py | 21 +++---- pyproject.toml | 6 +- tests/conftest.py | 3 +- tests/test_client_contract.py | 3 +- tests/test_fake.py | 18 ++---- tests/test_fastapi.py | 9 +-- tests/test_integration.py | 51 ++++++----------- tests/test_metrics_prometheus.py | 3 +- tests/test_middleware_opentelemetry.py | 6 +- tests/test_middleware_prometheus.py | 6 +- tests/test_relay.py | 27 +++------ tests/test_unit.py | 48 ++++++---------- 39 files changed, 144 insertions(+), 291 deletions(-) diff --git a/faststream_outbox/_import_checker.py b/faststream_outbox/_import_checker.py index 797ee46..41df8df 100644 --- a/faststream_outbox/_import_checker.py +++ b/faststream_outbox/_import_checker.py @@ -1,5 +1,4 @@ -""" -Centralized probes for optional-extra imports. +"""Centralized probes for optional-extra imports. Each ``is_*_installed`` is a module-level boolean derived from ``importlib.util.find_spec`` so consumers can guard runtime imports without a @@ -22,8 +21,7 @@ def missing_extra_message(component: str, extra: str) -> str: - """ - Build the friendly "this needs an optional extra" install hint. + """Build the friendly "this needs an optional extra" install hint. Single source of truth for the message text so the import-time guard and the ``__init__`` probe guard in each middleware module stay in sync (B13). diff --git a/faststream_outbox/_scheduling.py b/faststream_outbox/_scheduling.py index 9aebbd1..3a969af 100644 --- a/faststream_outbox/_scheduling.py +++ b/faststream_outbox/_scheduling.py @@ -1,5 +1,4 @@ -""" -Pure activate-args resolution + validation, shared by the real and fake publish paths. +"""Pure activate-args resolution + validation, shared by the real and fake publish paths. ``activate_in`` / ``activate_at`` are the user-facing scheduling knobs. Turning them into a single ``next_attempt_at`` (client clock) and deciding whether a row is diff --git a/faststream_outbox/annotations.py b/faststream_outbox/annotations.py index b12eadb..7d912b4 100644 --- a/faststream_outbox/annotations.py +++ b/faststream_outbox/annotations.py @@ -1,5 +1,4 @@ -""" -Annotated context shortcuts for handler signatures. +"""Annotated context shortcuts for handler signatures. Mirrors the native FastStream convention (see ``faststream.kafka.annotations``): the class names are imported with ``as`` aliases and re-exported as diff --git a/faststream_outbox/broker.py b/faststream_outbox/broker.py index 264cd04..3640544 100644 --- a/faststream_outbox/broker.py +++ b/faststream_outbox/broker.py @@ -1,5 +1,4 @@ -""" -OutboxBroker — a FastStream broker whose queue is a Postgres table. +"""OutboxBroker — a FastStream broker whose queue is a Postgres table. Producers call ``broker.publish(body, queue=..., session=session)`` inside their own SQLAlchemy transaction; the row commits with their domain writes. The broker @@ -56,8 +55,7 @@ def _spec_url(engine: "AsyncEngine | None", outbox_table: "Table") -> list[str]: - """ - AsyncAPI server URL(s) for the broker spec. + """AsyncAPI server URL(s) for the broker spec. **Must be non-empty.** Upstream's AsyncAPI generator only emits channels/operations for brokers whose spec carries a non-empty ``url`` (it populates ``broker_servers`` @@ -73,8 +71,7 @@ def _spec_url(engine: "AsyncEngine | None", outbox_table: "Table") -> list[str]: class _CaptureExceptionMiddleware(BaseMiddleware): - """ - Stash the handler exception on the inner row before AckMiddleware nacks. + """Stash the handler exception on the inner row before AckMiddleware nacks. FastStream's AcknowledgementMiddleware catches the handler exception in its own ``after_processed`` and calls ``message.nack()`` directly — the exception @@ -255,8 +252,7 @@ def _warn_on_duplicate_queues(self) -> None: ) def _warn_on_unstarted_foreign_publishers(self) -> None: - """ - Emit one WARNING per foreign-publisher broker that has not been started. + """Emit one WARNING per foreign-publisher broker that has not been started. Foreign-publisher decorators stacked on outbox subscribers only work if the foreign broker's producer is wired. When it is not, the first @@ -377,8 +373,7 @@ async def publish( # ty: ignore[invalid-method-override] activate_at: _dt.datetime | None = None, timer_id: str | None = None, ) -> int | None: - """ - Insert one outbox row using *session*'s open transaction. + """Insert one outbox row using *session*'s open transaction. Must be called inside a transaction the caller owns (typically inside an ``async with session.begin():`` block). ``publish`` does not flush, commit, @@ -415,8 +410,7 @@ async def publish_batch( # ty: ignore[invalid-method-override] activate_in: _dt.timedelta | None = None, activate_at: _dt.datetime | None = None, ) -> None: - """ - Insert multiple outbox rows via *session*. Same transactional contract as ``publish``. + """Insert multiple outbox rows via *session*. Same transactional contract as ``publish``. Each row gets its own auto-generated ``correlation_id``; pass *headers* to share static headers across all rows. *activate_in* / *activate_at* schedule @@ -459,8 +453,7 @@ async def cancel_timer( timer_id: str, session: AsyncSession, ) -> bool: - """ - Delete a not-yet-leased timer row. Idempotent at the ``(queue, timer_id)`` level. + """Delete a not-yet-leased timer row. Idempotent at the ``(queue, timer_id)`` level. Same transactional contract as :meth:`publish` — runs on the caller's session and commits with their transaction. @@ -505,8 +498,7 @@ async def fetch_unprocessed( queue: str | None = None, limit: int = 1000, ) -> list[OutboxInnerMessage]: - """ - Return outbox rows currently in the table — pending, in-flight, or future-dated. + """Return outbox rows currently in the table — pending, in-flight, or future-dated. Intended for test assertions and lease-free operator inspection (the lease-free read path `get_one()`/`__aiter__()` point you here): a successful delivery deletes diff --git a/faststream_outbox/client.py b/faststream_outbox/client.py index 9d629e3..f923af7 100644 --- a/faststream_outbox/client.py +++ b/faststream_outbox/client.py @@ -1,5 +1,4 @@ -""" -Postgres outbox client. +"""Postgres outbox client. All read/write paths against the outbox table live here. The fetch query is the load-bearing piece: a single CTE that selects available rows ``FOR UPDATE SKIP LOCKED`` @@ -74,8 +73,7 @@ class AbstractOutboxClient(abc.ABC): - """ - Outbox client interface. + """Outbox client interface. Satisfied by both :class:`OutboxClient` (real Postgres) and ``FakeOutboxClient`` (in-memory test substitute, defined in ``testing.py``). The subscriber's ``_client`` @@ -159,8 +157,7 @@ def table(self) -> "Table": @property def engine(self) -> "AsyncEngine": - """ - The underlying ``AsyncEngine``. + """The underlying ``AsyncEngine``. Used by the subscriber loop to open its own long-lived fetch connection and to drive ``LISTEN/NOTIFY``. @@ -175,8 +172,7 @@ async def fetch( limit: int, lease_ttl_seconds: float, ) -> list[OutboxInnerMessage]: - """ - Atomically claim up to *limit* available rows for the given queue names on *conn*. + """Atomically claim up to *limit* available rows for the given queue names on *conn*. A row is available iff its lease is unset (``acquired_token IS NULL``) or its lease is older than *lease_ttl_seconds*. Returns the freshly-leased rows; each @@ -249,8 +245,7 @@ async def delete_with_lease( *, dlq_payload: "Mapping[str, typing.Any] | None" = None, ) -> bool: - """ - Delete *message_id* iff it still holds *acquired_token*. Returns True if deleted. + """Delete *message_id* iff it still holds *acquired_token*. Returns True if deleted. Issues a single ``DELETE`` on *conn* with no explicit transaction wrapper — the production writer connection is configured ``isolation_level="AUTOCOMMIT"`` by @@ -310,8 +305,7 @@ def _build_dlq_cte_stmt( acquired_token: uuid.UUID, dlq_payload: "Mapping[str, typing.Any]", ) -> "tuple[typing.Any, dict[str, typing.Any]]": - """ - Compose the single-statement DLQ CTE plus the parameter dict. + """Compose the single-statement DLQ CTE plus the parameter dict. Identifiers are quoted via the dialect's identifier preparer so reserved words and odd characters survive interpolation. The outbox/DLQ table names are @@ -374,8 +368,7 @@ async def mark_pending_with_lease( first_attempt_at: _dt.datetime, last_attempt_at: _dt.datetime, ) -> bool: - """ - Release the lease on *message_id* and reschedule it for retry, iff it still holds the lease. + """Release the lease on *message_id* and reschedule it for retry, iff it still holds the lease. Issues a single ``UPDATE`` on *conn* with no explicit transaction wrapper — the production writer connection is configured ``isolation_level="AUTOCOMMIT"`` by @@ -407,8 +400,7 @@ async def mark_pending_with_lease( return (result.rowcount or 0) > 0 async def validate_schema(self) -> None: - """ - Validate that the database table(s) match the package's expected columns. + """Validate that the database table(s) match the package's expected columns. Raises ``RuntimeError`` listing every mismatch across the outbox table and, when configured, the DLQ table. Opt-in: call from your startup hook or @@ -494,8 +486,7 @@ def _normalize_predicate(predicate: str) -> str: def _validate_index_predicates_sync(connection: "Connection", table: "Table") -> list[str]: - """ - Compare the live partial-index WHERE predicates against what the package expects (S2). + """Compare the live partial-index WHERE predicates against what the package expects (S2). Alembic's index diff ignores ``postgresql_where``, so the alembic autogenerate pass (:func:`_run_validate`) does not catch two drifts that break the producer's ``ON CONFLICT`` @@ -552,8 +543,7 @@ def _validate_index_predicates_sync(connection: "Connection", table: "Table") -> def _validate_check_constraints_sync(connection: "Connection", table: "Table") -> list[str]: - """ - Verify the live DB carries a CHECK enforcing each invariant the package needs. + """Verify the live DB carries a CHECK enforcing each invariant the package needs. Alembic's ``compare_metadata`` registers no check-constraint comparator, so a missing or altered lease CHECK — the ``(acquired_token IS NULL) = (acquired_at IS NULL)`` invariant @@ -625,8 +615,7 @@ def _run_validate( table: "Table", canonical_factory: "Callable[[MetaData, str], Table]", ) -> list[str]: - """ - Run Alembic's autogenerate diff against the live DB and surface any "missing schema" drift. + """Run Alembic's autogenerate diff against the live DB and surface any "missing schema" drift. The canonical schema is whatever ``canonical_factory`` produces — the same Table the user attaches to their own ``MetaData`` via ``make_outbox_table`` / ``make_dlq_table``. Delegating @@ -680,8 +669,7 @@ def _include_name(name: str | None, type_: str, parent_names: "Mapping[str, str def _flatten_drift_errors(diff: "Sequence[typing.Any]", table_name: str) -> list[str]: - """ - Walk Alembic's nested diff and surface only the ops that mean *missing schema*. + """Walk Alembic's nested diff and surface only the ops that mean *missing schema*. Top-level entries are tuples for table-level ops (``add_table``, ``remove_table``) and lists of nested tuples for column / index ops on existing tables. ``remove_*`` ops are @@ -702,8 +690,7 @@ def _flatten_drift_errors(diff: "Sequence[typing.Any]", table_name: str) -> list def _drift_entry_to_error(entry: "tuple[typing.Any, ...]", table_name: str) -> str | None: - """ - Map one Alembic op tuple to a human-readable error string, or None to ignore. + """Map one Alembic op tuple to a human-readable error string, or None to ignore. Tuple shapes per Alembic's autogenerate contract: add_column -> (op, schema, table_name, Column) diff --git a/faststream_outbox/configs.py b/faststream_outbox/configs.py index 51fcb4e..eb206b8 100644 --- a/faststream_outbox/configs.py +++ b/faststream_outbox/configs.py @@ -1,5 +1,4 @@ -""" -Broker config for the outbox transport. +"""Broker config for the outbox transport. The user owns the ``AsyncEngine``; the broker never closes it. The engine is stored on ``OutboxBrokerConfig`` so the broker can hand the same reference to its diff --git a/faststream_outbox/envelope.py b/faststream_outbox/envelope.py index 42f9d37..dc4875b 100644 --- a/faststream_outbox/envelope.py +++ b/faststream_outbox/envelope.py @@ -17,8 +17,7 @@ def _encode_payload( correlation_id: str | None = None, serializer: "SerializerProto | None" = None, ) -> tuple[bytes, dict[str, str]]: - """ - Serialize *body* into ``(payload_bytes, headers_dict)`` for an outbox row. + """Serialize *body* into ``(payload_bytes, headers_dict)`` for an outbox row. *body* may be ``bytes``, a pydantic model, a dataclass, a ``dict``, or any value FastStream's ``encode_message`` accepts. *correlation_id* is auto-generated if diff --git a/faststream_outbox/fastapi/__init__.py b/faststream_outbox/fastapi/__init__.py index b6ff5e4..9906ccc 100644 --- a/faststream_outbox/fastapi/__init__.py +++ b/faststream_outbox/fastapi/__init__.py @@ -1,5 +1,4 @@ -""" -Public surface for the outbox FastAPI integration. +"""Public surface for the outbox FastAPI integration. Mirrors ``faststream.kafka.fastapi``: the router and the ``Annotated[..., Context(...)]`` shortcuts handlers will reference. Import from this module when wiring an outbox diff --git a/faststream_outbox/fastapi/router.py b/faststream_outbox/fastapi/router.py index 161bf05..7d80e71 100644 --- a/faststream_outbox/fastapi/router.py +++ b/faststream_outbox/fastapi/router.py @@ -1,5 +1,4 @@ -""" -FastAPI integration for the outbox transport. +"""FastAPI integration for the outbox transport. ``OutboxRouter`` is a thin subclass of FastStream's ``StreamRouter`` (which is itself an ``APIRouter``). Mounting the router into a FastAPI app via diff --git a/faststream_outbox/message.py b/faststream_outbox/message.py index 923c991..74cdd88 100644 --- a/faststream_outbox/message.py +++ b/faststream_outbox/message.py @@ -1,5 +1,4 @@ -""" -Outbox message representations. +"""Outbox message representations. ``OutboxInnerMessage`` is the in-memory mirror of a row claimed by the fetch loop. Its ``ack``/``nack``/``reject`` methods only mutate in-memory intent — the actual @@ -39,8 +38,7 @@ @dataclass(kw_only=True) class OutboxInnerMessage: - """ - In-memory copy of a claimed outbox row, plus ack/nack/reject intent helpers. + """In-memory copy of a claimed outbox row, plus ack/nack/reject intent helpers. The ack/nack/reject methods set in-memory intent flags (``to_delete``, ``pending_delay_seconds``). The worker loop reads those flags and issues the @@ -165,8 +163,7 @@ def allow_delivery(self, *, max_deliveries: int | None, logger: "LoggerProto | N return True async def assert_state_set(self, logger: "LoggerProto | None") -> None: - """ - Fallback when the consume pipeline returned without recording ack/nack/reject intent. + """Fallback when the consume pipeline returned without recording ack/nack/reject intent. Two distinct shapes land here: diff --git a/faststream_outbox/metrics/__init__.py b/faststream_outbox/metrics/__init__.py index 3358712..f994549 100644 --- a/faststream_outbox/metrics/__init__.py +++ b/faststream_outbox/metrics/__init__.py @@ -1,5 +1,4 @@ -""" -Metrics seam: a single callable invoked at well-defined instrumentation points. +"""Metrics seam: a single callable invoked at well-defined instrumentation points. The seam is intentionally minimal — ``Callable[[str, Mapping[str, Any]], None]`` — so adapters live next to the library without dragging Prometheus, OpenTelemetry, @@ -72,8 +71,7 @@ def _noop_recorder(_event: str, _tags: Mapping[str, typing.Any]) -> None: def _safe_emit(recorder: MetricsRecorder, event: str, tags: Mapping[str, typing.Any]) -> None: - """ - Invoke ``recorder`` swallowing exceptions and logging at DEBUG. + """Invoke ``recorder`` swallowing exceptions and logging at DEBUG. Shared by every call site that emits metrics from the test broker. A broken user-supplied recorder must never poison the dispatch path — DEBUG-level diff --git a/faststream_outbox/metrics/opentelemetry.py b/faststream_outbox/metrics/opentelemetry.py index 2e6b96a..f89b9d2 100644 --- a/faststream_outbox/metrics/opentelemetry.py +++ b/faststream_outbox/metrics/opentelemetry.py @@ -1,5 +1,4 @@ -""" -OpenTelemetry meter adapter for the ``MetricsRecorder`` seam. +"""OpenTelemetry meter adapter for the ``MetricsRecorder`` seam. Instrument names, units, attribute keys, and constructor argument names mirror ``faststream.opentelemetry.TelemetryMiddleware`` (meter side). No ``outbox`` @@ -69,8 +68,7 @@ class OpenTelemetryRecorder: - """ - Drop-in OpenTelemetry meter adapter for ``MetricsRecorder``. + """Drop-in OpenTelemetry meter adapter for ``MetricsRecorder``. Args: meter_provider: optional. Defaults to the globally configured meter diff --git a/faststream_outbox/metrics/prometheus.py b/faststream_outbox/metrics/prometheus.py index 2707858..c30be6e 100644 --- a/faststream_outbox/metrics/prometheus.py +++ b/faststream_outbox/metrics/prometheus.py @@ -1,5 +1,4 @@ -""" -Prometheus adapter for the ``MetricsRecorder`` seam. +"""Prometheus adapter for the ``MetricsRecorder`` seam. Drop-in shape parity with ``faststream.prometheus.PrometheusMiddleware``. @@ -75,8 +74,7 @@ class PrometheusRecorder: - """ - Drop-in Prometheus adapter for ``MetricsRecorder``. + """Drop-in Prometheus adapter for ``MetricsRecorder``. Args: registry: Prometheus collector registry. Required (no global default so diff --git a/faststream_outbox/opentelemetry/__init__.py b/faststream_outbox/opentelemetry/__init__.py index 7e70b00..ba0fadf 100644 --- a/faststream_outbox/opentelemetry/__init__.py +++ b/faststream_outbox/opentelemetry/__init__.py @@ -1,5 +1,4 @@ -""" -Native OpenTelemetry integration for the outbox broker. +"""Native OpenTelemetry integration for the outbox broker. Mirrors upstream FastStream's ``faststream//opentelemetry/`` directory convention. Use this when you want spans + meters via FastStream's middleware diff --git a/faststream_outbox/opentelemetry/middleware.py b/faststream_outbox/opentelemetry/middleware.py index 955f0dd..bff3977 100644 --- a/faststream_outbox/opentelemetry/middleware.py +++ b/faststream_outbox/opentelemetry/middleware.py @@ -1,5 +1,4 @@ -""" -Outbox subclass of FastStream's ``TelemetryMiddleware``. +"""Outbox subclass of FastStream's ``TelemetryMiddleware``. Register via ``broker_middlewares=[...]`` to get spans + meters wrapping ``publish_scope`` and ``consume_scope`` — same registration pattern as the diff --git a/faststream_outbox/prometheus/__init__.py b/faststream_outbox/prometheus/__init__.py index 90c3f83..ac975ce 100644 --- a/faststream_outbox/prometheus/__init__.py +++ b/faststream_outbox/prometheus/__init__.py @@ -1,5 +1,4 @@ -""" -Native Prometheus integration for the outbox broker. +"""Native Prometheus integration for the outbox broker. Mirrors upstream FastStream's ``faststream//prometheus/`` directory convention. Use this when you want consume / publish counters + duration diff --git a/faststream_outbox/prometheus/middleware.py b/faststream_outbox/prometheus/middleware.py index 0abdd4a..376b0ae 100644 --- a/faststream_outbox/prometheus/middleware.py +++ b/faststream_outbox/prometheus/middleware.py @@ -1,5 +1,4 @@ -""" -Outbox subclass of FastStream's ``PrometheusMiddleware``. +"""Outbox subclass of FastStream's ``PrometheusMiddleware``. Register via ``broker_middlewares=[...]`` for consume + publish counters and duration histograms. Same registration pattern as ``KafkaPrometheusMiddleware`` diff --git a/faststream_outbox/publisher/fake.py b/faststream_outbox/publisher/fake.py index f60e31f..18a4175 100644 --- a/faststream_outbox/publisher/fake.py +++ b/faststream_outbox/publisher/fake.py @@ -1,5 +1,4 @@ -""" -Internal response publisher for handlers that ``return OutboxResponse(...)``. +"""Internal response publisher for handlers that ``return OutboxResponse(...)``. Wired up by ``OutboxSubscriber._make_response_publisher``. The ``isinstance(cmd, OutboxPublishCommand)`` gate is load-bearing: plain handler diff --git a/faststream_outbox/publisher/producer.py b/faststream_outbox/publisher/producer.py index eb51888..51495e9 100644 --- a/faststream_outbox/publisher/producer.py +++ b/faststream_outbox/publisher/producer.py @@ -1,5 +1,4 @@ -""" -``OutboxProducer`` — the canonical insert path for outbox rows. +"""``OutboxProducer`` — the canonical insert path for outbox rows. Both ``OutboxBroker.publish`` / ``publish_batch`` and ``OutboxPublisher.publish`` route through this producer via FastStream's ``_basic_publish(cmd, producer=...)`` diff --git a/faststream_outbox/publisher/usecase.py b/faststream_outbox/publisher/usecase.py index 1ecaaa2..dba50c2 100644 --- a/faststream_outbox/publisher/usecase.py +++ b/faststream_outbox/publisher/usecase.py @@ -1,5 +1,4 @@ -""" -``OutboxPublisher`` — typed, queue-scoped handle around ``broker.publish``. +"""``OutboxPublisher`` — typed, queue-scoped handle around ``broker.publish``. The publisher is intentionally **not** usable as a relay decorator on a subscriber (``__call__`` raises). The dispatch flow that would invoke @@ -76,8 +75,7 @@ async def publish( # ty: ignore[invalid-method-override] activate_at: _dt.datetime | None = None, timer_id: str | None = None, ) -> int | None: - """ - Insert one outbox row scoped to this publisher's queue. + """Insert one outbox row scoped to this publisher's queue. Same transactional contract as :meth:`OutboxBroker.publish`: runs on the caller's session and commits with their transaction. Static *headers* on diff --git a/faststream_outbox/registrator.py b/faststream_outbox/registrator.py index b003416..9bc4bbd 100644 --- a/faststream_outbox/registrator.py +++ b/faststream_outbox/registrator.py @@ -21,8 +21,7 @@ def _default_retry_strategy() -> "RetryStrategyProto": - """ - Fallback retry policy when the user passes nothing. + """Fallback retry policy when the user passes nothing. An outbox is a reliability primitive; defaulting to "delete on first error" turns every transient handler failure into silent data loss. Defaulting to a bounded @@ -108,8 +107,7 @@ def publisher( # ty: ignore[invalid-method-override] schema: Any | None = None, include_in_schema: bool = True, ) -> OutboxPublisher: - """ - Construct a queue-scoped publisher. + """Construct a queue-scoped publisher. The publisher is standalone-only — call ``await pub.publish(body, session=session)`` from inside your own transaction. Attempting to use it as a relay decorator on a diff --git a/faststream_outbox/response.py b/faststream_outbox/response.py index 43141c8..03a0cdc 100644 --- a/faststream_outbox/response.py +++ b/faststream_outbox/response.py @@ -1,5 +1,4 @@ -""" -Publish-side DTO for the outbox transport. +"""Publish-side DTO for the outbox transport. ``OutboxPublishCommand`` carries the domain fields that travel from the publisher / broker into ``OutboxProducer.publish``: the user's ``AsyncSession`` @@ -36,8 +35,7 @@ def _validate_publish_args( activate_in: _dt.timedelta | None, activate_at: _dt.datetime | None, ) -> None: - """ - Fail-fast validation shared by every real outbox publish entry point. + """Fail-fast validation shared by every real outbox publish entry point. ``OutboxPublishCommand``, ``OutboxResponse`` and ``broker.publish_batch`` all route through here so a misconfigured ``session`` / ``queue`` / ``activate_*`` @@ -148,8 +146,7 @@ def from_cmd( class OutboxResponse(Response): - """ - Handler return type — auto-published as a follow-on outbox row. + """Handler return type — auto-published as a follow-on outbox row. Idiomatic FastStream shape: ``async def h(...) -> OutboxResponse``. Requires ``session=...`` for the same reason ``broker.publish`` does — the new row must diff --git a/faststream_outbox/retry.py b/faststream_outbox/retry.py index 0cbcded..6fab447 100644 --- a/faststream_outbox/retry.py +++ b/faststream_outbox/retry.py @@ -25,8 +25,7 @@ def _validate_jitter_factor(jitter_factor: float) -> None: class RetryStrategyProto(Protocol): - """ - Decides whether a Nack'ed row gets another attempt and how long to wait. + """Decides whether a Nack'ed row gets another attempt and how long to wait. Implementations return the delay in seconds before the next attempt, or ``None`` to signal terminal failure (the row will be deleted). The DB diff --git a/faststream_outbox/router.py b/faststream_outbox/router.py index 5afdbc5..b9fffc5 100644 --- a/faststream_outbox/router.py +++ b/faststream_outbox/router.py @@ -63,8 +63,7 @@ def __init__( # noqa: PLR0913 class OutboxRouter(OutboxRegistrator, BrokerRouter[OutboxInnerMessage, OutboxBrokerConfig]): - """ - Includable router for ``OutboxBroker``. + """Includable router for ``OutboxBroker``. Use it to register subscribers in a separate module and attach them to the broker via ``broker.include_router(router)``. There is no ``prefix`` knob: diff --git a/faststream_outbox/schema.py b/faststream_outbox/schema.py index 46e0385..3cfac7d 100644 --- a/faststream_outbox/schema.py +++ b/faststream_outbox/schema.py @@ -1,5 +1,4 @@ -""" -Outbox table factory. +"""Outbox table factory. The package does not own the schema — users attach the returned ``Table`` to their own ``MetaData`` and write Alembic migrations themselves — and the partial indexes that the @@ -54,8 +53,7 @@ def validate_table_identifiers(table_name: str) -> None: - """ - Raise ``ValueError`` if any identifier derived from *table_name* exceeds Postgres' 63-byte limit. + """Raise ``ValueError`` if any identifier derived from *table_name* exceeds Postgres' 63-byte limit. Byte length, not char count — UTF-8 multibyte chars expand and would silently truncate identifiers. Guards the LONGEST derived identifier: the NOTIFY channel ``outbox_`` AND @@ -85,8 +83,7 @@ def validate_table_identifiers(table_name: str) -> None: def make_outbox_table(metadata: "MetaData", table_name: str = "outbox") -> Table: - """ - Build the outbox ``Table`` (with the partial fetch index) and attach it to *metadata*. + """Build the outbox ``Table`` (with the partial fetch index) and attach it to *metadata*. The user wires the returned table into their own SQLAlchemy ``MetaData`` so it is discovered by Alembic's autogenerate. They are responsible for the actual migration. @@ -176,8 +173,7 @@ def make_outbox_table(metadata: "MetaData", table_name: str = "outbox") -> Table def make_dlq_table(metadata: "MetaData", table_name: str = "outbox_dlq") -> Table: - """ - Build the dead-letter-queue ``Table`` and attach it to *metadata*. + """Build the dead-letter-queue ``Table`` and attach it to *metadata*. Opt-in companion to :func:`make_outbox_table`. Pass the returned table to ``OutboxBroker(..., dlq_table=...)`` to enable archive-on-terminal-failure: the diff --git a/faststream_outbox/subscriber/config.py b/faststream_outbox/subscriber/config.py index a08d996..1169ec9 100644 --- a/faststream_outbox/subscriber/config.py +++ b/faststream_outbox/subscriber/config.py @@ -29,8 +29,7 @@ def _is_internal_frame(filename: str) -> bool: def _subscriber_warn(message: str) -> None: - """ - Attribute a subscriber-config ``UserWarning`` to the user's ``@subscriber`` call (P27). + """Attribute a subscriber-config ``UserWarning`` to the user's ``@subscriber`` call (P27). Computes ``stacklevel`` by walking out to the first non-internal frame instead of using ``warnings.warn(skip_file_prefixes=...)``: the 3.13 C ``warn`` does not skip the @@ -72,8 +71,7 @@ def __post_init__(self) -> None: self._validate() def _validate(self) -> None: # noqa: C901 # flat sequence of independent knob checks - """ - Reject impossible knob values, warn on combos that silently misbehave. + """Reject impossible knob values, warn on combos that silently misbehave. Errors are raised here (not deferred to runtime) so the user gets a traceback pointing at the ``@broker.subscriber(...)`` decorator. Warnings use diff --git a/faststream_outbox/subscriber/usecase.py b/faststream_outbox/subscriber/usecase.py index d7e8e15..46053eb 100644 --- a/faststream_outbox/subscriber/usecase.py +++ b/faststream_outbox/subscriber/usecase.py @@ -1,5 +1,4 @@ -""" -Outbox subscriber — the consume loop that backs ``@broker.subscriber("queue")``. +"""Outbox subscriber — the consume loop that backs ``@broker.subscriber("queue")``. Two async tasks run per subscriber: @@ -98,8 +97,7 @@ class _OutboxConfigError(RuntimeError): ... def _compute_backoff(attempt: int, ceiling: float, *, base: float = 1.0) -> float: - """ - Exponential backoff with ±50% jitter, capped at *ceiling*. + """Exponential backoff with ±50% jitter, capped at *ceiling*. *attempt* is 1-based — the first attempt sleeps ~``base * U(0.5, 1.5)``. """ @@ -118,8 +116,7 @@ def _render_last_exception( exc: BaseException | None, renderer: "Callable[[BaseException], str | None] | None", ) -> str | None: - """ - Render *exc* for the DLQ ``last_exception`` column, then bound its length. + """Render *exc* for the DLQ ``last_exception`` column, then bound its length. Default (``renderer is None``) is ``repr(exc)`` — full forensic detail. A deployment handling PII can pass ``last_exception_renderer`` to redact (e.g. ``type(exc).__name__``) @@ -302,8 +299,7 @@ async def stop(self) -> None: @property def _notify_channel(self) -> str: - """ - LISTEN channel name. + """LISTEN channel name. One channel per outbox table; subscribers ignore queues they don't care about (cheap — wake-up does an empty fetch and goes back to sleep). @@ -324,8 +320,7 @@ async def _open_fetch_resources( self, engine: "AsyncEngine | None", ) -> AsyncIterator[Mapping[str, object]]: - """ - Yield the kwargs ``_fetch_inner`` needs, owning fetch_conn + listen_conn lifetimes. + """Yield the kwargs ``_fetch_inner`` needs, owning fetch_conn + listen_conn lifetimes. Production path opens a long-lived ``AsyncConnection`` for the fetch CTE and a separate raw asyncpg connection for LISTEN. Test-broker path (``engine is None``) @@ -348,8 +343,7 @@ async def _fetch_inner( fetch_conn: "AsyncConnection | None", listen_conn: "_asyncpg.Connection | None", ) -> None: - """ - Fetch + adaptive backoff, with NOTIFY-driven wakeup. + """Fetch + adaptive backoff, with NOTIFY-driven wakeup. Returns when ``self.running`` goes False, or raises on any DB error so the outer loop can rebuild the connection. Periodically probes ``listen_conn`` with a bounded @@ -407,8 +401,7 @@ async def _wait_for_notify_or_timeout(self, timeout: float) -> None: # noqa: AS self._notify_event.clear() async def _open_listen_connection(self, engine: "AsyncEngine") -> "_asyncpg.Connection | None": - """ - Open a dedicated raw asyncpg connection and register LISTEN on it. + """Open a dedicated raw asyncpg connection and register LISTEN on it. Returns the connection on success, ``None`` on any failure (asyncpg not installed, non-asyncpg driver, permission error, network problem). The fetch loop falls back @@ -452,8 +445,7 @@ async def _open_listen_connection(self, engine: "AsyncEngine") -> "_asyncpg.Conn return conn if listening else None async def _close_listen_connection(self, listen_conn: "_asyncpg.Connection") -> None: - """ - Close the raw LISTEN connection without letting teardown wedge the fetch loop (S1). + """Close the raw LISTEN connection without letting teardown wedge the fetch loop (S1). A graceful ``close()`` on a half-dead socket can block on the kernel keepalive (the same socket the bounded health probe may have just flagged). Cap the graceful @@ -467,8 +459,7 @@ async def _close_listen_connection(self, listen_conn: "_asyncpg.Connection") -> listen_conn.terminate() def _on_notify(self, *args: object) -> None: - """ - Asyncpg notification callback: ``(connection, pid, channel, payload)``. + """Asyncpg notification callback: ``(connection, pid, channel, payload)``. The payload is the publisher's queue name (``pg_notify('outbox_
', queue)``). We only wake for queues this subscriber serves — on a busy multi-queue table that @@ -494,8 +485,7 @@ async def _open_worker_resources( self, engine: "AsyncEngine | None", ) -> AsyncIterator[Mapping[str, object]]: - """ - Yield ``writer_conn`` for ``_worker_inner``, owning its lifetime across all flushes. + """Yield ``writer_conn`` for ``_worker_inner``, owning its lifetime across all flushes. One long-lived ``AsyncConnection`` per outer reconnect cycle — every terminal/retry write reuses it, so a drain of N rows costs O(workers) pool checkouts, not O(rows). @@ -522,8 +512,7 @@ async def _run_with_reconnect( inner: Callable[..., Awaitable[None]], halt_on_drain: bool = False, ) -> None: - """ - Reconnect-with-backoff scaffold shared by ``_fetch_loop`` and ``_worker_loop``. + """Reconnect-with-backoff scaffold shared by ``_fetch_loop`` and ``_worker_loop``. Reads the client lazily inside the loop (the test broker patches it in/out via ``mock.patch``, so it can be ``None`` after teardown — returning cleanly avoids @@ -567,8 +556,7 @@ async def _run_with_reconnect( await anyio.sleep(_compute_backoff(error_attempt, _BACKOFF_MAX_SECONDS)) async def _worker_inner(self, *, writer_conn: "AsyncConnection | None") -> None: - """ - Pull rows from the inflight queue and dispatch each, threading *writer_conn* through. + """Pull rows from the inflight queue and dispatch each, threading *writer_conn* through. Returns when ``self.running`` goes False, or raises on any DB error from the terminal write so :meth:`_worker_loop` can rebuild the connection. @@ -597,8 +585,7 @@ async def dispatch_one( # linear pipeline: guard, consume, branch on outcome, f *, writer_conn: "AsyncConnection | None" = None, ) -> None: - """ - Run a single already-leased row through the full consume pipeline. + """Run a single already-leased row through the full consume pipeline. Mirrors the per-row body of ``_worker_loop`` so ``TestOutboxBroker`` can drive the handler synchronously from ``broker.publish``, matching the FastStream @@ -692,8 +679,7 @@ async def _safe_flush( terminal: bool, writer_conn: "AsyncConnection | None", ) -> bool: - """ - Run the terminal/retry write, propagating errors only when ``writer_conn`` is set. + """Run the terminal/retry write, propagating errors only when ``writer_conn`` is set. Returns True iff the write landed (rowcount > 0). False means the lease was lost (a newer fetch reclaimed the row) or — on the test-broker path — the flush raised @@ -719,8 +705,7 @@ async def _safe_flush( return await flush(row, writer_conn=writer_conn) def _emit_lease_lost(self, row: OutboxInnerMessage, *, phase: str) -> None: - """ - Log + record the ``lease_lost`` event shared by the terminal and retry flush paths. + """Log + record the ``lease_lost`` event shared by the terminal and retry flush paths. A terminal/retry write that finds ``rowcount == 0`` means a newer fetch reclaimed the row (its lease expired mid-handler) — the row will be redelivered, so the @@ -832,8 +817,7 @@ async def __aiter__(self) -> AsyncIterator["StreamMessage[OutboxInnerMessage]"]: @typing.override async def consume(self, msg: OutboxInnerMessage) -> typing.Any: - """ - Override to propagate ``_OutboxConfigError`` from programming guards. + """Override to propagate ``_OutboxConfigError`` from programming guards. ``SubscriberUsecase.consume`` swallows all ``Exception`` subclasses except ``StopConsume`` / ``SystemExit``. Programming guards (e.g. the @@ -883,8 +867,7 @@ def _make_response_publisher( @typing.override async def process_message(self, msg: OutboxInnerMessage) -> "Response": # noqa: C901 - """ - Outbox-specific process_message — header propagation (G3) hook. + """Outbox-specific process_message — header propagation (G3) hook. Optionally fills empty Response headers with the inbound message's headers when ``propagate_inbound_headers=True``, and runs the @@ -977,8 +960,7 @@ def _maybe_propagate_inbound_headers( result_msg: "Response", message: typing.Any, ) -> None: - """ - Fill empty Response headers from the inbound message when configured. + """Fill empty Response headers from the inbound message when configured. ``propagate_inbound_headers=True`` carries the inbound row's headers onto a response that didn't set its own. For a chained ``OutboxResponse`` the @@ -1002,8 +984,7 @@ def _reject_outbox_response_with_foreign_publisher( result_msg: "Response", handler: typing.Any, ) -> None: - """ - Refuse the dual-fire combination: OutboxResponse + foreign publisher. + """Refuse the dual-fire combination: OutboxResponse + foreign publisher. OutboxResponse(body=..., queue=..., session=...) writes to the outbox in the caller's transaction; a foreign-publisher decorator also publishes diff --git a/faststream_outbox/testing.py b/faststream_outbox/testing.py index 2b0eded..f6b60d4 100644 --- a/faststream_outbox/testing.py +++ b/faststream_outbox/testing.py @@ -1,5 +1,4 @@ -""" -Test broker with an in-memory ``OutboxClient`` substitute. +"""Test broker with an in-memory ``OutboxClient`` substitute. ``TestOutboxBroker`` wraps an ``OutboxBroker`` and swaps in a ``FakeOutboxClient`` backed by a list of ``_FakeRow`` records. Defaults to **sync dispatch**: ``await broker.publish(...)`` @@ -56,8 +55,7 @@ class _FakeRow: def _claim_fake_row(row: _FakeRow, *, now: _dt.datetime, token: uuid.UUID) -> None: - """ - Set the lease and bump ``deliveries_count`` — the shared claim mechanics (F1-08). + """Set the lease and bump ``deliveries_count`` — the shared claim mechanics (F1-08). Both ``FakeOutboxClient.fetch`` (loop mode) and ``_sync_dispatch`` (sync mode) route through here, so the ``max_deliveries`` boundary is exercised on one path. Eligibility @@ -258,8 +256,7 @@ def _to_inner(row: _FakeRow) -> OutboxInnerMessage: def _find_subscriber_for_queue(broker: OutboxBroker, queue: str) -> "OutboxSubscriber | None": - """ - First matching subscriber wins (deterministic). + """First matching subscriber wins (deterministic). NB: this does NOT mirror production for *overlapping* subscribers — there, multiple subscribers on the same queue compete via ``FOR UPDATE SKIP LOCKED``, so which one @@ -304,8 +301,7 @@ def _emit_published(broker: OutboxBroker, queue: str, *, count: int, size_bytes: def _notify_subscriber(broker: OutboxBroker, queue: str) -> None: - """ - P30: wake the matching subscriber's fetch loop immediately, mirroring production NOTIFY. + """P30: wake the matching subscriber's fetch loop immediately, mirroring production NOTIFY. Only meaningful in loop mode; callers gate on ``run_loops``. Lets loop-mode tests rely on prompt wakeup instead of a tight ``min_fetch_interval`` poll. @@ -352,8 +348,7 @@ async def _fake_publish_many( next_at: "_dt.datetime | None", run_loops: bool, ) -> None: - """ - Shared batch insert for both batch paths (P29). + """Shared batch insert for both batch paths (P29). S5: insert the WHOLE batch, emit ``published``, then dispatch — mirroring production (atomic batch INSERT -> published -> subscriber fetch), so a handler never observes a @@ -376,8 +371,7 @@ async def _fake_publish_many( class FakeOutboxProducer: - """ - In-memory ``OutboxProducer`` substitute routing inserts through ``FakeOutboxClient``. + """In-memory ``OutboxProducer`` substitute routing inserts through ``FakeOutboxClient``. Used by ``TestOutboxBroker`` so ``broker.publisher(queue).publish(body, session=...)`` drives the same in-memory fake store as ``broker.publish(body, session=...)``. The @@ -565,8 +559,7 @@ async def fake_fetch_unprocessed( class TestOutboxBroker(TestBroker[OutboxBroker, OutboxBroker]): # ty: ignore[invalid-type-arguments] - """ - Test harness for ``OutboxBroker``. Two dispatch modes. + """Test harness for ``OutboxBroker``. Two dispatch modes. Default (``run_loops=False``): ``broker.publish`` synchronously drives the matching subscriber's consume pipeline, so handlers run before ``publish`` returns. Matches the diff --git a/pyproject.toml b/pyproject.toml index 24ad9d8..6a82f77 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -72,7 +72,7 @@ ignore = [ "TRY003", # allow long exception messages "EM102", # allow f-string for exception message "D203", # "one-blank-line-before-class" conflicting with D211 - "D212", # "multi-line-summary-second-line" conflicting with D213 + "D213", # "multi-line-summary-second-line" conflicting with D212 "COM812", # flake8-commas "Trailing comma missing" "ISC001", # flake8-implicit-str-concat "G004", # Logging statement uses f-string @@ -93,10 +93,6 @@ max-args = 15 "S101", # allow asserts "PLR2004", # allow magic values ] -# planning/index.py is vendored verbatim from lesnik512/planning-convention and -# linted in that repo's CI; its file-level noqa is tuned for the canonical D212 -# docstring choice, which this repo inverts (D213), so tolerate the unused noqa. -"planning/index.py" = ["RUF100"] [tool.pytest.ini_options] addopts = "--cov=. --cov-report term-missing --cov-fail-under=100" diff --git a/tests/conftest.py b/tests/conftest.py index 6feb704..263256a 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -27,8 +27,7 @@ async def pg_engine() -> AsyncIterator[AsyncEngine]: @pytest.fixture async def outbox_table(pg_engine: AsyncEngine) -> AsyncIterator[Table]: - """ - Per-test outbox table. + """Per-test outbox table. The partial fetch index is declared on the Table itself, so ``create_all`` brings it up alongside the table. diff --git a/tests/test_client_contract.py b/tests/test_client_contract.py index 4ed869b..866ed3d 100644 --- a/tests/test_client_contract.py +++ b/tests/test_client_contract.py @@ -1,5 +1,4 @@ -""" -Cross-adapter contract for ``AbstractOutboxClient``. +"""Cross-adapter contract for ``AbstractOutboxClient``. The two adapters — ``OutboxClient`` (SQL/Postgres) and ``FakeOutboxClient`` (in-memory) — implement the same rules in different substrates: the real client runs diff --git a/tests/test_fake.py b/tests/test_fake.py index 77a2a2a..8054a98 100644 --- a/tests/test_fake.py +++ b/tests/test_fake.py @@ -48,8 +48,7 @@ def _fake_session() -> AsyncMock: - """ - Build an ``AsyncMock(spec=AsyncSession)`` for tests where the session is ignored. + """Build an ``AsyncMock(spec=AsyncSession)`` for tests where the session is ignored. ``OutboxPublishCommand`` requires an ``AsyncSession``; the fake producer doesn't touch it. The mock passes ``isinstance`` and lets publisher tests focus on the @@ -645,8 +644,7 @@ async def handle(body: str) -> None: async def test_loop_mode_feed_wakes_fetch_loop_via_notify() -> None: - """ - P30: feeding a row in loop mode wakes the fetch loop via NOTIFY, not the full poll interval. + """P30: feeding a row in loop mode wakes the fetch loop via NOTIFY, not the full poll interval. With a deliberately slow 30s poll, the handler must still run within 2s — a 15x margin that the poll path provably cannot meet, so only the _notify_event wakeup (the production @@ -1358,8 +1356,7 @@ async def handle(body: str) -> None: async def test_fake_broker_manual_handler_without_ack_is_rejected_via_dispatch() -> None: - """ - T2: a forgetful MANUAL handler (returns, no ack/nack/reject, no exception) is rejected through dispatch. + """T2: a forgetful MANUAL handler (returns, no ack/nack/reject, no exception) is rejected through dispatch. Goes through the full ``broker.publish -> dispatch_one`` path (not a direct ``assert_state_set`` call). Deleting the fallback would emit a false ``acked`` @@ -1525,8 +1522,7 @@ async def handle_next(body: dict) -> None: async def test_propagate_inbound_headers_does_not_poison_cross_content_type_outbox_relay() -> None: - """ - propagate_inbound_headers must not copy content-type onto a chained OutboxResponse. + """propagate_inbound_headers must not copy content-type onto a chained OutboxResponse. Regression for audit F5-01: a ``str`` inbound body (content-type ``text/plain``) had its content-type copied onto an OutboxResponse carrying a ``dict`` @@ -1557,8 +1553,7 @@ async def handle_next(body: dict) -> None: async def test_propagate_inbound_headers_does_not_poison_custom_correlation_id_outbox_relay() -> None: - """ - propagate_inbound_headers must not copy correlation_id onto a chained OutboxResponse. + """propagate_inbound_headers must not copy correlation_id onto a chained OutboxResponse. Regression for audit F5-02: the inbound correlation_id rode along in the propagated headers and conflicted with the handler's explicit @@ -1957,8 +1952,7 @@ def recorder(event: str, tags: typing.Any) -> None: async def test_test_broker_aenter_returns_single_outbox_broker() -> None: - """ - 0.7.1's EnterType binding means TestOutboxBroker yields a single OutboxBroker, not a list/tuple. + """0.7.1's EnterType binding means TestOutboxBroker yields a single OutboxBroker, not a list/tuple. Guards the contract through the upstream typing refactor: even if the base class signature changes again, our single-broker subclass must always hand diff --git a/tests/test_fastapi.py b/tests/test_fastapi.py index 9f3dd01..08e6ef3 100644 --- a/tests/test_fastapi.py +++ b/tests/test_fastapi.py @@ -1,5 +1,4 @@ -""" -Tests for the FastAPI integration (``faststream_outbox.fastapi.OutboxRouter``). +"""Tests for the FastAPI integration (``faststream_outbox.fastapi.OutboxRouter``). Exercises the canonical lifecycle: build a FastAPI app, mount the router, publish through the router's inner broker, and verify subscribers fire. The @@ -78,8 +77,7 @@ async def handle(body: dict) -> None: def test_subscriber_misconfig_warning_attributed_to_user_via_fastapi_router() -> None: - """ - P27: through the FastAPI router (extra frames) the misconfig warning points at the user's call site. + """P27: through the FastAPI router (extra frames) the misconfig warning points at the user's call site. Old static ``stacklevel=4`` landed on a faststream-internal frame on this path; the ``skip_file_prefixes`` attribution lands on the user's ``@router.subscriber(...)`` line. @@ -212,8 +210,7 @@ async def test_outbox_router_publisher_delegates_to_broker() -> None: async def test_fastapi_handler_chains_via_outbox_response_with_per_delivery_session() -> None: - """ - Exercise the transactional contract end-to-end through the FastAPI wrapper. + """Exercise the transactional contract end-to-end through the FastAPI wrapper. The Depends-resolved session flows into the chained OutboxResponse, and each delivery resolves its own fresh session (session-per-delivery). diff --git a/tests/test_integration.py b/tests/test_integration.py index a46c6e7..d478c3e 100644 --- a/tests/test_integration.py +++ b/tests/test_integration.py @@ -67,8 +67,7 @@ async def test_validate_schema_detects_non_partial_index_on_present_index( pg_engine: AsyncEngine, outbox_table: Table, ) -> None: - """ - S2 (review #1): an expected partial index recreated NON-partial breaks ON CONFLICT too, and is caught. + """S2 (review #1): an expected partial index recreated NON-partial breaks ON CONFLICT too, and is caught. Alembic's diff can't distinguish a plain UNIQUE (queue, timer_id) from the partial form (it ignores postgresql_where), and the indpred-NULL row would otherwise be @@ -137,8 +136,7 @@ async def fetch_n(n: int) -> list[int]: async def test_fetch_skips_rows_locked_by_another_transaction(pg_engine, outbox_table) -> None: - """ - F7-04: fetch uses FOR UPDATE SKIP LOCKED — it skips rows another transaction holds, not blocks. + """F7-04: fetch uses FOR UPDATE SKIP LOCKED — it skips rows another transaction holds, not blocks. Seeds 30 rows, locks 20 in an open transaction, then asserts a concurrent fetch promptly claims the disjoint 10. A regression to plain ``FOR UPDATE`` would block on the locked rows @@ -178,8 +176,7 @@ async def test_fetch_skips_rows_locked_by_another_transaction(pg_engine, outbox_ async def test_writer_connection_autocommit_round_trip(pg_engine: AsyncEngine, outbox_table: Table) -> None: - """ - Autocommit-configured writer conn runs ``delete_with_lease`` end-to-end against real Postgres. + """Autocommit-configured writer conn runs ``delete_with_lease`` end-to-end against real Postgres. The connection is configured exactly the way ``_open_worker_resources`` configures the worker writer (``isolation_level="AUTOCOMMIT"``); ``delete_with_lease`` runs with no @@ -315,8 +312,7 @@ async def test_validate_schema_fails_when_columns_missing(pg_engine, outbox_tabl async def test_validate_schema_fails_when_timer_id_unique_index_missing(pg_engine, outbox_table) -> None: - """ - Missing partial unique index on (queue, timer_id) must be caught before runtime. + """Missing partial unique index on (queue, timer_id) must be caught before runtime. Without it, ``publish(timer_id=…)`` raises ``InvalidColumnReference`` on first call. """ @@ -369,8 +365,7 @@ async def test_validate_schema_fails_when_nullability_changed(pg_engine, outbox_ async def test_validate_schema_ignores_user_added_extras(pg_engine, outbox_table) -> None: - """ - Extra columns / indexes the user adds to their outbox table must NOT fail validation. + """Extra columns / indexes the user adds to their outbox table must NOT fail validation. Users may add audit columns or their own indexes; the validator's contract is to flag *missing* schema only, not extras. @@ -401,8 +396,7 @@ async def test_publish_inserts_in_caller_transaction(pg_engine, outbox_table) -> async def test_outbox_response_followon_row_commits_with_handler_transaction(pg_engine, outbox_table) -> None: - """ - F7-02: a returned OutboxResponse's follow-on row commits with the handler's transaction. + """F7-02: a returned OutboxResponse's follow-on row commits with the handler's transaction. The worker loop publishes a returned OutboxResponse through ``OutboxFakePublisher`` (``result_msg.as_publish_command()`` → ``producer.publish`` on the response's own @@ -815,8 +809,7 @@ async def test_fetch_unprocessed_reads_uncommitted_writes_in_same_session(pg_eng async def test_terminal_writes_reuse_writer_conn_under_load(pg_engine, outbox_table) -> None: - """ - M3 — per-worker cached writer conn drains N rows without N pool checkouts. + """M3 — per-worker cached writer conn drains N rows without N pool checkouts. A drain of N rows must trigger O(workers) pool checkouts during the steady-state drain, not one per row (the pre-M3 behavior). With ``max_workers=1`` and one fetch @@ -872,8 +865,7 @@ def _on_checkout(*_args: object) -> None: async def test_concurrent_drain_with_eight_workers_holds_pool_bounded(pg_engine, outbox_table) -> None: - """ - T2 — multi-worker drain: 500 rows + max_workers=8 keeps pool checkouts O(workers). + """T2 — multi-worker drain: 500 rows + max_workers=8 keeps pool checkouts O(workers). The M3 baseline (test above) exercises max_workers=1 (2 steady-state pool connections: 1 fetch + 1 writer). This test raises the bar to max_workers=8 @@ -1242,8 +1234,7 @@ async def handle(body: dict) -> None: async def test_dlq_writes_to_schema_qualified_tables(pg_engine: AsyncEngine) -> None: - """ - B10: with a non-default ``MetaData(schema=...)`` the DLQ CTE must target ``schema.table``. + """B10: with a non-default ``MetaData(schema=...)`` the DLQ CTE must target ``schema.table``. The buggy ``quote(table.name)`` dropped the schema, so the raw DELETE+INSERT CTE referenced a bare ``outbox`` / ``outbox_dlq`` not on the search_path → ``UndefinedTable`` @@ -1386,8 +1377,7 @@ async def test_relay_at_least_once_under_foreign_publish_failure( pg_engine: AsyncEngine, outbox_table: Table, ) -> None: - """ - Assert at-least-once delivery to a foreign broker under simulated publish failure. + """Assert at-least-once delivery to a foreign broker under simulated publish failure. Foreign publish that fails on the first attempt is retried via the outbox's retry_strategy; the row eventually clears after a successful @@ -1452,8 +1442,7 @@ async def test_lease_expiry_during_inflight_handler_redelivers_without_clobber( pg_engine: AsyncEngine, outbox_table: Table, ) -> None: - """ - The lease-token invariant, driven end-to-end through the live worker loop. + """The lease-token invariant, driven end-to-end through the live worker loop. A handler that outlives ``lease_ttl_seconds`` has its row reclaimed and redelivered to a second worker mid-flight. The slow holder's terminal DELETE @@ -1520,8 +1509,7 @@ async def test_relay_dual_fire_guard_through_worker_loop_leaves_row_and_logs( pg_engine: AsyncEngine, outbox_table: Table, ) -> None: - """ - The OutboxResponse + foreign-publisher dual-fire guard, through the live worker loop. + """The OutboxResponse + foreign-publisher dual-fire guard, through the live worker loop. Every existing relay-chain test runs in ``TestOutboxBroker(run_loops=False)``, where the handler runs synchronously inside ``publish`` and the guard raises out @@ -1575,8 +1563,7 @@ async def test_listen_failure_falls_back_to_polling_against_real_postgres( pg_engine: AsyncEngine, outbox_table: Table, ) -> None: - """ - With the LISTEN connection unavailable, delivery still happens via polling. + """With the LISTEN connection unavailable, delivery still happens via polling. Prior coverage stubbed ``asyncpg.connect``/``add_listener`` to fail in unit tests; this drives a real subscriber against live Postgres whose ``_open_listen_connection`` @@ -1607,8 +1594,7 @@ async def test_worker_rebuilds_writer_connection_after_flush_failure( pg_engine: AsyncEngine, outbox_table: Table, ) -> None: - """ - A terminal-write failure poisons the worker's writer connection; it must rebuild and recover. + """A terminal-write failure poisons the worker's writer connection; it must rebuild and recover. Prior coverage only drove this with MagicMock engines (asserting ``connect`` was called twice). Here a real terminal ``delete_with_lease`` raises once against live Postgres, @@ -1672,8 +1658,7 @@ async def test_graceful_timeout_none_still_bounds_drain( pg_engine: AsyncEngine, outbox_table: Table, ) -> None: - """ - ``graceful_timeout=None`` must not hang ``stop()`` on a wedged handler. + """``graceful_timeout=None`` must not hang ``stop()`` on a wedged handler. None stays "unbounded" for ``ping()``, but the drain clamps it to a finite fallback so a single stuck handler can't make ``stop()`` (hence pod shutdown) hang forever. Without @@ -1721,8 +1706,7 @@ async def test_validate_schema_fails_when_lease_check_constraint_missing( async def test_validate_schema_passes_under_ck_naming_convention( pg_engine: AsyncEngine, ) -> None: - """ - A MetaData with a ``ck`` naming convention renames the lease CHECK to ``ck___lease_ck``. + """A MetaData with a ``ck`` naming convention renames the lease CHECK to ``ck___lease_ck``. The probe must look it up under that resolved name (carried on the constraint object), not the literal ``_lease_ck`` — otherwise a perfectly valid schema falsely raises "missing CHECK @@ -1765,8 +1749,7 @@ async def test_validate_schema_fails_when_lease_check_constraint_predicate_wrong async def test_validate_schema_passes_under_ck_convention_with_literally_named_constraint( pg_engine: AsyncEngine, ) -> None: - """ - Convention metadata + a literally-named lease CHECK must validate (the reported case). + """Convention metadata + a literally-named lease CHECK must validate (the reported case). A ``MetaData`` carries a ``ck`` naming convention, but the lease CHECK was created by a hand-written migration under the **literal** ``
_lease_ck`` name (Alembic op functions diff --git a/tests/test_metrics_prometheus.py b/tests/test_metrics_prometheus.py index 9c9712d..bd04037 100644 --- a/tests/test_metrics_prometheus.py +++ b/tests/test_metrics_prometheus.py @@ -147,8 +147,7 @@ def test_prometheus_published_without_count_defaults_to_zero() -> None: def test_prometheus_published_error_status_total_fires_at_count_zero() -> None: - """ - P28: a status="error" published event (count=0) must increment the error-status total. + """P28: a status="error" published event (count=0) must increment the error-status total. The old ``if count > 0`` gate left ``published_messages_total{status="error"}`` — the exact series dashboards alert on — permanently at zero. diff --git a/tests/test_middleware_opentelemetry.py b/tests/test_middleware_opentelemetry.py index e7fbc77..d7d21d5 100644 --- a/tests/test_middleware_opentelemetry.py +++ b/tests/test_middleware_opentelemetry.py @@ -1,5 +1,4 @@ -""" -Tests for the native ``OutboxTelemetryMiddleware`` subclass + its provider. +"""Tests for the native ``OutboxTelemetryMiddleware`` subclass + its provider. End-to-end consume-scope tests drive through ``TestOutboxBroker``. Provider unit tests exercise attribute mapping directly. We use OTel SDK's @@ -208,8 +207,7 @@ def test_outbox_telemetry_provider_publish_destination_name() -> None: async def test_outbox_telemetry_middleware_publish_scope_does_not_fire_under_test_broker() -> None: - """ - ``TestOutboxBroker`` patches ``broker.publish`` directly, bypassing ``_basic_publish``. + """``TestOutboxBroker`` patches ``broker.publish`` directly, bypassing ``_basic_publish``. The middleware's ``publish_scope`` therefore must not fire. The recorder seam (via ``FakeOutboxProducer`` / ``_build_fake_publish``) is the publish-side diff --git a/tests/test_middleware_prometheus.py b/tests/test_middleware_prometheus.py index 07d492f..8d99120 100644 --- a/tests/test_middleware_prometheus.py +++ b/tests/test_middleware_prometheus.py @@ -1,5 +1,4 @@ -""" -Tests for the native ``OutboxPrometheusMiddleware`` subclass + its provider. +"""Tests for the native ``OutboxPrometheusMiddleware`` subclass + its provider. End-to-end consume-scope tests drive through ``TestOutboxBroker``. One smoke test exercises the real ``broker.publish`` path (no test broker patching) so @@ -231,8 +230,7 @@ def _acked_from(reg: CollectorRegistry) -> float: async def test_outbox_prometheus_middleware_publish_scope_does_not_fire_under_test_broker() -> None: - """ - ``TestOutboxBroker`` patches ``broker.publish`` directly, bypassing ``_basic_publish``. + """``TestOutboxBroker`` patches ``broker.publish`` directly, bypassing ``_basic_publish``. The middleware's ``publish_scope`` therefore must not fire. The recorder seam (via ``FakeOutboxProducer`` / ``_build_fake_publish``) is the publish-side diff --git a/tests/test_relay.py b/tests/test_relay.py index e5d4246..e7830e4 100644 --- a/tests/test_relay.py +++ b/tests/test_relay.py @@ -16,8 +16,7 @@ async def test_naked_decorator_chain_relays_plain_return_to_kafka() -> None: - """ - A handler decorated `@kafka_pub @outbox.subscriber(...)` returning plain value. + """A handler decorated `@kafka_pub @outbox.subscriber(...)` returning plain value. This publishes the value through the Kafka publisher chain. """ @@ -41,8 +40,7 @@ async def relay(body: dict[str, Any]) -> dict[str, Any]: async def test_relay_via_kafka_router_publisher() -> None: - """ - Relay via KafkaRouter publisher works like broker-direct publisher. + """Relay via KafkaRouter publisher works like broker-direct publisher. Confirms ConfigComposition.add_config correctly prepends broker config so publisher._outer_config.producer resolves at publish time. @@ -67,8 +65,7 @@ async def relay(body: dict[str, Any]) -> dict[str, Any]: async def test_relay_via_outbox_router_subscriber() -> None: - """ - Relay via OutboxRouter subscriber works like broker-direct subscriber. + """Relay via OutboxRouter subscriber works like broker-direct subscriber. Confirms the symmetric outbox-side router shape with foreign-decorated subscribers from an OutboxRouter the same as broker-direct ones. @@ -93,8 +90,7 @@ async def relay(body: dict[str, Any]) -> dict[str, Any]: async def test_propagate_inbound_headers_true_forwards_outbox_headers_to_kafka() -> None: - """ - With propagate_inbound_headers=True, inbound headers are forwarded to the relay. + """With propagate_inbound_headers=True, inbound headers are forwarded to the relay. The headers are placed onto the Response before the foreign-publisher chain fires. """ @@ -131,8 +127,7 @@ async def capture_publish(cmd: Any, **kwargs: Any) -> Any: async def test_propagate_inbound_headers_false_drops_inbound_headers() -> None: - """ - Default propagate_inbound_headers=False drops inbound headers from the relay. + """Default propagate_inbound_headers=False drops inbound headers from the relay. Response.headers stays empty even when the inbound outbox row carries headers. """ @@ -168,8 +163,7 @@ async def capture_publish(cmd: Any, **kwargs: Any) -> Any: async def test_propagate_inbound_headers_true_does_not_override_explicit_response_headers() -> None: - """ - Even with propagate_inbound_headers=True, explicit user-set Response headers win. + """Even with propagate_inbound_headers=True, explicit user-set Response headers win. The subscriber only fills headers when ``result_msg.headers`` is empty; a handler that returns ``Response(value, headers=...)`` keeps its choice. @@ -207,8 +201,7 @@ async def capture_publish(cmd: Any, **kwargs: Any) -> Any: async def test_outbox_response_with_foreign_publisher_raises() -> None: - """ - A handler that returns OutboxResponse and is decorated by a foreign publisher raises. + """A handler that returns OutboxResponse and is decorated by a foreign publisher raises. The guard fires at dispatch time so the user does not silently dual-fire (row in outbox + Kafka publish). @@ -230,8 +223,7 @@ async def relay(body: dict[str, Any]) -> OutboxResponse: async def test_outbox_response_with_foreign_publisher_fires_guard_before_side_effects() -> None: - """ - T5: the dual-fire guard must raise BEFORE the publisher chain runs. + """T5: the dual-fire guard must raise BEFORE the publisher chain runs. Moving the guard below the chain still raises (so ``pytest.raises`` passes), but the Kafka publish AND the follow-on outbox insert have already happened by then. Pin the @@ -259,8 +251,7 @@ async def relay(body: dict[str, Any]) -> OutboxResponse: async def test_unstarted_foreign_broker_warns_on_start(caplog: pytest.LogCaptureFixture) -> None: - """ - Log one WARNING per unstarted foreign broker at start() time. + """Log one WARNING per unstarted foreign broker at start() time. If a foreign-publisher decorator is on an outbox subscriber but the foreign broker has not been started, start(broker_outbox) logs a single diff --git a/tests/test_unit.py b/tests/test_unit.py index 1be652a..c6676e7 100644 --- a/tests/test_unit.py +++ b/tests/test_unit.py @@ -82,8 +82,7 @@ def _make_broker(engine: object | None = None, table_name: str = "outbox") -> Ou def _make_session_mock(*, scalar_return: object = 42) -> AsyncMock: - """ - Build an AsyncSession mock whose ``execute()`` returns a sync MagicMock. + """Build an AsyncSession mock whose ``execute()`` returns a sync MagicMock. AsyncMock(spec=AsyncSession) makes the return_value of execute() default to an AsyncMock — so ``result.scalar()`` would itself return a coroutine. The broker's @@ -1846,8 +1845,7 @@ async def test_open_listen_connection_returns_none_when_asyncpg_connect_fails() async def test_open_listen_connection_passes_multihost_kwargs_to_asyncpg() -> None: - """ - Multi-host URLs must reach asyncpg as host/port lists, not a re-rendered DSN. + """Multi-host URLs must reach asyncpg as host/port lists, not a re-rendered DSN. ``?host=h1:5432&host=h2:5432`` renders back as URL-encoded host tokens asyncpg can't parse. SQLAlchemy-only kwargs (``prepared_statement_cache_size``, ...) must @@ -1948,8 +1946,7 @@ async def _hang(*_args: object) -> None: def _make_broker_for_dispatch(fake: FakeOutboxClient) -> tuple[OutboxBroker, TestOutboxBroker]: - """ - Build a broker + TestOutboxBroker harness so logger / config wiring is initialized. + """Build a broker + TestOutboxBroker harness so logger / config wiring is initialized. The caller enters the harness via ``async with`` to run dispatch_one against a real subscriber instance whose ``_outer_config.logger`` is wired by FastStream's lifecycle. @@ -2063,8 +2060,7 @@ async def test_flush_retry_threads_writer_conn_into_mark_pending(writer_conn: ob async def test_dispatch_one_outer_except_swallows_consume_failure() -> None: - """ - The defensive outer except in dispatch_one catches consume/assert_state_set bugs. + """The defensive outer except in dispatch_one catches consume/assert_state_set bugs. Handler errors are normally caught by AckPolicy middleware. The outer except is the safety net for middleware-bypassing failures — patch ``consume`` directly to exercise it. @@ -2085,8 +2081,7 @@ async def _boom(_row: object) -> None: async def test_dispatch_one_preserves_row_when_consume_early_exits_on_shutdown() -> None: - """ - Shutdown race in dispatch_one. + """Shutdown race in dispatch_one. ``SubscriberUsecase.consume()`` returns ``None`` without invoking ``process_message`` when ``running`` is False. Previously ``dispatch_one`` fell @@ -2121,8 +2116,7 @@ async def handle(body: dict) -> None: ... async def test_dispatch_one_preserves_row_when_consume_raises() -> None: - """ - T3: a consume()-escaping exception preserves the row (no DELETE, no false ack/nack). + """T3: a consume()-escaping exception preserves the row (no DELETE, no false ack/nack). ``consume()`` swallows ordinary handler exceptions, but a middleware-bypassing failure (or a framework bug) can escape it. ``dispatch_one`` catches that, logs, @@ -2163,8 +2157,7 @@ async def handle(body: dict) -> None: ... async def test_dispatch_one_lease_lost_emits_only_lease_lost_not_acked() -> None: - """ - P17: a lease-lost terminal flush emits ``lease_lost`` only — not a false ``acked`` that double-counts. + """P17: a lease-lost terminal flush emits ``lease_lost`` only — not a false ``acked`` that double-counts. Before P17 the acked/nacked metric fired before the flush, so a row whose lease was reclaimed (flush rowcount 0 → redelivered) was counted once here and again on redelivery. @@ -2194,8 +2187,7 @@ async def handle(body: dict) -> None: ... async def test_worker_inner_swallows_config_error_without_reconnect() -> None: - """ - P18: an _OutboxConfigError in the worker loop is logged and swallowed, not propagated. + """P18: an _OutboxConfigError in the worker loop is logged and swallowed, not propagated. Letting it reach _run_with_reconnect would tear down the writer connection and back off (throttling unrelated rows). The worker must continue; the row's lease expires. @@ -2222,8 +2214,7 @@ async def _raise_then_stop(row: object, *, writer_conn: object) -> None: async def test_dispatch_one_max_deliveries_emits_terminal_without_dispatched() -> None: - """ - T7: the max_deliveries terminal emits nacked_terminal(reason=max_deliveries) with NO preceding 'dispatched'. + """T7: the max_deliveries terminal emits nacked_terminal(reason=max_deliveries) with NO preceding 'dispatched'. The handler never runs (``allow_delivery`` short-circuits), so ``dispatched`` — which carries the in-process gauge's ``.inc()`` — must not fire. The Prometheus adapter tests @@ -2705,8 +2696,7 @@ async def test_outbox_client_mark_pending_with_lease_uses_caller_conn() -> None: async def test_fetch_inner_does_not_claim_during_drain() -> None: - """ - T4: with _stopping set, _fetch_inner must claim NO rows (the drain "no new claims" invariant). + """T4: with _stopping set, _fetch_inner must claim NO rows (the drain "no new claims" invariant). Goes through the real loop guard. Changing it to ``while self.running:`` (dropping the ``_stopping`` conjunct) would keep claiming rows during drain — here the fetch @@ -2811,8 +2801,7 @@ def test_linear_retry_without_jitter_is_deterministic() -> None: def _register_subscriber(broker: OutboxBroker, **subscriber_kwargs: object) -> None: - """ - Trigger registration-time misconfig validation; no handler needed. + """Trigger registration-time misconfig validation; no handler needed. ``_validate_subscriber_config`` runs inside ``create_subscriber``, before ``add_call``, so the warning/error fires from the ``broker.subscriber(...)`` @@ -3325,8 +3314,7 @@ async def test_delete_with_lease_raises_when_dlq_payload_but_no_dlq_table() -> N async def test_fetch_cte_carries_partial_index_predicates_as_conjuncts() -> None: - """ - T6: each OR arm of the fetch WHERE must carry its partial-index predicate as a conjunct. + """T6: each OR arm of the fetch WHERE must carry its partial-index predicate as a conjunct. Branch A is ``acquired_token IS NULL``; Branch B is ``acquired_token IS NOT NULL AND acquired_at < cutoff``. The naive single-OR form (``acquired_at < cutoff`` without the @@ -3652,8 +3640,7 @@ async def test_flush_terminal_dlq_payload_short_exception_not_truncated() -> Non async def test_metrics_manual_reject_without_exception_emits_nacked_terminal_rejected() -> None: - """ - Manual ``msg.reject()`` (no exception raised) emits ``nacked_terminal(reason="rejected")``. + """Manual ``msg.reject()`` (no exception raised) emits ``nacked_terminal(reason="rejected")``. Previously routed to ``acked`` because ``last_exception is None`` was checked first. """ @@ -3679,8 +3666,7 @@ async def handle(body: dict, msg: AnnotatedOutboxMessage) -> None: async def test_metrics_reject_on_error_terminal_emits_reason_rejected() -> None: - """ - REJECT_ON_ERROR + handler raise emits ``reason="rejected"`` (was ``"retry_terminal"``). + """REJECT_ON_ERROR + handler raise emits ``reason="rejected"`` (was ``"retry_terminal"``). The metric branch previously hardcoded ``"retry_terminal"`` for the post-handler terminal path; it now reads ``terminal_failure_reason`` and includes ``exception_type``. @@ -3721,8 +3707,7 @@ def test_default_retry_strategy_pins_documented_parameters() -> None: async def test_dlq_cte_insert_columns_match_make_dlq_table() -> None: - """ - Guard the hardcoded DLQ INSERT column list against drift from ``make_dlq_table``. + """Guard the hardcoded DLQ INSERT column list against drift from ``make_dlq_table``. ``_build_dlq_cte_stmt`` hardcodes the DLQ column list as an f-string with nothing linking it to the table definition (audit 2026-06-14). A future NOT-NULL-without- @@ -3813,8 +3798,7 @@ async def test_fetch_unprocessed_rejects_non_positive_limit() -> None: def test_asyncapi_document_populates_channels_and_operations() -> None: - """ - Regression: ``BrokerSpec(url=[])`` produced a structurally empty AsyncAPI document. + """Regression: ``BrokerSpec(url=[])`` produced a structurally empty AsyncAPI document. Upstream's generator only emits channels/operations for brokers with a non-empty spec url; with ``url=[]`` the assembled doc had ``servers={} channels={} operations={}`` even