DX-2809: step-level flow control and retries via context.run().withSettings()

[CahidArda]

Summary

Adds step-level settings to workflow steps. Until now, flow control and retries could only be set when triggering the run (client.trigger), and QStash applied them to every request of the run. This PR lets a single step override them:

const result = await context
  .run("charge", () => chargeCustomer())
  .withSettings({
    flowControl: { key: "payment-provider", parallelism: 3 },
    retries: 5,
    retryDelay: "1000",
    timeout: "30s",
  });

withSettings must be chained synchronously on the context.run call. The settings override the trigger-level configuration for that step only; requests carrying them signal QStash with a WF_StepConfig feature flag so they aren't clobbered by the run's trigger configuration (server support shipped separately).

Architecture

A step's settings must ride on the request whose delivery executes that step. That request is created at different times depending on the previous step, so there are three mechanisms:

1. Deferred submission (step after run/sleep/sleepUntil)

Previously the auto executor executed a step, submitted its result and immediately threw WorkflowAbort. Now (for steps whose result is available in-process) it executes the step, keeps the result as a pending submission and resolves the step's promise with the result, letting the workflow function continue. The next context.* call reveals the next step — branching on the previous result works because the real result was returned (after a JSON round-trip, so the value is identical to what later replays observe). The pending result is then submitted with the next step's settings attached, and the usual abort is thrown.

If the workflow function returns or throws before another step appears, triggerRouteFunction flushes the pending submission first. An error thrown after a step executed is swallowed for that invocation (it re-occurs deterministically on replay), guaranteeing a step never executes twice.

Parallel steps carry their own settings on their plan steps, since a plan step's delivery is what executes its target step.

2. Discovery on the call callback (step after context.call)

The result of a context.call is produced by QStash, so the SDK can't continue past the step at submission time. When the callback arrives, the SDK fetches the run's steps (getSteps), then runs the route function in a new discovery mode: memoized steps (including the fresh call result) replay as usual, and on reaching the first un-executed step the executor throws an internal WorkflowDiscoveryAbort carrying that step's settings — without executing anything. The discovered settings are attached to the call-result submission the SDK performs anyway. Discovery is best-effort: any failure falls back to submitting without settings.

3. Discovery + hidden redelivery (step after context.invoke)

The invoke result step is published by QStash directly, so the SDK marks the invoke submission with a forwarded header (Upstash-Workflow-Invoke-Result: true) which comes back on the delivery carrying the invoke result. Seeing the marker, serve runs the discovery replay using the steps already in the delivery (no extra fetch):

• next step has no settings → the step executes in the same delivery; zero overhead.
• next step has settings → instead of executing the step in the current (ungated) delivery, the SDK publishes a small helper request with the settings and the discovery call type, and returns. QStash delivers it gated by the step-level flow control, and that delivery executes the step. QStash doesn't treat the helper request as a step and hides it from the step logs. The redelivery isn't marked, so there is no loop; its body carries the target step index so content-based deduplication doesn't collapse redeliveries of different steps.

Limitations (fall back to trigger-level settings)

• the first step of a run (the trigger request executes it)
• the step right after waitForEvent/waitForWebhook (could get the same marker treatment as invoke later)
• the step right after a parallel group

These are documented on the StepSettings type.

Changes

• context.run returns a RunStepPromise with the chainable withSettings method; settings are stored on the lazy step (StepSettings type: flowControl, retries, retryDelay, timeout).
• AutoExecutor: pending-submission bookkeeping and flush, plus a discoveryMode which throws WorkflowDiscoveryAbort instead of executing fresh steps.
• submit-steps.ts split into executeStep / submitStepResult; settings headers (Upstash-Flow-Control-Key/Value, Upstash-Retries, Upstash-Retry-Delay, Upstash-Timeout + WF_StepConfig feature) overlaid on submissions and parallel plan steps.
• handleThirdPartyCallResult: fetches the run steps once and runs next-step discovery before republishing the call result.
• serve: invoke-result marker handling and publishStepSettingsRedelivery (new src/serve/discovery.ts).
• Refactors to avoid circular imports: raw step parsing extracted to src/raw-steps.ts, recreateUserHeaders moved to utils.ts.
• Logs stay unchanged — step-level values surface through the existing BaseStepLog fields.

Requires the matching qstash-server change (same ticket) which makes QStash keep a message's own flow control/retries when the WF_StepConfig feature is present, and hides discovery call-type requests from step logs.

Testing

• Unit tests: settings headers on deferred submissions and parallel plan steps, branching on a fresh step result, discovery-settings attachment on call-result republish, flush semantics in triggerRouteFunction. bun test src: 394 pass.
• New live CI routes under examples/ci/app/test-routes/flow-control/ (step, call, invoke — see its README): each has an increment step with step-level parallelism: 1 which increments a shared Redis counter, holds it ~3s and decrements it. The test triggers 3 concurrent runs with permissive trigger-level flow control (parallelism: 5); every run asserts it never observed more than 1 concurrent execution. Verified against a local qstash-server: increments were serialized ~3.5s apart in all three variants, and negative controls (removing .withSettings) make the tests fail.
• Full CI suite: 29/29 pass. Run logs verified: step-level values visible via BaseStepLog, discovery redeliveries absent from step logs.

[linear-code]

DX-2809