WIP: Feat/fairness scheduler

[chall37]

This is a non-feature gated implementation of the entire pipeline. I still need to shake down some issues, but I think the architecture is in place. I'm still working on the feature-gated parallel implementation as described in #567 and #568 but it's a heavy lift. In the meantime, this can serve as a point of direct comparison between legacy and proposed architectural changes.

Fairness Scheduler PR - Changes Summary

This document summarizes changes in this PR compared to gnachman:master.

Problem Statement

iTerm2 has two starvation points that cause one high-throughput session (e.g., running yes) to block other sessions:

1. Read-level starvation (TaskNotifier thread): A single-threaded select() loop monitors all PTY file descriptors. When TokenExecutor.addTokens() blocks on semaphore.wait(), the entire TaskNotifier thread is blocked, preventing ALL other sessions from being read.

2. Execution-level starvation (mutation queue): All sessions share a single mutation queue. Whichever session's execute() runs first processes all its tokens with no mechanism ensuring sessions take turns.

Solution Overview

Replace blocking I/O with event-driven dispatch sources and add a round-robin fairness scheduler:

1. Per-PTY dispatch sources - Decouples reading so one session can't block others
2. Non-blocking backpressure - Replace semaphore blocking with suspend/resume
3. FairnessScheduler - Coordinates round-robin execution on the mutation queue
4. Token budget per turn - Execute groups until ~500 tokens consumed, then yield

Discussion (continued from PR #560):

"Fairness means that each PTY has some of its tokens executed after waiting for other PTYs to have no more than one turn getting their tokens executed."

"A round-robin scheduler 'busy list' that tracks a list of PTYs that have tokens to execute provides exactly that."

"This plays naturally into using dispatch_source for reading rather than select()."

• Busy list

Implemented as described. Sessions are added to the tail when they have work, removed from the head when granted a turn, and moved to the tail if work remains after the turn.

• Token budget

The gist pseudocode used a fixed chunk size. This implementation uses a ~500-token budget checked at group boundaries, ensuring atomic group execution while providing approximate fairness. (This is a lever we can adjust, if necessary.)

• Read suspension

Currently, TokenExecutor.addTokens blocks on a semaphore to backpressure PTY reads. In this implementation, the blocking wait is replaced with per-PTY reads gated by dispatch sources in PTYTask (the read source starts suspended; updateReadSourceState resumes or suspends it).

• Per-PTY vs global backpressure: A global capacity counter gating admission across all producers was suggested in Add backpressure counter for monitoring token queue load #560, plus per-producer thresholds for self-throttling. This implementation uses per-PTY only—each TokenExecutor tracks its own availableSlots with no global counter. This should be sufficient for isolation—when Session A is full, only A suspends while B and C continue reading, and it maps directly to per-PTY dispatch sources without requiring scheduler coordination. With the current per-session limit of 64 TokenArrays, this seems acceptable, though global backpressure could be added later if memory pressure becomes a concern.

  • Implementation cost

    • New plumbing: BackpressureLevel / availableSlots, consumption callbacks in TokenExecutor, and backpressureReleaseHandler wiring in PTYSession.
    • Added dispatch-source read/write state management in PTYTask to suspend and resume I/O based on backpressureLevel, paused, and ioAllowed.

  • Alternatives considered

    • Global capacity counter: Add a shared counter across all sessions. Suspend all reads when total queued tokens exceed a system-wide limit. Provides tighter memory bounds under heavy multi-session load at the cost of reduced isolation and greater overhead.
    • Acceptance-based resumption: Resume reading when the scheduler grants a turn rather than after consumption. Simpler signaling (scheduler → PTYTask directly), but potentially more complicated coordination, such as preemptive slot reservation to avoid enqueueing into a full queue at turn-start.
    • Scheduler-driven suspension: More centralized control, but adds coupling between scheduler and I/O, blurring SoC.
    • No backpressure: Remove read-side throttling entirely; let queues grow unbounded. Not viable without other memory limits.

• Coprocess handling: TaskNotifier is retained in a reduced role for coprocess FD handling and process lifecycle management. Full elimination of TaskNotifier is deferred to a future phase to limit scope and risk.

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                         READING (Decoupled)                         │
│                                                                     │
│   PTYTask A              PTYTask B              PTYTask C           │
│   dispatch_source        dispatch_source        dispatch_source     │
│        │                      │                      │              │
│        ▼                      ▼                      ▼              │
│   TokenExecutor A        TokenExecutor B        TokenExecutor C     │
│   (suspend/resume)       (suspend/resume)       (suspend/resume)    │
└────────┼──────────────────────┼──────────────────────┼──────────────┘
         │                      │                      │
         └──────────────────────┼──────────────────────┘
                                ▼
┌─────────────────────────────────────────────────────────────────────┐
│                 FAIRNESS SCHEDULER (Gatekeeper)                     │
│                                                                     │
│   Busy List: [A] → [B] → [C] → ...  (round-robin)                   │
│   Grants permission: "Execute groups until ~500 tokens"             │
└─────────────────────────────────────────────────────────────────────┘

Key Changes by Component

1. FairnessScheduler (NEW)

File: sources/FairnessScheduler.swift

Round-robin scheduler that controls which session executes on the mutation queue. Does NOT buffer tokens - only controls execution order. Each session gets a turn with a ~500 token budget before yielding.

Justification: Provides deterministic fairness guarantees. No session can monopolize execution regardless of output rate.

2. PTYTask Dispatch Sources

Files: sources/PTYTask.h, sources/PTYTask.m

Adds per-PTY dispatch_source_t for read and write operations:

• _readSource - DISPATCH_SOURCE_TYPE_READ
• _writeSource - DISPATCH_SOURCE_TYPE_WRITE
• Unified state predicates (shouldRead/shouldWrite)
• State update methods (updateReadSourceState/updateWriteSourceState)

Justification: Decouples PTY I/O from the shared TaskNotifier thread. One session's backpressure cannot block another session's reads.

3. TokenExecutor Modifications

File: sources/TokenExecutor.swift

• Removes semaphore blocking - addTokens() never blocks
• Adds executeTurn() - Fairness-limited execution with token budget
• Adds backpressureReleaseHandler - Closure to signal PTYTask when backpressure clears
• Removes activeSessionsWithTokens - Legacy foreground preemption no longer needed

Justification: Non-blocking token admission is required for dispatch sources (handlers must return quickly). The fairness scheduler replaces the ad-hoc foreground preemption.

4. TaskNotifier Changes

Files: sources/TaskNotifier.h, sources/TaskNotifier.m

• PTY tasks with dispatch sources skip select() for their FDs
• Still handles: coprocess FDs, deadpool/waitpid, unblock pipe
• New useDispatchSource protocol method (@optional)

Justification: Hybrid approach allows incremental migration. Coprocesses remain on select() for now (rare feature, low risk).

5. VT100ScreenMutableState Integration

File: sources/VT100ScreenMutableState.m

• Registers TokenExecutor with FairnessScheduler on init
• Unregisters on setEnabled:NO
• Stores _fairnessSessionId for stable identification

Justification: Lifecycle management ensures sessions are properly tracked by the scheduler.

6. TwoTierTokenQueue

File: sources/TwoTierTokenQueue.swift

• Adds discardAllAndReturnCount() for cleanup accounting

Justification: Prevents availableSlots drift when sessions close with unconsumed tokens.

7. PTYSession Wiring

File: sources/PTYSession.m

• Wires backpressureReleaseHandler to updateReadSourceState
• Adds re-kick calls via scheduleTokenExecution for state transitions

Justification: Connects TokenExecutor backpressure signals to PTYTask dispatch source control.

Location: ModernTests/FairnessScheduler/

Test Methods organized by component:

Test File	Coverage Area
FairnessSchedulerTests.swift	Round-robin scheduling, busy list management, session lifecycle, edge cases (empty list, single session, rapid add/remove)
TokenExecutorFairnessTests.swift	Token budget enforcement, executeTurn() behavior, backpressure handler wiring, accounting invariants, high-priority token handling
PTYTaskDispatchSourcePredicateTests.swift	shouldRead/shouldWrite predicate logic under all condition combinations (paused, ioAllowed, backpressure, writeBuffer)
PTYTaskDispatchSourceHandlerTests.swift	Read/write event handler behavior, coprocess data bridging, error handling
PTYTaskDispatchSourceBasicTests.swift	Dispatch source lifecycle (setup, teardown, resume-before-cancel)
PTYTaskDispatchSourceIntegrationTests.swift	End-to-end PTYTask I/O flow with real file descriptors
TaskNotifierDispatchSourceTests.swift	Hybrid mode (dispatch source tasks skip select), coprocess FD handling preserved
TwoTierTokenQueueTests.swift	discardAllAndReturnCount(), queue accounting
IntegrationTests.swift	Full system integration: multiple sessions, fairness under load, session close with pending tokens
PTYSessionWiringTests.swift	Backpressure handler wiring, re-kick on state transitions

Mock/utility infrastructure: MockTokenExecutorDelegate, MockPTYTaskDelegate, EnqueuingPTYTaskDelegate, SpyVT100Screen, and test utilities.

Key invariants:

• availableSlots accounting (no drift after enqueue/consume cycles)
• Round-robin ordering (each session permitted exactly one turn before others)
• Dispatch source state consistency (suspend count never goes negative)
• Graceful handling of session close with unconsumed tokens

Files Unchanged

Component	Why Unchanged
TwoTierTokenQueue (mostly)	Per-session high-priority handling already correct
High-priority token paths	Triggers, API injection work correctly within session turns
Side effects system	Unrelated to fairness (30fps cadence on main queue)
Mutation queue	Still single-threaded; fairness controls who uses it

[chall37]

Implemented behind feature flag in #580.