Keep background Browser WebViews automatable#6027
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
📝 WalkthroughWalkthroughThis PR implements automation command leases for offscreen WebView hosting during browser automation. A new ChangesAutomation Command Lease for Offscreen Hosting
Sequence DiagramsequenceDiagram
participant TerminalController
participant BrowserPanel
participant BrowserHiddenWebViewDiscardManager
participant OffscreenRenderHostLease
participant ScreenshotSnapshotter
TerminalController->>BrowserPanel: beginAutomationCommandLease("browser.socketCommand")
BrowserPanel->>BrowserPanel: increment activeVisualAutomationCaptureCount
BrowserPanel->>BrowserPanel: cancel hiddenWebViewDiscard
BrowserPanel->>OffscreenRenderHostLease: create lease (if offscreen enabled)
BrowserPanel-->>TerminalController: return lease
TerminalController->>ScreenshotSnapshotter: capture with offscreen render
ScreenshotSnapshotter->>OffscreenRenderHostLease: use lease for hosting
ScreenshotSnapshotter-->>TerminalController: return screenshot
TerminalController->>BrowserPanel: endAutomationCommandLease(lease, "browser.socketCommand")
BrowserPanel->>OffscreenRenderHostLease: end() - restore WebView
BrowserPanel->>BrowserPanel: decrement activeVisualAutomationCaptureCount
BrowserPanel->>BrowserPanel: update webViewLastAutomationActivityAt
BrowserPanel->>BrowserHiddenWebViewDiscardManager: scheduleIfNeeded (if count == 0)
BrowserHiddenWebViewDiscardManager->>BrowserPanel: read hiddenWebViewDiscardLastAutomationActivityAt
BrowserHiddenWebViewDiscardManager->>BrowserHiddenWebViewDiscardManager: compute effectiveHiddenAt (max of hidden, wake, automation activity)
BrowserHiddenWebViewDiscardManager-->>BrowserPanel: schedule discard if needed
🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 20 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (20 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@Sources/Panels/BrowserScreenshotSnapshotter.swift`:
- Around line 63-184: Move the OffscreenRenderHostLease class (and the
BrowserScreenshotOffscreenRenderPanel type if it’s declared in the same block)
out of BrowserScreenshotSnapshotter.swift into a new file under Sources/Panels
(e.g., OffscreenRenderHostLease.swift); keep the `@MainActor` attribute, any
imports, and the same access level, and ensure the new file declares the same
types (OffscreenRenderHostLease and BrowserScreenshotOffscreenRenderPanel) so
existing callers (e.g., BrowserScreenshotWebViewSnapshotter.restoreWebView and
any usages in BrowserScreenshotSnapshotter) still compile; after moving, remove
the nested/inline definition from BrowserScreenshotSnapshotter.swift and run a
build to fix any missing import/access issues or to adjust visibility if
necessary.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 1efa4673-b211-4e5d-9981-2bd079346d37
📒 Files selected for processing (4)
Sources/Panels/BrowserPanel.swiftSources/Panels/BrowserScreenshotSnapshotter.swiftSources/TerminalController.swiftcmuxTests/BrowserPanelTests.swift
👮 Files not reviewed due to content moderation or server errors (1)
- Sources/TerminalController.swift
| @MainActor | ||
| final class OffscreenRenderHostLease { | ||
| private weak var webView: WKWebView? | ||
| private let previousSuperview: NSView? | ||
| private let previousFrame: NSRect | ||
| private let previousBounds: NSRect | ||
| private let previousAutoresizingMask: NSView.AutoresizingMask | ||
| private let previousTranslatesAutoresizingMaskIntoConstraints: Bool | ||
| private let restoreAnchor: NSView? | ||
| private let restorePosition: NSWindow.OrderingMode | ||
| private let window: BrowserScreenshotOffscreenRenderPanel | ||
| private var isActive = true | ||
|
|
||
| init(webView: WKWebView, viewportSize: NSSize) { | ||
| self.webView = webView | ||
| previousSuperview = webView.superview | ||
| let previousSubviews = previousSuperview?.subviews ?? [] | ||
| let previousIndex = previousSubviews.firstIndex(of: webView) | ||
| previousFrame = webView.frame | ||
| previousBounds = webView.bounds | ||
| previousAutoresizingMask = webView.autoresizingMask | ||
| previousTranslatesAutoresizingMaskIntoConstraints = webView.translatesAutoresizingMaskIntoConstraints | ||
|
|
||
| if let previousIndex, previousIndex > 0 { | ||
| restoreAnchor = previousSubviews[previousIndex - 1] | ||
| restorePosition = .above | ||
| } else if let previousIndex, previousIndex == 0, previousSubviews.count > 1 { | ||
| restoreAnchor = previousSubviews[1] | ||
| restorePosition = .below | ||
| } else { | ||
| restoreAnchor = nil | ||
| restorePosition = .above | ||
| } | ||
|
|
||
| let normalizedSize = Self.normalizedViewportSize(viewportSize) | ||
| let frame = NSRect( | ||
| x: -100_000 - normalizedSize.width, | ||
| y: -100_000 - normalizedSize.height, | ||
| width: normalizedSize.width, | ||
| height: normalizedSize.height | ||
| ) | ||
| let window = BrowserScreenshotOffscreenRenderPanel( | ||
| contentRect: frame, | ||
| styleMask: [.borderless, .nonactivatingPanel], | ||
| backing: .buffered, | ||
| defer: false | ||
| ) | ||
| window.isReleasedWhenClosed = false | ||
| window.identifier = NSUserInterfaceItemIdentifier("cmux.browserVisualAutomationRender") | ||
| window.hasShadow = false | ||
| window.isOpaque = false | ||
| window.backgroundColor = .clear | ||
| window.alphaValue = 0.01 | ||
| window.ignoresMouseEvents = true | ||
| window.hidesOnDeactivate = false | ||
| window.collectionBehavior = [.transient, .ignoresCycle, .stationary, .canJoinAllSpaces] | ||
| window.isExcludedFromWindowsMenu = true | ||
| self.window = window | ||
|
|
||
| let contentView = NSView(frame: NSRect(origin: .zero, size: normalizedSize)) | ||
| contentView.wantsLayer = true | ||
| webView.removeFromSuperview() | ||
| webView.frame = contentView.bounds | ||
| webView.autoresizingMask = [.width, .height] | ||
| contentView.addSubview(webView) | ||
| window.contentView = contentView | ||
| window.orderFrontRegardless() | ||
| } | ||
|
|
||
| func end() { | ||
| guard isActive else { return } | ||
| isActive = false | ||
| if let webView { | ||
| Self.restoreWebView( | ||
| webView, | ||
| to: previousSuperview, | ||
| frame: previousFrame, | ||
| bounds: previousBounds, | ||
| autoresizingMask: previousAutoresizingMask, | ||
| translatesAutoresizingMaskIntoConstraints: previousTranslatesAutoresizingMaskIntoConstraints, | ||
| anchor: restoreAnchor, | ||
| position: restorePosition | ||
| ) | ||
| } | ||
| window.orderOut(nil) | ||
| window.contentView = nil | ||
| window.close() | ||
| } | ||
|
|
||
| deinit { | ||
| guard isActive else { return } | ||
| MainActor.assumeIsolated { | ||
| end() | ||
| } | ||
| } | ||
|
|
||
| private static func normalizedViewportSize(_ viewportSize: NSSize) -> NSSize { | ||
| BrowserScreenshotWebViewSnapshotter.normalizedViewportSize(viewportSize) | ||
| } | ||
|
|
||
| private static func restoreWebView( | ||
| _ webView: WKWebView, | ||
| to superview: NSView?, | ||
| frame: NSRect, | ||
| bounds: NSRect, | ||
| autoresizingMask: NSView.AutoresizingMask, | ||
| translatesAutoresizingMaskIntoConstraints: Bool, | ||
| anchor: NSView?, | ||
| position: NSWindow.OrderingMode | ||
| ) { | ||
| BrowserScreenshotWebViewSnapshotter.restoreWebView( | ||
| webView, | ||
| to: superview, | ||
| frame: frame, | ||
| bounds: bounds, | ||
| autoresizingMask: autoresizingMask, | ||
| translatesAutoresizingMaskIntoConstraints: translatesAutoresizingMaskIntoConstraints, | ||
| anchor: anchor, | ||
| position: position | ||
| ) | ||
| } | ||
| } |
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major | ⚡ Quick win
Extract the lease/panel helpers to a dedicated file to stay within the Swift file-size budget.
This new lease abstraction is coherent, but adding it here pushes Sources/Panels/BrowserScreenshotSnapshotter.swift past the 800-line budget. Please move OffscreenRenderHostLease (and, if appropriate, BrowserScreenshotOffscreenRenderPanel) into a focused companion file under Sources/Panels/.
As per coding guidelines, {Sources,CLI,Packages,cmuxTests,cmuxUITests}/**/*.swift should be flagged when a production Swift file exceeds 800 lines, even with coherent responsibility.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@Sources/Panels/BrowserScreenshotSnapshotter.swift` around lines 63 - 184,
Move the OffscreenRenderHostLease class (and the
BrowserScreenshotOffscreenRenderPanel type if it’s declared in the same block)
out of BrowserScreenshotSnapshotter.swift into a new file under Sources/Panels
(e.g., OffscreenRenderHostLease.swift); keep the `@MainActor` attribute, any
imports, and the same access level, and ensure the new file declares the same
types (OffscreenRenderHostLease and BrowserScreenshotOffscreenRenderPanel) so
existing callers (e.g., BrowserScreenshotWebViewSnapshotter.restoreWebView and
any usages in BrowserScreenshotSnapshotter) still compile; after moving, remove
the nested/inline definition from BrowserScreenshotSnapshotter.swift and run a
build to fix any missing import/access issues or to adjust visibility if
necessary.
Source: Coding guidelines
Greptile SummaryThis PR makes background and portal-hidden browser WebViews automatable by wrapping every socket browser command in a
Confidence Score: 4/5The core accounting logic (counter increment/decrement, discard cancellation/rescheduling, timestamp restart) is correct and well-tested; the main risks—noted in prior review threads—centre on OffscreenRenderHostLease teardown and concurrent lease creation on the same panel. The new OffscreenRenderHostLease class and the beginAutomationCommandLease/endAutomationCommandLease pair handle the sequential case cleanly: the counter is always balanced, effectiveHiddenAt correctly takes lastAutomationActivityAt as an additional lower bound, and tests cover the portal-hosted, discarded, and offscreen-only paths. The concerns already raised about deinit/assumeIsolated and concurrent same-panel leases remain in the code and represent real edge-case risks for WebKit process stability. Sources/Panels/BrowserScreenshotSnapshotter.swift (OffscreenRenderHostLease deinit path) and Sources/TerminalController.swift (concurrent v2BrowserWithPanelContext calls for the same hidden panel) Important Files Changed
Sequence DiagramsequenceDiagram
participant W as Worker Thread
participant M as Main Actor
participant BP as BrowserPanel
participant L as OffscreenRenderHostLease
participant DM as HiddenWebViewDiscardManager
W->>M: v2MainSync beginAutomationCommandLease
activate M
M->>BP: "webViewLastAutomationActivityAt = now"
M->>BP: "activeVisualAutomationCaptureCount += 1"
M->>DM: cancelHiddenWebViewDiscard
M->>BP: restoreDiscardedWebViewIfNeeded
alt shouldUseOffscreenRenderHost
M->>L: new OffscreenRenderHostLease
activate L
L->>L: move webView to OffscreenWindow
end
M-->>W: automationLease or nil
deactivate M
W->>W: body runs JS eval or snapshot
W->>M: defer v2MainSync endAutomationCommandLease
activate M
alt lease not nil
M->>L: end
L->>L: restore webView to previousSuperview
deactivate L
end
M->>BP: "webViewLastAutomationActivityAt = now"
M->>BP: "activeVisualAutomationCaptureCount -= 1"
M->>BP: refreshWebViewLifecycleState
alt count 0 and still hidden
M->>DM: scheduleHiddenWebViewDiscardIfNeeded
end
deactivate M
Reviews (3): Last reviewed commit: "Fix browser automation test budget" | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| deinit { | ||
| guard isActive else { return } | ||
| MainActor.assumeIsolated { | ||
| end() | ||
| } | ||
| } |
There was a problem hiding this comment.
assumeIsolated in deinit is an unchecked crash
@MainActor final class does not guarantee that deinit runs on the main actor in current Swift (SE-0371 "Isolated synchronous deinit" was returned for revision and is not in the language). If the last strong reference to an OffscreenRenderHostLease is released off the main thread when isActive is still true, MainActor.assumeIsolated will preconditionFailure-crash the app.
In the current flow the guard short-circuits in every reachable path (endAutomationCommandLease sets isActive = false before dropping the lease), but the safety net is meant to fire for unexpected drops — precisely the case where thread provenance is unknown. The correct defensive pattern is DispatchQueue.main.async { self.end() } (accepting that cleanup is deferred by one run-loop turn) rather than an unchecked assertion about the current thread.
| @@ -5325,6 +5326,7 @@ class TerminalController { | |||
| failure = .err(code: "invalid_params", message: "Surface is not a browser", data: ["surface_id": surfaceId.uuidString]) | |||
| return | |||
| } | |||
| automationLease = browserPanel.beginAutomationCommandLease(reason: "browser.socketCommand") | |||
| resolved = V2BrowserPanelContext( | |||
| workspaceId: ws.id, | |||
| surfaceId: surfaceId, | |||
| @@ -5333,6 +5335,11 @@ class TerminalController { | |||
| ) | |||
| } | |||
| guard let resolved else { return failure } | |||
| defer { | |||
| v2MainSync { | |||
| resolved.browserPanel.endAutomationCommandLease(automationLease, reason: "browser.socketCommand") | |||
| } | |||
| } | |||
| return body(resolved) | |||
There was a problem hiding this comment.
v2BrowserWithPanelContext is nonisolated and its body runs on a worker thread. When two socket commands arrive simultaneously for the same hidden panel, both enter v2MainSync and create OffscreenRenderHostLeases sequentially on the main thread — lease A moves the webview to window_A, then lease B moves it to window_B (capturing window_A.contentView as previousSuperview). The two defers then race to post back to the main queue via DispatchQueue.main.sync.
If lease A's endAutomationCommandLease is processed first (FIFO, wrong order): restoreWebView removes the webview from window_B and adds it to originalSuperview, then window_A.contentView = nil is called — making lease B's strong previousSuperview reference point to a detached view. Lease B then removes the webview from originalSuperview and adds it to that detached view. The webview ends up parented to a view that belongs to no window, and scheduleHiddenWebViewDiscardIfNeeded is called with the panel in an inconsistent state.
The fix is to record which lease is outermost and only allow restoration when the counter returns to its pre-lease value, or to adopt a stack/push-pop model where each lease validates that it still owns the current position before restoring.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
Sources/Panels/BrowserPanel.swift (1)
3784-3794:⚠️ Potential issue | 🟠 Major | ⚡ Quick winClear automation-activity timestamp on every lifecycle reset
Line 3792 only clears
webViewLastAutomationActivityAtwhenresetVisibilityistrue. Paths that callresetWebViewLifecycleMetadata(resetVisibility: false)after replacing/swapping the webview can carry stale automation activity into discard decisions for a different webview identity. Clear this timestamp unconditionally inresetWebViewLifecycleMetadata(...).Suggested fix
private func resetWebViewLifecycleMetadata(resetVisibility: Bool = true) { cancelHiddenWebViewDiscard() webViewLifecycleState = .newTab + webViewLastAutomationActivityAt = nil if resetVisibility { webViewLastVisibleAt = nil webViewLastHiddenAt = nil webViewLastVisibilityChangeAt = nil webViewLastVisibilityChangeReason = nil - webViewLastAutomationActivityAt = nil isWebViewVisibleInUI = false } hiddenWebViewDiscardManager.resetMetadata() isClosingWebViewLifecycle = false }As per coding guidelines, correctness-sensitive discard/automation paths must not decide using stale cached identity state.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@Sources/Panels/BrowserPanel.swift` around lines 3784 - 3794, The webViewLastAutomationActivityAt timestamp is only cleared when resetVisibility is true in the resetWebViewLifecycleMetadata function, but it should be cleared unconditionally to prevent stale automation activity timestamps from a previous webview identity from affecting discard decisions when the function is called with resetVisibility: false. Move the line that sets webViewLastAutomationActivityAt to nil outside of the if resetVisibility block so it executes on every call to resetWebViewLifecycleMetadata.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@Sources/Panels/BrowserPanel.swift`:
- Around line 3784-3794: The webViewLastAutomationActivityAt timestamp is only
cleared when resetVisibility is true in the resetWebViewLifecycleMetadata
function, but it should be cleared unconditionally to prevent stale automation
activity timestamps from a previous webview identity from affecting discard
decisions when the function is called with resetVisibility: false. Move the line
that sets webViewLastAutomationActivityAt to nil outside of the if
resetVisibility block so it executes on every call to
resetWebViewLifecycleMetadata.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: c12623d6-b489-4cfe-890d-20cd18cfea36
📒 Files selected for processing (3)
Sources/Panels/BrowserHiddenWebViewDiscardManager.swiftSources/Panels/BrowserPanel.swiftcmuxTests/BrowserPanelTests.swift
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit e3fd55b. Configure here.
| if activeVisualAutomationCaptureCount == 0, !isWebViewVisibleInUI { | ||
| scheduleHiddenWebViewDiscardIfNeeded(reason: "\(reason).finished") | ||
| } | ||
| } |
There was a problem hiding this comment.
Lease ends during active capture
Medium Severity
endAutomationCommandLease always tears down the automation offscreen host and restores the WKWebView when a socket command finishes, even if activeVisualAutomationCaptureCount is still above zero after decrement. Overlapping browser.screenshot (main actor) with a socket-worker command that held the lease can move the webview back to a hidden portal while PNG capture is still in progress, causing failed or incorrect screenshots.
Reviewed by Cursor Bugbot for commit e3fd55b. Configure here.
1 participant
Summary
Testing
xcodebuild test -project cmux.xcodeproj -scheme cmux-unit -configuration Debug -destination 'platform=macOS' -derivedDataPath /tmp/cmux-browser-bg-test -only-testing:cmuxTests/BrowserPanelVisualAutomationRestoreHostTests\n-./scripts/lint-pbxproj-test-wiring.sh\n-./scripts/reload-cloud.sh --tag bgbrws\n- Preflighted tagged appbgbrws: created Browser surface, backgrounded it, ranbrowser --surface surface:2 eval 'document.body.dataset.ready'andbrowser --surface surface:2 snapshot --compact, captured/tmp/cmux-bgbrws-browser.png.\nNeed help on this PR? Tag
/codesmithwith what you need. Autofix is disabled.Note
Medium Risk
Changes WebKit view hierarchy, hidden-webview memory discard timing, and the socket automation path; incorrect restore or lease teardown could crash WebKit or break background eval/snapshot, but behavior is guarded with tests and existing visual-automation blockers.
Overview
Socket-driven browser commands now run under a BrowserPanel automation lease that restores memory-discarded
WKWebViews, cancels hidden-webview discard while work is in flight, and optionally hosts the view in the same offscreen render panel used for screenshots when the surface is hidden or not in a window.OffscreenRenderHostLeasecentralizes attach/restore logic inBrowserScreenshotWebViewSnapshotter;TerminalControlleracquires the lease for the fullv2BrowserWithPanelContextsocket path. After a command, last automation activity resets the hidden-discard countdown (alongside wake time) so pages are not torn down immediately.Tests move to
BrowserPanelVisualAutomationRestoreHostTestsand add coverage for portal-hosted hidden tabs and discard blocking during leases.Reviewed by Cursor Bugbot for commit e3fd55b. Bugbot is set up for automated code reviews on this repo. Configure here.
Summary by cubic
Keeps background browser surfaces automatable by hosting hidden or discarded WebViews in an offscreen window during socket commands, then restoring normal lifecycle after. Discard is paused during commands and the hidden-discard countdown restarts using the last automation time at command end to avoid immediate discard.
New Features
BrowserPanel(beginAutomationCommandLease/endAutomationCommandLease), used byTerminalControllerfor all socket JS commands.Refactors
BrowserScreenshotWebViewSnapshotter.OffscreenRenderHostLeaseto reuse the screenshot offscreen render host for automation and snapshots.BrowserScreenshotWebViewSnapshotterto use the lease, reducing duplicated attach/detach logic.Written for commit e3fd55b. Summary will update on new commits.
Summary by CodeRabbit
New Features
Improvements
Tests