Verify shipped zips, Gatekeeper path, and install.sh in release CI - #50
Open
aboullaite wants to merge 5 commits into
Open
Verify shipped zips, Gatekeeper path, and install.sh in release CI#50aboullaite wants to merge 5 commits into
aboullaite wants to merge 5 commits into
Conversation
added 5 commits
July 20, 2026 11:33
The verify job previously ran the raw cross-compiled binaries, so the zips users actually download were never executed. Packaging now happens in the build job and verify exercises the real artifact end to end: checksum sidecar, user-style extraction (unzip / Expand-Archive), a simulated quarantined download on macOS so Gatekeeper actually assesses the notarization ticket, a version-output assertion to catch broken ldflags stamping, a doctor smoke run, and a skill-bundle presence check. A new reusable verify-release workflow covers the last mile after publishing: install.sh against the published assets on macOS and Linux. It runs as the final job of the Release workflow (a standalone 'on: release' trigger would never fire for GITHUB_TOKEN-created releases) and is also dispatchable for ad-hoc checks. The Release workflow itself gains a workflow_dispatch dry-run mode that builds, packages, and verifies without publishing anything.
- Extract the version assertion and doctor smoke run into scripts/verify-binary.sh, shared by both workflows via sparse checkouts, so the smoke-test definition lives in one place - Replace the hardcoded skill list in the verify job with a manifest the build job derives from plugin/skills, so the check cannot drift from what packaging actually copies - Use GITHUB_REF_TYPE/GITHUB_REF_NAME default env vars in the version step instead of re-plumbing github.ref_name through env - Use shasum -a 256 -c unconditionally (present on all six runners) instead of the sha256sum fallback dance - Drop the dead zip/sha256 guard in the packaging loop, the redundant needs edge on the release job, and the redundant tag guard on verify-published (skipped transitively when release is skipped) - Retain the run-scoped dist artifact for 1 day instead of the default
This reverts commit f06b092.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The release pipeline's verify job previously ran the raw cross-compiled binaries with
--version, so the zips users actually download were never executed, and Gatekeeper never assessed the notarization (CI artifacts carry no quarantine attribute). This PR makes the sanity checks exercise what users actually consume:Release workflow restructure
releasejob intobuild, soverifyoperates on the exact published artifacts.verify(all 6 native platforms) now: checks the sha256 sidecar, extracts like a user (unzip/Expand-Archive) with no chmod so a lost executable bit fails in CI, asserts--versionoutput matches the tag (catches broken ldflags stamping), and runsdoctorplus a skill-bundle presence check.verifyrunscodesign --verify --strictand setscom.apple.quarantinebefore executing, recreating the real download path so syspolicyd actually assesses the signature and notarization ticket. This is the check that would have caught the v0.1.4 incident class.workflow_dispatchdry-run mode: builds, packages, and verifies without publishing (version stamped as0.0.0-dryrun.N).New
verify-release.ymlcovers the last mile after publishing: runsinstall.sh --version <tag>against the published GitHub release assets on macOS (arm64 + intel) and Linux (amd64 + arm64), then asserts the installed binary reports the right version and passesdoctor. It runs as the final job of the Release workflow viaworkflow_call(a standaloneon: releasetrigger never fires for releases created withGITHUB_TOKEN) and is also dispatchable for ad-hoc checks of any existing release.Windows install.sh coverage is intentionally out of scope: users run it under Git Bash, which CI cannot reproduce faithfully; Windows binaries are covered by the native verify matrix.
Validation
valid on disk,satisfies its Designated Requirement, then version output).