Description

Adds commit-on-branch verification to the checkout phase.

As part of our agent checkout improvement work, we are adding support for git commit verification. This addresses a potential security issue: if an attacker adds a malicious commit to the repo, and is then able to trigger a build which specifies commit: 3v1l5ha123 branch: main we would build that commit as if it were on the main branch, without verifying that this is the case, which could potentially lead to a production deployment of malicious code.

This change introduces a BUILDKITE_GIT_COMMIT_VERIFICATION environment variable, which defaults to false. If set to true, when we are given a commit and branch, before checkout and build, we do the following:

1. Use git merge-base --is-ancestor to check if the commit really belongs to the specified branch.
2. If it does, proceed; if it definitely does not, fail and exit.
3. In ambiguous situations, it's possible that we only have a shallow clone of the repo. In that case we:
4. Deepen the clone by 50 commits, with the assumption that this should be enough to find the specified SHA.
5. If we still get an ambiguous result, to a full unshallow and check again.
6. Proceed if the commit matches, fail and exit if it does not.

In terms of alternatives, the only change I considered was just going straight to unshallow before verifying any commit. But I didn't want users to have to completely give up shallow commits to get verification, so I went with the "deepen first, then unshallow if necessary" approach.

Context

See SUP-6535.

Changes

Adds a BUILDKITE_GIT_COMMIT_VERIFICATION env var to enable to functionality. When set, we perform various git operations in between fetch and checkout, to make sure the provided commit is valid on the provided branch.

Testing

• Tests have run locally (with go test ./...). Buildkite employees may check this if the pipeline has run automatically.
• Code is formatted (with go tool gofumpt -extra -w .)

Disclosures / Credits

I used Claude to help me plan the feature and teach me some Go fundamentals. I coded the core functionality myself, and let Claude write the tests.