feat: add sparse checkout support - #3903
Merged
Merged
Conversation
mcncl
commented
May 7, 2026
mcncl
commented
May 7, 2026
mcncl
force-pushed
the
SUP-6409/sparse_checkout
branch
from
May 7, 2026 05:00
318d9ef to
163cbea
Compare
mcncl
marked this pull request as ready for review
May 7, 2026 05:08
DrJosh9000
reviewed
May 7, 2026
DrJosh9000
left a comment
DrJosh9000
left a comment
Contributor
There was a problem hiding this comment.
My first reaction is "why not _Git_SparseCheckoutPaths", but I'm not fussed
Contributor
Author
|
@DrJosh9000 thats a fair call and removes any chance of ambiguity |
Contributor
mcncl
force-pushed
the
SUP-6409/sparse_checkout
branch
from
May 7, 2026 23:11
163cbea to
9e93039
Compare
zhming0
reviewed
May 14, 2026
zhming0
left a comment
zhming0
left a comment
Contributor
There was a problem hiding this comment.
I left some suggestions re code organization, increase signal-to-noise ratio on integration test and adding an e2e test. 🙏🏿
so far no blocker, but I plan to take another look later 👀
mcncl
force-pushed
the
SUP-6409/sparse_checkout
branch
from
June 1, 2026 02:18
f844882 to
a93473e
Compare
petetomasik
reviewed
Jun 1, 2026
Contributor
|
hey @mcncl i think there's one things to fix up with the lint and we can get this merged |
Contributor
Author
|
@quinn-diesel cheers mate. Ran out of time today but will get this sorted first thing |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds git sparse checkout (cone mode) support to the Buildkite agent/executor via BUILDKITE_GIT_SPARSE_CHECKOUT_PATHS, including handling reuse of checkout directories by reconfiguring or disabling sparse checkout appropriately, and updating integration/unit tests to cover the new behavior.
Changes:
- Add a new executor/agent config surface (
--git-sparse-checkout-paths,BUILDKITE_GIT_SPARSE_CHECKOUT_PATHS) and propagate it through agent → bootstrap → executor. - Run sparse checkout setup after fetch and before checkout; disable sparse checkout when not requested.
- Skip submodule initialization when sparse checkout is active, and add coverage in unit/integration tests.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| internal/job/integration/checkout_integration_test.go | Adds sparse checkout integration coverage (basic behavior, reconfigure existing checkout dir, disable/restore full tree, and submodule skip behavior). |
| internal/job/integration/checkout_git_mirrors_integration_test.go | Adds sparse checkout coverage when using git mirrors. |
| internal/job/config.go | Adds GitSparseCheckoutPaths to executor config mapped from BUILDKITE_GIT_SPARSE_CHECKOUT_PATHS. |
| internal/job/config_test.go | Extends env→config mapping test to include sparse checkout paths. |
| internal/job/checkout.go | Invokes sparse checkout setup in the checkout flow; skips submodule init when sparse checkout is active. |
| internal/job/checkout_sparse.go | Introduces sparse-checkout enable/disable logic, git version gating, and “reuse checkout dir” safety behavior. |
| internal/job/checkout_sparse_test.go | Adds unit tests for sparse checkout path cleaning, git version parsing, enable/disable flows, and fallbacks. |
| clicommand/global.go | Adds --git-sparse-checkout-paths CLI flag shared across commands. |
| clicommand/bootstrap.go | Wires sparse checkout paths into bootstrap config and executor config construction. |
| clicommand/agent_start.go | Wires sparse checkout paths into agent start config and runtime agent configuration. |
| agent/job_runner.go | Propagates sparse checkout paths into the bootstrap/job environment. |
| agent/agent_configuration.go | Adds GitSparseCheckoutPaths to runtime agent configuration struct. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
quinn-diesel
approved these changes
Jun 12, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This adds sparse checkout support to the agent via
BUILDKITE_GIT_SPARSE_CHECKOUT_PATHS, allowing checkout to materialize only selected paths in the working tree.It also handles reused checkout directories safely by reconfiguring sparse checkout when paths change, and disabling sparse checkout when a later job does not request it.
Changes
git-sparse-checkout-pathssupport toagent startandbootstrapBUILDKITE_GIT_SPARSE_CHECKOUT_PATHSthrough the agent/bootstrap environmentExecutor.setupSparseCheckoutafter fetch and before checkoutinternal/job/checkout_sparse.gointernal/job/checkout_sparse_test.goTesting
go test ./...). Buildkite employees may check this if the pipeline has run automatically.go tool gofumpt -extra -w .)Disclosures / Credits
I used Codex to assist with writing this change. The core function (non
_test.go) changes were implemented by me, I then used Codex to assist with writing theTestfunctions.I used Amp to check the changes, which it highlighted a couple of issues with (fixed) and a missing test case (added).