test: serialize DuckDB extension installs across parallel tests #564

Workflow file for this run

	name: CI

	# A workflow skipped by `paths-ignore` never creates its check runs, and a
	# required status check is satisfied only by a check run that exists and
	# succeeded — "absent" reads as "pending", not "not applicable", so a docs-only
	# pull request would be blocked from merging forever. The triggers therefore
	# match every push and pull request, and each expensive job short-circuits
	# internally via its "Detect code changes" step.
	on:
	push:
	branches: [ main ]
	pull_request:
	branches: [ main ]

	# Without this, pushing three times to a pull request leaves three full five-job
	# fan-outs alive at once, all competing for the same runner pool — the queueing
	# dominates the wall clock a reviewer actually waits for. Pull requests are
	# grouped per PR number so a new push cancels the previous run; pushes to main
	# are grouped per commit and never cancel, so every merged commit on main keeps
	# its own CI result and stays bisectable.
	concurrency:
	group: ${{ github.workflow }}-${{ github.event_name == 'pull_request' && github.event.pull_request.number \|\| github.sha }}
	cancel-in-progress: ${{ github.event_name == 'pull_request' }}

	env:
	RUST_BACKTRACE: 1
	CARGO_TERM_COLOR: always
	RUSTC_WRAPPER: "sccache"
	# Incremental artifacts are useless on a fresh CI runner and inflate the target
	# directory that the cache below has to carry, so keep them out of it. (This is
	# also what sccache needs in order to cache Rust at all — it declines to cache
	# incrementally-compiled crates, hence the "Cache hits rate (Rust) 0.00 %" in
	# run logs. sccache still only caches within a run: its local-disk store is not
	# persisted, and its GHA backend needs the retired Actions Cache v1 API.)
	CARGO_INCREMENTAL: "0"

	jobs:
	fmt:
	name: Rustfmt
	runs-on: ubuntu-latest
	steps:
	- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6

	- name: Install Rust
	uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # stable
	with:
	components: rustfmt

	- name: Check formatting
	run: cargo fmt --all -- --check

	check:
	name: Check & Build
	runs-on: ${{ matrix.os }}
	strategy:
	matrix:
	os:
	- ubuntu-latest
	- macos-latest
	rust: [stable]
	env:
	# Smaller artifacts + faster links for the bundled-DuckDB build; disk
	# headroom itself comes from the scratch-disk step below (Linux).
	CARGO_PROFILE_TEST_DEBUG: "0"
	CARGO_PROFILE_DEV_DEBUG: "0"
	steps:
	- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
	with:
	# Two commits, so HEAD^1 exists: on a pull_request HEAD is the merge
	# commit and HEAD^1 is the base; on a push to main HEAD^1 is the
	# previous commit. Either way the diff is this change's own files.
	fetch-depth: 2

	- name: Detect code changes
	id: changes
	shell: bash
	run: \|
	files=$(git diff --name-only HEAD^1 HEAD)
	echo "Changed files:"; echo "$files"
	# Assigned first rather than piped: `grep -q` exits on its first match
	# and would SIGPIPE git, which `pipefail` turns into a failed
	# pipeline — read as "no code changed", silently skipping the build.
	if grep -qvE '(\.md$\|^LICENSE$\|^\.gitignore$\|^\.github/CODEOWNERS$)' <<<"$files"; then
	echo "code=true" >> "$GITHUB_OUTPUT"
	else
	echo "code=false" >> "$GITHUB_OUTPUT"
	echo "Docs and repository metadata only — skipping build and tests."
	fi

	- name: Build on the runner's large scratch disk (Linux)
	if: steps.changes.outputs.code == 'true' && runner.os == 'Linux'
	run: \|
	sudo mkdir -p /mnt/cargo-target
	sudo chown "$(id -un)" /mnt/cargo-target
	echo "CARGO_TARGET_DIR=/mnt/cargo-target" >> "$GITHUB_ENV"
	df -h / /mnt

	- name: Install Rust
	if: steps.changes.outputs.code == 'true'
	uses: dtolnay/rust-toolchain@3c5f7ea28cd621ae0bf5283f0e981fb97b8a7af9 # master
	with:
	toolchain: ${{ matrix.rust }}
	components: clippy

	- name: Install sccache
	if: steps.changes.outputs.code == 'true'
	uses: mozilla-actions/sccache-action@fd02668681acd5f960e1372061bee5e3e987195c # v0.0.11

	# One step for both cargo-home paths. As two steps, the `~/.cargo/git` one
	# logged "Path Validation Error: Path(s) ... do(es) not exist, hence no
	# cache is being saved" on every run — the crate has no git dependencies, so
	# that directory is never created. Cached together, the registry always
	# exists so the step is valid, and a future git dependency is still covered.
	- name: Cache cargo home
	if: steps.changes.outputs.code == 'true'
	uses: actions/cache@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5
	with:
	path: \|
	~/.cargo/registry
	~/.cargo/git
	key: ${{ runner.os }}-cargo-home-${{ hashFiles('**/Cargo.lock') }}
	restore-keys: \|
	${{ runner.os }}-cargo-home-

	# Cache the directory cargo actually writes to. This said `path: target` while
	# the Linux scratch-disk step above redirects `CARGO_TARGET_DIR` to
	# /mnt/cargo-target, so every Linux run logged "Path Validation Error: Path(s)
	# ... do(es) not exist, hence no cache is being saved" and rebuilt from
	# scratch. macOS has no scratch-disk step, so `target` is correct there.
	# Restore on every run, save only from main. GitHub gives the repository a
	# 10 GB cache budget and evicts least-recently-used across it, and a run's
	# writes land in its own ref's scope — which no other pull request can read.
	# A pull request that changes Cargo.lock therefore gets a fresh key, builds
	# cold, and writes multi-gigabyte entries that are useful to nobody but
	# itself while evicting the main-scoped entries every other run restores
	# from. That is not hypothetical: PR #211 held 9.21 GB across four entries
	# (two `macOS-target-stable` generations at 3.17 GB each) and had evicted
	# every main-scoped cache in the repository, so unrelated pull requests were
	# building the two slowest jobs from scratch.
	#
	# `Cargo.toml` joins the key because `[profile.dev.package.libduckdb-sys]`
	# now changes the emitted artifacts. Without it the key would be unchanged,
	# so main would hit exactly, skip saving, and the cache would keep serving
	# unoptimised DuckDB objects that every run then recompiles and discards.
	#
	# `rust-toolchain.toml` is in the key because it, not `matrix.rust`, decides
	# the compiler: it pins 1.94.0 and overrides the toolchain the `stable`
	# action installs (the run log says as much — "note that the toolchain
	# '1.94.0-...' is currently in use (overridden by ...rust-toolchain.toml)").
	- name: Restore target directory
	id: target-cache
	if: steps.changes.outputs.code == 'true'
	uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5
	with:
	path: ${{ runner.os == 'Linux' && '/mnt/cargo-target' \|\| 'target' }}
	key: ${{ runner.os }}-target-${{ matrix.rust }}-${{ hashFiles('**/Cargo.lock', 'rust-toolchain.toml', 'Cargo.toml') }}
	restore-keys: \|
	${{ runner.os }}-target-${{ matrix.rust }}-

	# `--workspace` so this also lints the `benchmark` package. That coverage
	# used to come from `cargo check --workspace --all-targets` in
	# `feature-check`, which was deleted: `--all-features` is a superset of the
	# default features that check used, and `--all-targets` matches, so the
	# only thing it contributed was workspace scope. Bare cargo commands act on
	# the root package alone here, because the workspace root is itself a
	# package rather than a virtual manifest.
	#
	# Linux only, and measured rather than assumed: `ducklake-benchmark`
	# depends on `duckdb` with `bundled`, so pulling it in makes clippy compile
	# DuckDB's C++ a second time, in a second feature context. On macOS that
	# took this step from 56s to 208s (run 30531835616) — and macOS is the
	# longest required check, so it lands squarely on the critical path.
	# Restricting the flag also matches the coverage this replaced exactly:
	# `feature-check` is `runs-on: ubuntu-latest`, so the `benchmark` package
	# was only ever checked on Linux, and the crate has no `target_os` cfgs.
	# `autotests = false` means a file at `tests/<name>.rs` is not a test target
	# and is never built — silently. No warning, no error, CI green, and the
	# tests in it simply never run. That is an easy mistake to make while
	# rebasing a branch that predates the move to `tests/it/`, so fail loudly
	# instead. Linux only; this is a filesystem check, not a platform one.
	- name: Check for stray test files outside tests/it
	if: steps.changes.outputs.code == 'true' && runner.os == 'Linux'
	shell: bash
	run: \|
	stray=$(find tests -maxdepth 1 -name '*.rs' -type f)
	if [[ -n "$stray" ]]; then
	echo '::error::These files are not compiled. With autotests = false only tests/it/main.rs is a test target, so move them into tests/it/ and add a mod declaration there:'
	echo "$stray"
	exit 1
	fi
	echo "No stray test files."

	- name: Run clippy
	if: steps.changes.outputs.code == 'true'
	run: cargo clippy ${{ runner.os == 'Linux' && '--workspace' \|\| '' }} --all-targets --all-features --no-deps -- -D warnings

	# macOS only. This step used to run on Linux too, as
	# `cargo test --features write-sqlite` — but `--features` keeps the default
	# features on, so that feature set (metadata-duckdb + duckdb-bundled +
	# write + metadata-sqlite + write-sqlite) is a strict subset of the one
	# `test-full` uses, and no test file has a `cfg(not(feature = ...))` gate
	# that could make the smaller set enable something the larger one doesn't.
	# Every test therefore ran twice per pull request, ~8 minutes of it, for no
	# extra signal. Linux keeps clippy and the doc build below.
	#
	# macOS stays: it is the only remaining job that runs the suite with
	# `encryption` off, which is what exercises the `cfg(not(feature =
	# "encryption"))` read paths in `table.rs` and `table_changes.rs`. If this
	# job is ever reduced to a build, those paths lose all test coverage.
	#
	# `write-sqlite` pulls in the writer and sqlite-provider tests,
	# which are `#![cfg]`-gated; without it cargo compiles them to
	# zero-test binaries.
	- name: Run tests (macOS; Linux is covered by test-full)
	if: steps.changes.outputs.code == 'true' && runner.os == 'macOS'
	run: cargo test --features "skip-tests-with-docker write-sqlite"

	- name: Check documentation
	if: steps.changes.outputs.code == 'true'
	run: cargo doc --no-deps --all-features
	env:
	RUSTDOCFLAGS: -D warnings

	# Guarded on `cache-hit != 'true'`: that output is set only by an exact
	# primary-key match, where the entry is already current and `save` would
	# fail on the duplicate key. A restore-keys-only match reports 'false',
	# which is what lets the entry refresh after a lock, toolchain, or profile
	# change.
	- name: Save target directory (main only)
	if: \|
	steps.changes.outputs.code == 'true'
	&& github.ref == 'refs/heads/main'
	&& steps.target-cache.outputs.cache-hit != 'true'
	uses: actions/cache/save@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5
	with:
	path: ${{ runner.os == 'Linux' && '/mnt/cargo-target' \|\| 'target' }}
	key: ${{ runner.os }}-target-${{ matrix.rust }}-${{ hashFiles('**/Cargo.lock', 'rust-toolchain.toml', 'Cargo.toml') }}

	# Every other job builds with `--all-features` or `write-sqlite`, which only
	# proves the union of features compiles. It cannot catch a module that
	# accidentally depends on a sibling feature, because a stronger feature set
	# always supplies the missing one. That is a hard error for real consumers: an
	# ungated module referencing a `write`-gated one fails only when `write` is off,
	# and the sole build without `write` was the Benchmark workflow — push-to-main
	# only, so such a break stayed invisible until after the merge.
	#
	# Runs on one OS by design: this checks `#[cfg(feature = ...)]` resolution, which
	# is platform-independent (the crate has no `target_os`/`target_arch` cfgs and no
	# platform-conditional dependencies), so a second OS would only duplicate compute.
	# Its own job so it runs concurrently with `check` instead of doubling that job.
	feature-check:
	name: Check feature combinations
	runs-on: ubuntu-latest
	steps:
	- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
	with:
	# Two commits, so HEAD^1 exists: on a pull_request HEAD is the merge
	# commit and HEAD^1 is the base; on a push to main HEAD^1 is the
	# previous commit. Either way the diff is this change's own files.
	fetch-depth: 2

	- name: Detect code changes
	id: changes
	shell: bash
	run: \|
	files=$(git diff --name-only HEAD^1 HEAD)
	echo "Changed files:"; echo "$files"
	# Assigned first rather than piped: `grep -q` exits on its first match
	# and would SIGPIPE git, which `pipefail` turns into a failed
	# pipeline — read as "no code changed", silently skipping the build.
	if grep -qvE '(\.md$\|^LICENSE$\|^\.gitignore$\|^\.github/CODEOWNERS$)' <<<"$files"; then
	echo "code=true" >> "$GITHUB_OUTPUT"
	else
	echo "code=false" >> "$GITHUB_OUTPUT"
	echo "Docs and repository metadata only — skipping feature checks."
	fi

	- name: Build on the runner's large scratch disk
	if: steps.changes.outputs.code == 'true'
	run: \|
	sudo mkdir -p /mnt/cargo-target
	sudo chown "$(id -un)" /mnt/cargo-target
	echo "CARGO_TARGET_DIR=/mnt/cargo-target" >> "$GITHUB_ENV"

	- name: Install Rust
	if: steps.changes.outputs.code == 'true'
	uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # stable

	- name: Install sccache
	if: steps.changes.outputs.code == 'true'
	uses: mozilla-actions/sccache-action@fd02668681acd5f960e1372061bee5e3e987195c # v0.0.11

	# One step for both cargo-home paths, matching the other jobs: as two steps
	# the `~/.cargo/git` one logs "Path Validation Error" on every run, because
	# the crate has no git dependencies so that directory is never created.
	- name: Cache cargo home
	if: steps.changes.outputs.code == 'true'
	uses: actions/cache@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5
	with:
	path: \|
	~/.cargo/registry
	~/.cargo/git
	key: ${{ runner.os }}-cargo-home-${{ hashFiles('**/Cargo.lock') }}
	restore-keys: \|
	${{ runner.os }}-cargo-home-

	# Deliberately NOT target-cached, even though this job recompiles its twelve
	# feature graphs from scratch every run. Unlike `check` and `test-full`, this
	# is not a required status check, so its duration never gates a merge — it
	# costs runner minutes and delays the all-green tick, nothing else. Deleting
	# the redundant `--workspace --all-targets` check below recovers most of the
	# time without spending any of the repository's 10 GB cache budget.
	#
	# Adding one is now defensible if the runner minutes matter (the save-only-on-
	# main split elsewhere keeps the budget bounded at roughly 7 GB rather than
	# letting pull requests fill it), and the entry would be ~1.3 GB of
	# check-only rmeta. It just has to be worth a third of the budget.
	#
	# Isolate each independently-supported feature axis once.
	#
	# This used to be preceded by `cargo check --workspace --all-targets`,
	# which was 496s of the step's 1189s — ~427s of it compiling DuckDB's C++
	# via `libduckdb-sys`, since `--workspace` picks up the default features and
	# those include `duckdb-bundled`. It was redundant: `check`'s
	# `cargo clippy --all-targets --all-features` already covers a superset of
	# those features and the same targets, and it target-caches that C++ compile.
	# The one thing it did contribute was workspace scope (the `benchmark`
	# package), so `--workspace` moved onto that clippy invocation.
	#
	# No `-D warnings`: a read-only build legitimately reports dead_code for the
	# write-only helpers, and `#[cfg]`-gating struct fields to silence that would
	# be worse than the warning.
	- name: Check feature combinations
	if: steps.changes.outputs.code == 'true'
	run: \|
	for features in \
	"" \
	metadata-duckdb \
	metadata-sqlite \
	metadata-postgres \
	metadata-mysql \
	multicatalog-postgres \
	write \
	write-duckdb \
	write-sqlite \
	write-postgres \
	write-mysql \
	metadata-duckdb,encryption
	do
	echo "::group::cargo check --no-default-features --features '$features'"
	cargo check --no-default-features --features "$features"
	echo "::endgroup::"
	done


	test-full:
	name: Test (single-catalog backends + encryption, Docker)
	runs-on: ubuntu-latest
	env:
	# Bundled-DuckDB debug binaries are huge and this job links ~25 of them
	# under the full feature set, which exhausts the runner's disk during
	# linking (lld dies with SIGBUS). Dropping debuginfo keeps them small.
	CARGO_PROFILE_TEST_DEBUG: "0"
	CARGO_PROFILE_DEV_DEBUG: "0"
	steps:
	- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
	with:
	# Two commits, so HEAD^1 exists: on a pull_request HEAD is the merge
	# commit and HEAD^1 is the base; on a push to main HEAD^1 is the
	# previous commit. Either way the diff is this change's own files.
	fetch-depth: 2

	- name: Detect code changes
	id: changes
	shell: bash
	run: \|
	files=$(git diff --name-only HEAD^1 HEAD)
	echo "Changed files:"; echo "$files"
	# Assigned first rather than piped: `grep -q` exits on its first match
	# and would SIGPIPE git, which `pipefail` turns into a failed
	# pipeline — read as "no code changed", silently skipping the build.
	if grep -qvE '(\.md$\|^LICENSE$\|^\.gitignore$\|^\.github/CODEOWNERS$)' <<<"$files"; then
	echo "code=true" >> "$GITHUB_OUTPUT"
	else
	echo "code=false" >> "$GITHUB_OUTPUT"
	echo "Docs and repository metadata only — skipping build and tests."
	fi

	- name: Build on the runner's large scratch disk
	if: steps.changes.outputs.code == 'true'
	run: \|
	sudo mkdir -p /mnt/cargo-target
	sudo chown "$(id -un)" /mnt/cargo-target
	echo "CARGO_TARGET_DIR=/mnt/cargo-target" >> "$GITHUB_ENV"
	df -h / /mnt

	- name: Install Rust
	if: steps.changes.outputs.code == 'true'
	uses: dtolnay/rust-toolchain@3c5f7ea28cd621ae0bf5283f0e981fb97b8a7af9 # master
	with:
	toolchain: stable

	- name: Install sccache
	if: steps.changes.outputs.code == 'true'
	uses: mozilla-actions/sccache-action@fd02668681acd5f960e1372061bee5e3e987195c # v0.0.11

	# One step for both cargo-home paths. As two steps, the `~/.cargo/git` one
	# logged "Path Validation Error: Path(s) ... do(es) not exist, hence no
	# cache is being saved" on every run — the crate has no git dependencies, so
	# that directory is never created. Cached together, the registry always
	# exists so the step is valid, and a future git dependency is still covered.
	- name: Cache cargo home
	if: steps.changes.outputs.code == 'true'
	uses: actions/cache@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5
	with:
	path: \|
	~/.cargo/registry
	~/.cargo/git
	key: ${{ runner.os }}-cargo-home-${{ hashFiles('**/Cargo.lock') }}
	restore-keys: \|
	${{ runner.os }}-cargo-home-

	# This job always uses the scratch disk, so cache that path rather than the
	# `target` it never writes to. Restore/save split and key composition are
	# explained in the `check` job — this is the largest entry of the three, so
	# it is the one that most needs to stay out of pull-request cache scopes.
	- name: Restore target directory
	id: target-cache
	if: steps.changes.outputs.code == 'true'
	uses: actions/cache/restore@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5
	with:
	path: /mnt/cargo-target
	key: ${{ runner.os }}-target-full-${{ hashFiles('**/Cargo.lock', 'rust-toolchain.toml', 'Cargo.toml') }}
	restore-keys: \|
	${{ runner.os }}-target-full-

	# Runs the single-catalog suites across every metadata backend
	# (Postgres/MySQL via testcontainers, SQLite, DuckDB) + encryption.
	# `skip-tests-with-docker` is NOT set, so the container-backed suites run.
	# Multicatalog (the runtimedb-specific layer, gated on `write-postgres`) is
	# intentionally excluded here — its test files stay in-tree but don't run.
	- name: Run single-catalog test suite
	if: steps.changes.outputs.code == 'true'
	run: cargo test --features "duckdb-bundled encryption metadata-mysql metadata-postgres metadata-sqlite write-sqlite"

	- name: Save target directory (main only)
	if: \|
	steps.changes.outputs.code == 'true'
	&& github.ref == 'refs/heads/main'
	&& steps.target-cache.outputs.cache-hit != 'true'
	uses: actions/cache/save@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5
	with:
	path: /mnt/cargo-target
	key: ${{ runner.os }}-target-full-${{ hashFiles('**/Cargo.lock', 'rust-toolchain.toml', 'Cargo.toml') }}

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

test: serialize DuckDB extension installs across parallel tests #564

Workflow file

test: serialize DuckDB extension installs across parallel tests #564

Uh oh!

Workflow file for this run