-
Notifications
You must be signed in to change notification settings - Fork 498
ci: add multi-arch Docker image test workflow #448
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Komzpa
wants to merge
2
commits into
postgis:master
Choose a base branch
from
Komzpa:codex/multiarch-test-workflow
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,251 @@ | ||
| # testing : https://github.com/postgis/docker-postgis/pull/432 | ||
| # original author: https://github.com/BowlesCR | ||
|
|
||
| # To pin GitHub Actions versions by commit hash: 'pinact run -u --min-age 7' ( https://github.com/suzuki-shunsuke/pinact ) | ||
|
|
||
| name: TEST - Docker PostGIS CI | ||
| # Serialize publish-capable runs per ref so scheduled/push/manual runs queue instead of racing. | ||
| concurrency: | ||
| group: test-${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: false | ||
|
|
||
| on: | ||
| push: | ||
| pull_request: | ||
| workflow_dispatch: | ||
| schedule: | ||
| - cron: '15 13 * * *' | ||
|
|
||
| # ============================================================================ | ||
| # FOR FORKING: Modify these settings in your forked repository | ||
| # ============================================================================ | ||
| env: | ||
| DOCKERHUB_REPO: postgis/docker-postgis-test | ||
| GITHUB_REPO: postgis/docker-postgis | ||
| LATEST_VERSION: 17-3.5 | ||
| DOCKERHUB_SHORT_DESCRIPTION: "TEST REPO - PostGIS Docker" | ||
| DOCKERHUB_README_PREFIX: "# WARNING: This is a TEST repository ONLY\n\n" | ||
| RUNNER_PLATFORMS_JSON: '["ubuntu-24.04","ubuntu-24.04-arm"]' # Runner platforms used to expand the build matrix | ||
| # | ||
| # Also add these secrets in your repository settings: | ||
| # https://github.com/${GITHUB_REPO}/settings/secrets/actions | ||
| # - secrets.DOCKERHUB_USERNAME | ||
| # - secrets.DOCKERHUB_ACCESS_TOKEN ( READ, Write, Delete access ) | ||
| # ============================================================================ | ||
|
|
||
| defaults: | ||
| run: | ||
| shell: bash --noprofile --norc -euo pipefail {0} | ||
|
|
||
| jobs: | ||
|
|
||
| setup: | ||
| # This job sets up configuration constants and loads the CI matrix from matrix.yml. | ||
| # - Constants: CANONICAL_REPO and SHOULD_PUBLISH flag (fork-friendly configuration) | ||
| # - Matrix: BUILD_TARGETS and BUILD_INCLUDE arrays (automatically generated by ./update.sh) | ||
| name: Setup and Load Configuration | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| CANONICAL_REPO: ${{ steps.constants.outputs.CANONICAL_REPO }} | ||
| SHOULD_PUBLISH: ${{ steps.constants.outputs.SHOULD_PUBLISH }} | ||
| BUILD_INCLUDE: ${{ steps.matrix.outputs.BUILD_INCLUDE }} | ||
| BUILD_TARGETS: ${{ steps.matrix.outputs.BUILD_TARGETS }} | ||
| steps: | ||
| - name: Set constants | ||
| id: constants | ||
| env: | ||
| VAR_CANONICAL: ${{ vars.CANONICAL_REPO }} | ||
| run: | | ||
| CANONICAL_REPO="${VAR_CANONICAL:-$GITHUB_REPO}" | ||
| echo "CANONICAL_REPO=$CANONICAL_REPO" >> "$GITHUB_OUTPUT" | ||
|
|
||
| # Compute if we should publish | ||
| if [[ "${{ github.repository }}" == "$CANONICAL_REPO" ]] && \ | ||
| [[ "${{ github.ref }}" == "refs/heads/master" ]] && \ | ||
| [[ "${{ github.event_name }}" != "pull_request" ]]; then | ||
| echo "SHOULD_PUBLISH=true" >> "$GITHUB_OUTPUT" | ||
| else | ||
| echo "SHOULD_PUBLISH=false" >> "$GITHUB_OUTPUT" | ||
| fi | ||
|
|
||
| - name: Checkout repository | ||
| uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 | ||
|
|
||
| - name: Load and validate matrix | ||
| id: matrix | ||
| run: bash ci/matrix.sh | ||
|
|
||
| make-docker-images: | ||
| needs: setup | ||
| strategy: | ||
| matrix: | ||
| include: ${{ fromJSON(needs.setup.outputs.BUILD_INCLUDE) }} | ||
|
|
||
| name: "Build:${{ matrix.postgres }}-${{ matrix.postgis }}-${{ matrix.variant }} (${{ contains(matrix.runner-platform, 'arm') && 'arm64' || 'x86-64' }}) Docker image" | ||
| runs-on: ${{ matrix.runner-platform }} | ||
| continue-on-error: ${{ matrix.postgis == 'master' }} | ||
| env: | ||
| VERSION: ${{ matrix.postgres }}-${{ matrix.postgis }} | ||
| VARIANT: ${{ matrix.variant }} | ||
| # the "postgis/postgis" name is the expected test name; ( via ./test/postgis-config.sh ) | ||
| # changing it will break the official-images test script | ||
| # this is only for CI test and not for Docker hub publishing | ||
| CI_IMAGE_TAG: postgis/postgis:ci-${{ github.run_id }}-${{ matrix.postgres }}-${{ matrix.postgis }}-${{ matrix.variant }}-${{ contains(matrix.runner-platform, 'arm') && 'arm' || 'x64' }} | ||
|
|
||
| steps: | ||
| - name: Checkout source | ||
| uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1 | ||
|
|
||
| - name: Compute build directory | ||
| run: | | ||
| if [[ "$VARIANT" == "alpine" ]]; then | ||
| echo "BUILD_DIR=$VERSION/alpine" >> "$GITHUB_ENV" | ||
| else | ||
| echo "BUILD_DIR=$VERSION" >> "$GITHUB_ENV" | ||
| fi | ||
|
|
||
| - name: Build Docker image for ${{ env.VERSION }} ${{ env.VARIANT }} | ||
| id: build | ||
| uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0 | ||
| with: | ||
| context: ${{ env.BUILD_DIR }} | ||
| file: ${{ env.BUILD_DIR }}/Dockerfile | ||
| tags: ${{ env.CI_IMAGE_TAG }} | ||
| load: true | ||
| push: false # don't push until after testing | ||
|
|
||
| - name: Check out official-images repo | ||
| uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 | ||
| with: | ||
| repository: docker-library/official-images | ||
| path: official-images | ||
| sparse-checkout: | | ||
| test | ||
|
|
||
| - name: Test image with official-images | ||
| run: bash ci/test-image.sh "$CI_IMAGE_TAG" | ||
|
|
||
| - name: Login to dockerhub | ||
| id: login-dockerhub | ||
| uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0 | ||
| if: ${{ needs.setup.outputs.SHOULD_PUBLISH == 'true' }} | ||
| with: | ||
| username: ${{ secrets.DOCKERHUB_USERNAME }} | ||
| password: ${{ secrets.DOCKERHUB_ACCESS_TOKEN }} | ||
|
|
||
| - name: Push image by digest | ||
| id: push | ||
| uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0 | ||
| if: ${{ needs.setup.outputs.SHOULD_PUBLISH == 'true' && steps.login-dockerhub.outcome == 'success' }} | ||
| with: | ||
| context: ${{ env.BUILD_DIR }} | ||
| file: ${{ env.BUILD_DIR }}/Dockerfile | ||
| outputs: type=image,"name=${{ env.DOCKERHUB_REPO }}",push-by-digest=true,name-canonical=true,push=true | ||
|
|
||
| - name: Export digest | ||
| if: ${{ steps.push.outcome == 'success' }} | ||
| run: | | ||
| mkdir -p ${{ runner.temp }}/digests | ||
| digest="${{ steps.push.outputs.digest }}" | ||
| touch "${{ runner.temp }}/digests/${digest#sha256:}" | ||
|
|
||
| - name: Upload digests | ||
| if: ${{ steps.push.outcome == 'success' }} | ||
| uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0 | ||
| with: | ||
| name: digests-${{ github.run_id }}-${{ env.VERSION }}-${{ env.VARIANT }}-${{ matrix.runner-platform }} | ||
| path: ${{ runner.temp }}/digests/* | ||
| if-no-files-found: error | ||
| retention-days: 10 | ||
|
|
||
| merge-manifests: | ||
| name: "Merge:${{ matrix.postgres }}-${{ matrix.postgis }}-${{ matrix.variant }} manifests and push to DockerHub" | ||
| needs: [setup, make-docker-images] | ||
| runs-on: ubuntu-24.04-arm # Run on arm runner for manifest merge | ||
| if: ${{ needs.setup.outputs.SHOULD_PUBLISH == 'true' }} | ||
| # Ensure each tag variant is published by only one workflow run at a time to keep manifests consistent. | ||
| concurrency: | ||
| group: merge-${{ matrix.postgres }}-${{ matrix.postgis }}-${{ matrix.variant }} | ||
| cancel-in-progress: false | ||
| continue-on-error: ${{ matrix.postgis == 'master' }} | ||
| env: | ||
| VERSION: ${{ matrix.postgres }}-${{ matrix.postgis }} | ||
| VARIANT: ${{ matrix.variant }} | ||
| strategy: | ||
| matrix: | ||
| include: ${{ fromJSON(needs.setup.outputs.BUILD_TARGETS) }} | ||
|
|
||
| steps: | ||
| - name: Checkout source | ||
| uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 | ||
|
|
||
| - name: Login to dockerhub | ||
| id: login-dockerhub | ||
| uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0 | ||
| with: | ||
| username: ${{ secrets.DOCKERHUB_USERNAME }} | ||
| password: ${{ secrets.DOCKERHUB_ACCESS_TOKEN }} | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1 | ||
|
|
||
| - name: Download digests | ||
| uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0 | ||
| with: | ||
| path: ${{ runner.temp }}/digests | ||
| pattern: digests-${{ github.run_id }}-${{ env.VERSION }}-${{ env.VARIANT }}-* | ||
| merge-multiple: true | ||
|
|
||
| - name: Verify digests for all platforms | ||
| working-directory: ${{ runner.temp }}/digests | ||
| run: | | ||
| expected="$(jq 'length' <<< "$RUNNER_PLATFORMS_JSON")" | ||
| found="$(find . -maxdepth 1 -type f | wc -l | tr -d '[:space:]')" | ||
| if [[ "$found" -ne "$expected" ]]; then | ||
| echo "ERROR: Expected ${expected} digest(s) (one per runner platform), found ${found}." | ||
| echo "RUNNER_PLATFORMS_JSON=${RUNNER_PLATFORMS_JSON}" | ||
| ls -la | ||
| exit 1 | ||
| fi | ||
| echo "[OK] Found ${found}/${expected} digests" | ||
|
|
||
| - name: Create manifest list and push | ||
| env: | ||
| MATRIX_TAGS: ${{ matrix.tags }} | ||
| run: | | ||
| build_month="$(date -u +%Y%m)" | ||
| read -r -a tags_arr <<< "$MATRIX_TAGS" | ||
| extra_tags="" | ||
| if [[ "${#tags_arr[@]}" -ge 2 ]]; then | ||
| extra_tags+=" ${tags_arr[1]}-${build_month}" | ||
| fi | ||
| bash ci/push-manifest.sh "${{ env.DOCKERHUB_REPO }}" "${MATRIX_TAGS}${extra_tags}" "${{ runner.temp }}/digests" | ||
|
|
||
| - name: Inspect image # Purely for debugging | ||
| run: | | ||
| sleep 5 | ||
| docker buildx imagetools inspect ${{ env.DOCKERHUB_REPO }}:${{ env.VERSION }}${{ env.VARIANT == 'alpine' && '-alpine' || ''}} | ||
|
|
||
| dockerHubDescription: | ||
| needs: [merge-manifests] | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout source | ||
| uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 | ||
|
|
||
| - name: Prepare README with prefix (create ./_DOCKER-HUB-README.md ) | ||
| run: bash ci/prepare-dockerhub-readme.sh && ls -la ./_DOCKER-HUB-README.md | ||
|
|
||
| - name: Debug ./_DOCKER-HUB-README.md | ||
| run: cat ./_DOCKER-HUB-README.md | ||
| - name: Update Docker Hub Description | ||
| uses: peter-evans/dockerhub-description@1b9a80c056b620d92cedb9d9b5a223409c68ddfa # v5.0.0 | ||
| with: | ||
| username: ${{ secrets.DOCKERHUB_USERNAME }} | ||
| password: ${{ secrets.DOCKERHUB_ACCESS_TOKEN }} | ||
| repository: ${{ env.DOCKERHUB_REPO }} | ||
| short-description: "${{ env.DOCKERHUB_SHORT_DESCRIPTION }}" | ||
| readme-filepath: ./_DOCKER-HUB-README.md | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,118 @@ | ||
| #!/usr/bin/env bash | ||
| # | ||
| # matrix.sh - Parse matrix.yml and output build targets for CI workflows | ||
| # | ||
| # Called by: .github/workflows/*.yml | ||
| # Outputs: BUILD_TARGETS and BUILD_INCLUDE for GitHub Actions matrix strategy | ||
| # | ||
| set -Eeuo pipefail | ||
|
|
||
| # --- Logging (CI-only, no colors) --- | ||
| log_info() { echo "[INFO] $*" >&2; } | ||
| log_warn() { echo "[WARN] $*" >&2; } | ||
| log_error() { echo "[ERROR] $*" >&2; } | ||
| die() { log_error "$1"; exit "${2:-1}"; } | ||
|
|
||
| repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" | ||
| cd "$repo_root" | ||
|
|
||
| readonly MATRIX_FILE="matrix.yml" | ||
|
|
||
| # Environment variables (set by CI workflow) | ||
| runner_platforms_json="${RUNNER_PLATFORMS_JSON:-}" | ||
| github_output_file="${GITHUB_OUTPUT:-}" | ||
|
|
||
| # Write a line to GitHub Actions output file, or stdout if not in CI | ||
| set_github_output() { | ||
| local output_line="$1" | ||
| if [[ -n "$github_output_file" ]]; then | ||
| echo "$output_line" >> "$github_output_file" | ||
| else | ||
| echo "$output_line" | ||
| fi | ||
| } | ||
|
|
||
| # --- Input Validation --- | ||
| if [[ ! -f "$MATRIX_FILE" ]]; then | ||
| die "$MATRIX_FILE not found in repo root" | ||
| fi | ||
|
|
||
| if [[ -z "$runner_platforms_json" ]]; then | ||
| die "RUNNER_PLATFORMS_JSON environment variable is required" | ||
| fi | ||
|
|
||
| if ! command -v ruby >/dev/null 2>&1; then | ||
| die "ruby is required to parse ${MATRIX_FILE}" | ||
| fi | ||
|
|
||
| # --- Parse Matrix File --- | ||
| build_targets="$( | ||
| ruby -ryaml -rjson -e ' | ||
| matrix = YAML.load_file(ARGV.fetch(0)) | ||
| targets = matrix.fetch("build_targets") | ||
| puts JSON.generate(targets) | ||
| ' "$MATRIX_FILE" | ||
| )" | ||
| set_github_output "BUILD_TARGETS=$build_targets" | ||
|
|
||
| # Expand build_targets with runner platforms to create full build matrix. | ||
| # Each target is combined with each platform to create BUILD_INCLUDE entries. | ||
| runner_platforms="$(jq -c '.' <<< "$runner_platforms_json")" | ||
| build_include="$(jq -c --argjson platforms "$runner_platforms" ' | ||
| [ .[] as $combo | $platforms[] | $combo + {"runner-platform": .} ] | ||
| ' <<< "$build_targets")" | ||
| set_github_output "BUILD_INCLUDE=$build_include" | ||
|
|
||
| target_count="$(jq 'length' <<< "$build_targets")" | ||
| include_count="$(jq 'length' <<< "$build_include")" | ||
| log_info "Loaded BUILD_TARGETS with ${target_count} entries" | ||
| log_info "Expanded BUILD_INCLUDE with ${include_count} entries (targets x platforms)" | ||
|
|
||
| # --- Validation --- | ||
| log_info "Validating ./${MATRIX_FILE}..." | ||
|
|
||
| # 1. Check build_targets is not empty | ||
| if [[ "$target_count" -eq 0 ]]; then | ||
| die "matrix.yml has no build_targets" | ||
| fi | ||
|
|
||
| # 2. Check required fields: postgres, postgis, variant, tags (all must be non-empty) | ||
| invalid_entries="$(jq -c ' | ||
| [ .[] | select( | ||
| .postgres == null or .postgres == "" or | ||
| .postgis == null or .postgis == "" or | ||
| .variant == null or .variant == "" or | ||
| .tags == null or .tags == "" | ||
| )] | ||
| ' <<< "$build_targets")" | ||
|
|
||
| invalid_count="$(jq 'length' <<< "$invalid_entries")" | ||
| if [[ "$invalid_count" -gt 0 ]]; then | ||
| log_error "Found ${invalid_count} entries with missing or empty required fields (postgres/postgis/variant/tags):" | ||
| jq '.' <<< "$invalid_entries" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| # 3. Verify exactly one entry has 'latest' tag (prevents accidental duplicate latest) | ||
| latest_count="$(jq ' | ||
| [ .[] | select(.tags | tostring | test("(^| )latest( |$)")) ] | length | ||
| ' <<< "$build_targets")" | ||
|
|
||
| if [[ "$latest_count" -ne 1 ]]; then | ||
| log_error "Expected exactly 1 entry with 'latest' tag, found: $latest_count" | ||
| jq -r '.[] | select(.tags | tostring | test("(^| )latest( |$)"))' <<< "$build_targets" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| # 4. Verify exactly one entry has 'alpine' tag (the alpine equivalent of 'latest') | ||
| alpine_count="$(jq ' | ||
| [ .[] | select(.tags | tostring | test("(^| )alpine( |$)")) ] | length | ||
| ' <<< "$build_targets")" | ||
|
|
||
| if [[ "$alpine_count" -ne 1 ]]; then | ||
| log_error "Expected exactly 1 entry with 'alpine' tag, found: $alpine_count" | ||
| jq -r '.[] | select(.tags | tostring | test("(^| )alpine( |$)"))' <<< "$build_targets" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| log_info "[OK] matrix.yml valid: ${target_count} targets, all have required fields, 1 'latest' tag, 1 'alpine' tag" |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use a run-attempt-specific artifact name here, because
actions/upload-artifactfails when an artifact with the same name already exists in the same workflow run. This workflow keys the name only bygithub.run_id, so re-running failed jobs (or re-running the full workflow) for the same run can hit a 409 conflict on this upload step and block manifest merge/publish even when the rebuilt image is valid.Useful? React with 👍 / 👎.