Skip to content

feat: add Pulp upload scripts and pipeline integration#2570

Open
vamahaja wants to merge 1 commit into
mainfrom
upload-to-pulp
Open

feat: add Pulp upload scripts and pipeline integration#2570
vamahaja wants to merge 1 commit into
mainfrom
upload-to-pulp

Conversation

@vamahaja
Copy link
Copy Markdown
Member

This PR will introduce an optional path to publish built RPM/DEB artifacts to Pulp alongside the existing Chacra flow.

  • Introduce a PULP_UPLOAD boolean parameter (default false) on the ceph-dev job.
  • Add scripts/setup_pulp.sh to install pulp-cli (with pulp-cli-deb) via uv and write client config from Jenkins credentials.
  • Add scripts/pulp_upload.sh to upload discovered packages, attach them to the target repository, create a publication, and publish a distribution with project/version/ref/arch/sha labels.
  • Update the Jenkinsfile to install/configure the Pulp client when forcing a rebuild with Pulp upload while Chacra already has artifacts, and to invoke pulp_upload.sh after chacra_upload.sh when THROWAWAY is not true and PULP_UPLOAD is true.

@vamahaja vamahaja requested review from tchaikov and zmc April 23, 2026 16:41
Comment thread ceph-dev-pipeline/build/Jenkinsfile Outdated
@djgalloway
Copy link
Copy Markdown
Contributor

jenkins retest

@djgalloway
Copy link
Copy Markdown
Contributor

My comment is a nit. I'd like to push this to Jenkins to make sure we don't run over the character limit we saw a few months ago but otherwise okay with this. Exciting stuff! 👍

Copy link
Copy Markdown
Member

@zmc zmc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks great! I agree with David's suggestion. I think we should likely have PULP_URL be a parameter defaulting to the host we have in openshift.

@vamahaja vamahaja force-pushed the upload-to-pulp branch 3 times, most recently from 210595f to 5cee478 Compare April 27, 2026 07:12
Comment thread ceph-dev-pipeline/build/Jenkinsfile
@vamahaja vamahaja force-pushed the upload-to-pulp branch 6 times, most recently from 93eb758 to 4bb903f Compare April 28, 2026 08:49
@djgalloway
Copy link
Copy Markdown
Contributor

@vamahaja is this ready now?

@vamahaja
Copy link
Copy Markdown
Member Author

vamahaja commented May 4, 2026

@vamahaja is this ready now?

@djgalloway testing couple of scenarios, will complete it ASAP

@vamahaja vamahaja force-pushed the upload-to-pulp branch 12 times, most recently from 58c7717 to e3299fc Compare May 7, 2026 15:09
Add an optional path to publish built RPM/DEB artifacts to Pulp alongside the
existing Chacra flow.
  - Introduce a `PULP_UPLOAD` boolean parameter (default false) on the `ceph-dev` job.
  - Add `scripts/setup_pulp.sh` to install `pulp-cli` (with pulp-cli-deb) via uv and
    write client config from Jenkins credentials.
  - Add `scripts/pulp_upload.sh` to upload discovered packages, attach them to the
    target repository, create a publication, and publish a distribution with
    project/version/ref/arch/sha labels.
  - Update the Jenkinsfile to install/configure the Pulp client when forcing a
    rebuild with Pulp upload while Chacra already has artifacts, and to invoke
    `pulp_upload.sh` after `chacra_upload.sh` when `THROWAWAY` is not `true` and
    `PULP_UPLOAD` is `true`.

Signed-off-by: Vaibhav Mahajan <vamahaja@redhat.com>
@vamahaja
Copy link
Copy Markdown
Member Author

vamahaja commented May 8, 2026

Comment thread scripts/pulp_upload.sh
Comment on lines +13 to +33
resolve_branch_from_ceph_version() {
local branch=""
if [[ $CEPH_VERSION =~ ^21.* ]]; then
branch="main"
elif [[ $CEPH_VERSION =~ ^20.* ]]; then
branch="tentacle"
elif [[ $CEPH_VERSION =~ ^19.* ]]; then
branch="squid"
elif [[ $CEPH_VERSION =~ ^18.* ]]; then
branch="reef"
elif [[ $CEPH_VERSION =~ ^17.* ]]; then
branch="quincy"
elif [[ $CEPH_VERSION =~ ^16.* ]]; then
branch="pacific"
else
log "ERROR: Unsupported Ceph version '${CEPH_VERSION}'" \
"(expected 16.x–20.x for branch mapping)"
exit 1
fi
printf '%s\n' "$branch"
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to reuse https://github.com/ceph/ceph-build/blob/main/scripts/build_utils.sh#L13-L78? Having another distro map hard coded somewhere is not scalable.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we ought to drop a yaml or json file in, say, the repo root for this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants