ceph-perf: tighten PR regression gate by knrt10 · Pull Request #2566 · ceph/ceph-build

knrt10 · 2026-04-07T05:33:42Z

This describes the perf PR job as a release regression check, pin the benchmark definition to ceph-main, and surface the perf regression status in GitHub. This also reenables the ceph-perf-pull-request

Fixes: https://tracker.ceph.com/issues/75665

Copilot

Pull request overview

Updates the ceph-perf pull request performance job to behave as a stricter release-style regression gate by pinning the benchmark definition to ceph-main and surfacing regression status directly on GitHub PRs.

Changes:

Parameterize CBT workload selection in the run-cbt macro and pin the benchmark YAML to ceph-main.
Add/refresh GitHub PR status context strings to make perf regression results visible in the PR UI.
Re-enable the perf PR job template (disabled: false) and clarify the job’s purpose via description.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Matan-B

Few pieces still missing here. I’m not sure this would work with older compilers and exiting CBT support. I would suggest trying to run some of the tests locally first to see if the numbers make sense before we start blocking PRs with this job.
Is this check required once enabled?

We should probably also add some details to Crimson’s dev guide, how the test works, when a PR is blocked (under which cases) and where the results are stored. This should help devs mitigate failures.

Matan-B · 2026-04-07T07:22:03Z

                --cmake-args "-DCMAKE_CXX_COMPILER=$cxx_compiler -DCMAKE_C_COMPILER=$c_compiler -DCMAKE_BUILD_TYPE=Release -DWITH_CRIMSON=ON -DWITH_TESTS=OFF" \
                vstart-base crimson-osd
-              src/script/run-cbt.sh --build-dir $PWD/build --source-dir $PWD --cbt ${{WORKSPACE}}/cbt -a $archive_dir src/test/crimson/cbt/radosbench_4K_read.yaml
+              src/script/run-cbt.sh --build-dir $PWD/build --source-dir $PWD --cbt ${{WORKSPACE}}/cbt -a $archive_dir {benchmark-yaml}


run-cbt is outdated with crimson usage and still relies on cyanstore, which is no longer relevant for our purposes.

Matan-B

I'm not sure I understand if this is ready for review or was tested locally.
If it was, can you please share examples of how the results comparison work?

The clang version that was already mentioned in the previous review would still FTBFS.
CBT is still not updated for the relevant usage with Crimson.
Under which cases a PR is considered blocked (dev docs)?
It is also not clear what the strategy is here - is this intended to run only against Crimson PRs?

Please make sure there is a solid plan for how this is going to be implemented. Starting with a draft PR is also a good approach, but please be clear about the draft stage (what is missing, what is not yet tested/verified and what the next steps are or what is working so far). Otherwise, this PR does not seem ready for review.

My suggestion here would be to enable this check as a non-required PR check and iterate on it until it can be re-enabled as a required one. Alternatively, running parts of the jobs locally to make sure all is working as expected.
I would be happy to help in any way if you have any questions.

Matan-B · 2026-04-13T10:13:41Z

          cxx_compiler=g++
          c_compiler=gcc
-          for i in $(seq 15 -1 10); do
+          for i in $(seq 18 -1 12); do


Crimson would FTBFS with clang less than 19

Matan-B · 2026-04-13T10:15:06Z

-          fi
+          export WITH_CRIMSON=true
+          timeout 7200 src/script/run-make.sh \
+            --cmake-args "-DCMAKE_CXX_COMPILER=$cxx_compiler -DCMAKE_C_COMPILER=$c_compiler -DCMAKE_BUILD_TYPE=Release -DWITH_CRIMSON=ON -DWITH_TESTS=OFF" \


We should be able to disable more components if we would like to have a minimal build.

This describes the perf PR job as a release regression check, pin the benchmark definition to ceph-main, and surface the perf regression status in GitHub. This also reenables the ceph-perf-pull-request Fixes: https://tracker.ceph.com/issues/75665 Signed-off-by: Kautilya Tripathi <kautilya.tripathi@ibm.com>

tchaikov · 2026-04-30T13:51:31Z

+          cmake_args="-DCMAKE_CXX_COMPILER=$cxx_compiler -DCMAKE_C_COMPILER=$c_compiler -DCMAKE_BUILD_TYPE=Release -DWITH_CRIMSON=ON -DWITH_TESTS=OFF $CEPH_PERF_EXTRA_CMAKE_ARGS"
+          timeout 7200 src/script/run-make.sh \
+            --cmake-args "$cmake_args" \
+            vstart-base


why did you drop crimson-osd from the build target list? please note vstart-base alone does not build crimson-osd.

tchaikov · 2026-04-30T13:53:33Z

      - run-cbt:
          src-dir: "ceph-pr"
-          osd-flavor: '{osd-flavor}'
+          benchmark-yaml: "$WORKSPACE/ceph-pr/src/test/crimson/cbt/radosbench_4K_read.yaml"


i think both runs should use the same yaml file, so that it's an apple-to-apple comparison.

tchaikov · 2026-04-30T13:58:18Z

-              timeout 7200 src/script/run-make.sh --cmake-args "-DCMAKE_BUILD_TYPE=Release -DWITH_TESTS=OFF" vstart-base
-              src/script/run-cbt.sh --build-dir $PWD/build --source-dir $PWD --cbt ${{WORKSPACE}}/cbt -a $archive_dir src/test/crimson/cbt/radosbench_4K_read.yaml --classical
-          fi
+          export WITH_CRIMSON=true


does crimson require a certain version of gcc? i am asking because, if clang-19..22 is not installed, cxx_compiler stays g++ . and IIRC, seastar is only compatible with the latest two major versions of GCC and Clang.

tchaikov · 2026-04-30T13:59:27Z

          case $ID in
          debian|ubuntu)
-              sudo env DEBIAN_FRONTEND=noninteractive apt-get install -y python3-yaml python3-lxml python3-prettytable clang-12
+              sudo env DEBIAN_FRONTEND=noninteractive apt-get install -y python3-yaml python3-lxml python3-prettytable


clang-12 is dropped, but do we need to install a updated clang here?

Copilot AI review requested due to automatic review settings April 7, 2026 05:33

Copilot started reviewing on behalf of knrt10 April 7, 2026 05:34 View session

knrt10 force-pushed the reenable-ceph-perf-check branch from 25b9570 to 18e33c0 Compare April 7, 2026 05:34

Copilot AI reviewed Apr 7, 2026

View reviewed changes

Comment thread ceph-perf-pull-requests/config/definitions/ceph-perf-pull-requests.yml Outdated

Comment thread ceph-perf-pull-requests/config/definitions/ceph-perf-pull-requests.yml Outdated

knrt10 force-pushed the reenable-ceph-perf-check branch from 18e33c0 to c517923 Compare April 7, 2026 05:42

Matan-B requested changes Apr 7, 2026

View reviewed changes

Matan-B added this to Crimson Apr 7, 2026

Matan-B moved this to In Progress in Crimson Apr 7, 2026

Matan-B requested a review from perezjosibm April 7, 2026 09:51

knrt10 force-pushed the reenable-ceph-perf-check branch from c517923 to 7359ce6 Compare April 8, 2026 06:06

Matan-B requested changes Apr 13, 2026

View reviewed changes

knrt10 marked this pull request as draft April 13, 2026 13:22

knrt10 force-pushed the reenable-ceph-perf-check branch from 7359ce6 to 94866b0 Compare April 14, 2026 00:11

knrt10 force-pushed the reenable-ceph-perf-check branch from 94866b0 to 064d528 Compare April 14, 2026 00:11

tchaikov reviewed Apr 30, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ceph-perf: tighten PR regression gate#2566

ceph-perf: tighten PR regression gate#2566
knrt10 wants to merge 1 commit into
mainfrom
reenable-ceph-perf-check

knrt10 commented Apr 7, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Matan-B left a comment

Uh oh!

Uh oh!

Matan-B Apr 7, 2026

Uh oh!

Uh oh!

Uh oh!

Matan-B left a comment •

edited

Loading

Uh oh!

Matan-B Apr 13, 2026

Uh oh!

Matan-B Apr 13, 2026

Uh oh!

tchaikov Apr 30, 2026

Uh oh!

tchaikov Apr 30, 2026

Uh oh!

tchaikov Apr 30, 2026

Uh oh!

tchaikov Apr 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

knrt10 commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Matan-B left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Matan-B Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Matan-B left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Matan-B Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Matan-B Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

tchaikov Apr 30, 2026

Choose a reason for hiding this comment

Uh oh!

tchaikov Apr 30, 2026

Choose a reason for hiding this comment

Uh oh!

tchaikov Apr 30, 2026

Choose a reason for hiding this comment

Uh oh!

tchaikov Apr 30, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

knrt10 commented Apr 7, 2026 •

edited

Loading

Matan-B left a comment •

edited

Loading