Skip to content

Commit ae6c130

Browse files
committed
unix: parallelize PGO profile gathering
Contrary to popular belief, the system doesn't need to be "idle" to collect profiling data. See inline comment + https://github.com/llvm/llvm-project/blob/main/llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp for the PGO implementation for the evidence.
1 parent e1e1b5b commit ae6c130

1 file changed

Lines changed: 13 additions & 0 deletions

File tree

cpython-unix/build-cpython.sh

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -591,6 +591,19 @@ fi
591591
# Define the base PGO profiling task, which we'll extend below with ignores
592592
export PROFILE_TASK='-m test --pgo'
593593

594+
# Run tests in parallel to reduce wall time.
595+
#
596+
# LLVM's PGO instruments compiled code to increment counters to track
597+
# call count. Same story for BOLT in instrumented mode, which we also use.
598+
# This approach is in contrast to sample based profiling where the program is
599+
# sampled periodically to see which code is active. In instrumented mode,
600+
# the wall time execution doesn't matter: a counter will be incremented
601+
# regardless of how fast the code runs.
602+
#
603+
# Use of instrumented mode means that it is safe to profile in parallel
604+
# and there will be no loss in profile quality.
605+
PROFILE_TASK="${PROFILE_TASK} -j ${NUM_CPUS}"
606+
594607
# On 3.14+ `test_strftime_y2k` fails when cross-compiling for `x86_64_v2` and `x86_64_v3` targets on
595608
# Linux, so we ignore it. See https://github.com/python/cpython/issues/128104
596609
if [[ -n "${PYTHON_MEETS_MINIMUM_VERSION_3_14}" && -n "${CROSS_COMPILING}" && "${PYBUILD_PLATFORM}" != macos* ]]; then

0 commit comments

Comments
 (0)