Skip to content

ci(windows): replace S3 MSYS2 zip with cached pacman install of clang 18.1.8#6060

Merged
Fedr merged 51 commits into
masterfrom
ci/use-runner-msys2-pin-clang18
May 12, 2026
Merged

ci(windows): replace S3 MSYS2 zip with cached pacman install of clang 18.1.8#6060
Fedr merged 51 commits into
masterfrom
ci/use-runner-msys2-pin-clang18

Conversation

@Fedr
Copy link
Copy Markdown
Contributor

@Fedr Fedr commented May 7, 2026

Why

Master's Windows binding-generation step downloads msys64_meshlib_mrbind.zip (~725 MB) from vcpkg-export.s3.us-east-1.amazonaws.com. That has been observed:

  • Returning 404 / connection-resetting on transient days, breaking CI for everyone.
  • Containing a snapshot whose contents drift only when someone rebuilds and re-uploads the zip — there's no source of truth for "what's in it".

Replace it with a pacman-driven install on the runner's preinstalled C:\msys64, pinned to a sha256-verified lockfile, with GitHub Actions cache for the package archives.

Clang must be pinned to the same version master ships — the binding generator's output regressed on a newer clang during the mrbind experiments that surfaced this work (ImportError: initialization failed from MRTest's embedded-python smoke test). The pin chosen is clang 18.1.8-2 — same version the historical S3 zip shipped — and the entire clang-18-era runtime stack travels with it (pinning only clang causes STATUS_DLL_NOT_FOUND from cc.exe because the rest of MSYS2's runtime drifts forward).

Changes

Workflow (build-test-windows.yml, pip-build.yml)

  • Install MSYS2 for MRBind step is a composite action (.github/actions/install-msys2-mrbind) that now: (a) restores scripts/mrbind/msys2_packages from GitHub Actions cache keyed on the lockfile hash, then (b) pacman -Us a 49-package lockfile against the runner's preinstalled MSYS2. Was previously the S3-zip-downloading composite of the same name. Both build-test-windows.yml and pip-build.yml use uses: ./.github/actions/install-msys2-mrbind and share cache entries (same key + path).
  • MSYS2_DIR env var pointed at C:\msys64 (was the synthetic C:\msys64_meshlib_mrbind).
  • Install gettext utilities step removed — mingw-w64-clang-x86_64-gettext-tools-0.22.5-2 is now pinned in the clang18 lockfile alongside everything else (its three deps — gettext-runtime, libiconv, gcc-libs — are all satisfied; gcc-libs is provides=d by our pinned libc++-18.1.8-2). One fewer step, msgfmt always available, no pacman -S against drifting mirrors.

Lockfile + scripts (scripts/mrbind/)

  • New msys2_package_hashes_clang18.txt — 49 packages, all clang-18 era. URLs are derived from each filename's package-type prefix (matching msys2_remember_current_packages.sh's writer-side mapping), so no separate URLs lockfile is needed.
  • msys2_download_packages.sh and msys2_install_packages.sh parameterized by suffix (_clang18 here, default '' for the existing CI-zip-refresh path that master still uses).
  • Download script uses wget -nc so a cache hit skips every file without contacting the server; install script's sha256sum -c is the integrity gate.
  • Install script strips CRs before feeding to sha256sum -c (Windows runners' default core.autocrlf checks the LF lockfile out as CRLF; sha256sum doesn't tolerate trailing \r in filenames).

Relationship to master's lockfile

scripts/mrbind/msys2_package_hashes_clang18.txt is a strict, byte-for-byte subset of master's scripts/mrbind/msys2_package_hashes.txt:

  • All 49 packages in the clang18 lockfile appear verbatim in master's 139-package lockfile — same filename, same version, same sha256.
  • The 91 packages that are in master but not in clang18 are the rest of the MSYS2 base set (bash, coreutils, msys2-runtime, dash, file, filesystem, findutils, gawk, grep, gzip, less, etc.) that the runner image's preinstalled C:\msys64 already provides, so we don't pin them.
  • Bumping the clang stack here amounts to picking the same 49 lines out of a refreshed master msys2_package_hashes.txt next time master regenerates its zip — no risk of hash drift between the two lockfiles for shared packages.

Performance

Combined MSYS2-setup time across the 9-config Windows matrix on windows-2022 (range across configs / mean), measured on the latest full-matrix run of each side:

Install MSYS2 + Install gettext utilities Combined
Master (S3 zip + runtime pacman -S gettext-tools) 54–74 s / 64 s 6–9 s / 7 s 60–81 s / 71 s
#6060 first run, no cache ~110 s — (folded in) ~110 s
#6060 cache hit 46–70 s / 56 s — (folded in) 46–70 s / 56 s

Net ~15 s mean speedup vs master per Windows job on cache hit (~−8 s from cache replacing S3 zip download, ~−7 s from gettext-tools fold-in). Cache-miss runs are slower than master (~110 s vs ~71 s mean), but only hit when the lockfile changes — rare given the clang-18 pin.

Cache-hit breakdown of the composite step (~56 s mean):

  • Cache restore (actions/cache@v5): ~3 s
  • MSYS2 first-run init: ~2 s
  • wget -nc (all files present, network-free): ~1 s
  • sha256sum -c + pacman DB load: ~5 s
  • pacman -U extracting 49 archives (including gettext-tools) onto C:\msys64\: ~45 s (dominant, irreducible without restructuring)

AWS cost + reliability

Master fetches msys64_meshlib_mrbind.zip (~725 MB) from vcpkg-export.s3.us-east-1.amazonaws.com on every Windows binding-generation job. After this PR lands:

  • S3 egress for this artifact drops to ~0. Cache-hit reads from GitHub Actions cache (free for the first 10 GB/repo); cache-miss reads pulls 49 packages (700 MB) from repo.msys2.org (community mirror, no cost to MeshInspector). The zip itself can stay parked in S3 storage ($0.02/month) or be deleted as a separate housekeeping step.
  • The transient S3 404 / connection-reset failures that motivated this PR go away — S3 is no longer in the binding-generation hot path. New exposure is to repo.msys2.org, but only on cache miss (lockfile-pinned, rare). wget retries + sha256sum -c provide the same safety net.
  • A rough back-of-envelope (20 CI runs/day × 4 Windows configs × 30 days × 0.725 GB ≈ 1.7 TB/month egress at $0.09/GB beyond the first 100 GB free) suggests **$150/month** in S3 transfer for this artifact alone. Actual figure depends on CI volume.

Risk / open items

  • Mirror dependency moves from S3 (private, no SLA) to repo.msys2.org (community-maintained, also no SLA). If MSYS2 ever GCs the pinned 18.1.8-2 archives, lockfile URLs need updating. Lockfile-and-cache-key design makes that a one-touch change.
  • Cache miss cost is ~110 s vs master's ~71 s mean — only matters when the lockfile changes (rare, since clang-18 is pinned).
  • Long-term: bump the lockfile in lockstep when master moves off clang 18 (e.g. picks the corresponding 49 lines out of a refreshed msys2_package_hashes.txt).

Test plan

  • Windows binding-generation matrix passes — 9/9 on the latest full-matrix run (msvc-2019 / msvc-2022 × Debug/Release × CMake/MSBuild + iterator-debug).
  • meshlib.mrmeshpy import works (covered by the verify step from ci: verify meshlib.mrmeshpy import before Unit Tests #6069).
  • Cache hit path validated end-to-end on a re-run of the same lockfile.
  • wget -nc cuts the cache-hit network round-trips from ~28 s to ~1 s (HEAD requests gone).
  • Distribution / upload_artifacts: true flow validated: full 9-config Windows matrix (incl. msvc-2019 Release CMake) passes on run 25653... — gettext-tools-0.22.5-2 installs cleanly via the lockfile, msgfmt available.

Fedr and others added 12 commits April 30, 2026 17:00
Stop fetching the ~725 MB msys64_meshlib_mrbind.zip from S3 (which
has been seen returning 404 in CI) and stop relying on the
install-msys2-mrbind composite action. The windows-2025 runner
image already ships:

  * MSYS2 base at C:\msys64 (pacman 6.1.0)
  * Standalone LLVM 20.1.8 at C:\Program Files\LLVM (unused by
    MRBind, kept here for reference)

For MRBind we still need the MSYS2 -clang64 environment (mrbind
links against MSYS2's libclang/libc++, not the Windows-native
LLVM build). So this change pacman-installs the minimal toolchain
into the preinstalled C:\msys64:

  pacman -Sy --noconfirm --needed make \
    mingw-w64-clang-x86_64-{clang,clang-tools-extra,cmake,ninja,libc++}

and points all subsequent MRBind/binding-generation steps at
C:\msys64 by setting MSYS2_DIR (read by install_mrbind_windows_msys2.bat
and generate_win.bat). GETTEXT_ROOT moves from
C:\msys64_meshlib_mrbind\clang64 to C:\msys64\clang64.

The install-msys2-mrbind composite action is no longer used and is
deleted. install_deps_windows_msys2.bat (local-developer install
path) is unchanged: it still creates a separate
C:\msys64_meshlib_mrbind tree, untouched by this CI change.

Note: this swaps an S3-mirror dependency for an MSYS2-mirror
dependency. It also moves clang from a pinned 18.1.8 to whatever
mingw-w64-clang-x86_64-clang is currently in the clang64 repo
(22.x at the time of writing). MRBind's Windows path uses bare
`clang++` (no version suffix) so this works without further
changes; if a specific clang version becomes required later, pin
via pacman or use the MSYS2 archive.
mrbind's CMakeLists does find_package(Clang REQUIRED), which
transitively requires LLVMConfig.cmake from the llvm dev package.
That package is listed only as an *optional* dep of clang, so
pacman did not pull it in automatically — Build MRBind failed
with:

  Could not find a package configuration file provided by "LLVM"
  (requested version 22.1.4)

Add mingw-w64-clang-x86_64-llvm to the install list explicitly.
Two bugs combined to make a real failure look green in
https://github.com/MeshInspector/MeshLib/actions/runs/25392220595/job/74469566838:

1. The 'Generate and build Python bindings' step in
   build-test-windows.yml didn't carry the MSYS2_DIR=C:\msys64 env
   override that the other generate_win.bat / install_mrbind_windows_msys2.bat
   callers on this branch already have. So the script saw the
   default MSYS2_DIR=C:\msys64_meshlib_mrbind, which doesn't exist
   on the runner anymore.

2. generate_win.bat printed 'MSYS2 was NOT found' and then fell
   through to a normal exit, returning code 0. The step's `call`
   inherited that 0, GitHub Actions marked the step green, the
   .pyd never got built, and the failure only surfaced four steps
   later when 'Unit Tests' couldn't load mrmeshpy.pyd
   (LoadLibrary error 126).

Fix both:

* Add `MSYS2_DIR: C:\msys64` to the 'Generate and build Python
  bindings' step's env block, matching Build MRBind / Generate C
  bindings / Generate C# bindings.

* Replace generate_win.bat's silent fall-through with `exit /b 1`
  after the missing-MSYS2 message. Mirrors what
  install_mrbind_windows_msys2.bat already does on the same
  condition. Future workflow callers that forget MSYS2_DIR will
  fail at the right step instead of being papered over until
  something later trips on the missing artifacts.
Add a post-Unit-Tests step (always(), continue-on-error) that
captures state when MRTest.exe's embedded python smoke test
trips `ImportError: initialization failed` from `<string>(2):
<module>`. CPython's wrapper hides the real cause; this dumps:

* contents of source\x64\<config>\ (DLLs, .pyd, EXE) and
  source\x64\<config>\meshlib\
* dumpbin /exports of MR{Test,EmbeddedPython,Python}.{exe,dll}
  and each *.pyd, filtered to PyInit_* — confirms the module
  init function is exported
* dumpbin /dependents of the same, filtered to python*/MR*/CRT —
  shows which python.dll the bindings and the embedded
  interpreter resolve (catches ABI mismatches between vcpkg's
  Python 3.12 and any other python.dll on PATH)
* `py -0p` inventory of available pythons
* a standalone `py -3.12 -c "import meshlib.mrmeshpy"` with
  full traceback — gives the real Python exception instead of
  CPython's wrapped "initialization failed"
* python*.dll listings under C:\vcpkg\installed\...\bin and
  `where.exe` for python3.dll / python312.dll

Step is purely informational; it never fails the job.
Diagnostic on the previous run (PR #6021) revealed that MRBind's
bindings regress when generated with the current MSYS2 mirror's
clang (22.1.4): the `std_vector_const_Mesh` registration is
missing from the generated `mrmeshpy.pyd`, so module init fails
with `AttributeError: module 'meshlib.mrmeshpy' has no attribute
'std_vector_const_Mesh'` — wrapped by CPython as the opaque
`ImportError: initialization failed` MRTest's embedded-python
smoke test surfaces. The historical msys64_meshlib_mrbind.zip
bundle pinned clang 18.1.8-2 specifically because mrbind is
sensitive to libclang's AST shape.

Reproduce that pin without depending on the S3 bundle:

* `Install MRBind toolchain` step now downloads the 9 clang-stack
  .pkg.tar.zst files from `repo.msys2.org/mingw/clang64/` (those
  versioned packages are still served): clang, clang-libs,
  clang-tools-extra, llvm, llvm-libs, libc++, compiler-rt, lld,
  libunwind — all 18.1.8-2-any.
* `pacman -Sy` to refresh DBs, `pacman -S` cmake/ninja/make from
  the current mirror (build tools don't influence binding output
  and don't depend on clang), then `pacman -U` (no --needed) of
  the staged 18.1.8-2 archives so any newer libc++/etc. pulled in
  as cmake/ninja deps gets downgraded to match the pinned clang.
* Same change applied to `pip-build.yml`'s `windows-pip-build`
  job.

Trades the original `vcpkg-export.s3` dependency for
`repo.msys2.org` and locks clang at 18.1.8-2 indefinitely
(unless MSYS2 GCs the historical packages, in which case the
URLs need updating to a still-served version or to a private
mirror).
Fedr added 4 commits May 7, 2026 13:54
Previous run hit a dep-graph conflict downgrading libc++ from 22
back to 18:

  error: failed to prepare transaction (could not satisfy dependencies)
  :: installing mingw-w64-clang-x86_64-libc++ (18.1.8-2) breaks
     dependency 'mingw-w64-clang-x86_64-cc-libs' required by
     mingw-w64-clang-x86_64-cmake (and 10 other packages)

`mingw-w64-clang-x86_64-cc-libs` is a virtual marker recently
added to MSYS2 that newer libc++ provides; libc++ 18.1.8-2 predates
it. Tell pacman to treat cc-libs as still installed via
`--assume-installed mingw-w64-clang-x86_64-cc-libs=22.1.4-1` so the
downgrade transaction goes through. The actual runtime symbols
(libc++.dll etc.) ship with libc++ 18 regardless; cc-libs is just
a marker.
Previous run downgraded libc++ from 22 to 18 successfully (with
--assume-installed cc-libs), but cmake.exe and ninja.exe — built
against libc++ 22 ABI — couldn't load against libc++ 18 and exited
127 inside the -clang64 shell, taking down Build MRBind:

  Found MSYS2 at `C:\msys64`.
  The system cannot find the file specified.
  ##[error]Process completed with exit code 127.

libc++'s ABI is forward-loose (older binaries against newer libc++
work) but not backward (newer binaries depend on symbols/layout
not in old libc++). So drop libc++ and libunwind from the pinned
list and only keep the clang frontend / llvm / lld / compiler-rt
packages at 18.1.8-2:

* clang/clang-libs/clang-tools-extra
* llvm/llvm-libs
* compiler-rt
* lld

That keeps mrbind running through the clang 18 frontend (which is
what changed binding output and caused the std_vector_const_Mesh
loss) while leaving the runtime fresh enough that cmake/ninja keep
working. Also drop the --assume-installed cc-libs trick — no longer
needed since we no longer downgrade the libc++ that provides cc-libs.
Use `pacman -U --needed` so any future already-installed package
isn't downgrade-attempted.
Pinning only clang/llvm/lld at 18.1.8-2 wasn't enough — clang's
cc.exe failed to start with STATUS_DLL_NOT_FOUND (exit 127) when
the rest of the runtime (libwinpthread-git, crt-git, headers-git,
libc++) was at current versions. clang 18 binaries were built
against a specific late-2024 ABI snapshot that includes those
matching git revisions, and that snapshot is internally consistent
in a way "clang 18 + everything else current" isn't.

Pin the WHOLE mingw/clang64 toolchain to the era of clang 18.1.8-2
— the same 47-package set the historical msys64_meshlib_mrbind.zip
bundle shipped with (matching libwinpthread-git/crt-git/headers-git
12.0.0.r406, libc++ 18.1.8-2, libunwind 18.1.8-2, cmake 3.31.1-1,
ninja 1.12.1-1, etc.). All 47 URLs are still served on
mirror.msys2.org.

* New file: scripts/mrbind/msys2_package_urls_clang18.txt — the
  pinned URL list, extracted from the msys2_package_urls.txt
  snapshot at commit f0fe83c (filtered to mingw/clang64 only).
* Workflow steps: download all 47, then a single
  `pacman -U --noconfirm --needed --overwrite '*'` so dep
  resolution sees a self-consistent set with no version skews and
  no cc-libs marker confusion. --overwrite '*' tolerates files
  already on disk from the runner image's preinstalled MSYS2.
The two mrbind bumps on this branch (1f98820 'Update mrbind.' and
8070543 'Hopefully fix ambiguous Python names.') brought in newer
mrbind that's coupled to clang 22 — the clang frontend changes the
maintainer also reflected in the lockfile bump. Pinning clang back to
18.1.8-2 in CI but keeping the newer mrbind led to incompatibilities
(`AttributeError: module 'meshlib.mrmeshpy' has no attribute
'std_vector_const_Mesh'` resurfacing as a clang-version-specific PCH
macro-mismatch error from `MB_PB11_ADJUST_NAMES`).

Reset thirdparty/mrbind back to commit 2978653c3 (master's tip
before this PR) so the binding generation sees the same mrbind it
was working with under the original bundle's clang 18.1.8-2.
Fedr and others added 8 commits May 8, 2026 13:33
PR #6060's previous form had the download + install logic open-coded
in the workflow YAML — a PowerShell foreach loop calling
Invoke-WebRequest per package, then a single inline `pacman -U`
shelled out via msys2_shell.cmd. That duplicates the role of the
existing `scripts/mrbind/msys2_download_packages.sh` and
`scripts/mrbind/msys2_install_packages.sh` and skips the sha256
verification the install script does.

Generalize both scripts to take an optional suffix argument (default
empty, preserving the existing master-lockfile behavior). Suffix
`_clang18` → reads `msys2_package_urls_clang18.txt` /
`msys2_package_hashes_clang18.txt` instead.

* `scripts/mrbind/msys2_download_packages.sh _clang18`
  → `wget` the URLs from `msys2_package_urls_clang18.txt` into
    `msys2_packages/`.
* `scripts/mrbind/msys2_install_packages.sh _clang18`
  → `sha256sum -c msys2_package_hashes_clang18.txt`,
    then `pacman -U --noconfirm --needed` over the verified files.

Generate `msys2_package_hashes_clang18.txt` once now (47 entries,
sha256s computed from the live mirror.msys2.org content).

Workflow steps in build-test-windows.yml and pip-build.yml shrink to
two short msys2_shell.cmd invocations: one to seed pacman's DBs
(`pacman -Sy`), and one to run download+install via the scripts.
The PowerShell loop, the staging dir, and the inline pacman -U are
gone.
Last run failed with `bash: scripts/mrbind/msys2_download_packages.sh:
No such file or directory` — by default `msys2_shell.cmd -c` starts
the shell in the user's MSYS2 home (~ → /home/runneradmin), not the
workflow's checkout dir. Add `-here` to keep the checkout dir as the
shell's cwd so the relative `scripts/mrbind/...` paths the workflow
passes resolve.

Same `-here` flag the existing install_mrbind_windows_msys2.bat
already uses for the same reason.
The lockfile is stored as LF in git but Windows runners' default
autocrlf checks it out as CRLF. sha256sum -c then fails to open all
47 listed files because each filename has a trailing carriage return.
Strip CRs as we read the lockfile so it's robust to either checkout
encoding.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…f -D

Defining MB_PB11_ADJUST_NAMES as a -D flag with backslashes inside
nested shell quotes ('"s/\bMR:://g"') is fragile under MSYS2: different
bash/make versions strip a different number of backslashes between the
PCH-build recipe and the per-fragment-build recipe. Clang then refuses
to reuse the PCH because the macro definitions don't match textually:

    error: definition of macro 'MB_PB11_ADJUST_NAMES' differs between
    the precompiled header ('"s/\bMR:://g"') and the command line
    ('"s/\bMR:://g"')

Move the define into a force-included header, so the value travels
through C source — which has its own well-defined string-literal
quoting — instead of through bash. -include takes a bare filename
(no shell-quoting concerns), and scripts/mrbind/ is already on the
include path via -I$(makefile_dir) in generate.mk.

Other -D macros in compiler_only_flags.txt are unaffected because
their values contain no backslashes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… -include

The previous attempt routed the macro via `-include mrbind_pb11_defines.h`
in compiler_only_flags.txt, but clang requires the PCH-import `-include`
to be the FIRST `-include` on the command line. Adding any other
`-include` ahead of it silently disables the PCH:

    warning: precompiled header '<module>.combined_pch.hpp.gch' was
    ignored because '-include <module>.combined_pch.hpp' is not first
    '-include'
    fatal error: '<module>.combined_pch.hpp' file not found

Inject `#include <mrbind_pb11_defines.h>` into the generated
`<module>.combined.hpp` instead. That file is the PCH source, so the
macro is baked into the PCH itself and every fragment that imports the
PCH inherits the definition — no `-include` needed, no flag-order
conflict, and the macro value still travels via C source rather than
through the bash-quoting layer where backslashes get unevenly stripped.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The mrbind parser is invoked over <module>.combined.hpp using
$(COMPILER_FLAGS), which doesn't have -I$(makefile_dir) — only
$(COMPILER) does. The unguarded `#include <mrbind_pb11_defines.h>` I
added to .combined.hpp therefore broke parsing:

    fatal error: 'mrbind_pb11_defines.h' file not found

The parser doesn't need MB_PB11_ADJUST_NAMES (it parses the C++ AST,
not the regex-walking code path inside mrbind's pybind11/core.h).
Wrap the include in `#ifndef MR_PARSING_FOR_PB11_BINDINGS`, mirroring
the existing guard around `<pybind11/pybind11.h>` two lines down. PCH
build (`MR_COMPILING_PB11_BINDINGS`, no parser flag) still pulls it in
and bakes the macro into the PCH; fragments inherit via PCH-import.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fedr and others added 2 commits May 9, 2026 23:45
The "Generate fresh MSYS2 lockfiles" commit on this branch updated
scripts/mrbind/msys2_package_urls.txt and msys2_package_hashes.txt
(244 lines each) — the unsuffixed lockfiles that drive master's
historical S3-zip refresh procedure. They're orthogonal to this PR,
which only adds the suffixed clang18 pair. Restore them to master's
content so the PR's diff focuses on what it's actually changing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ckfile

Master kept two MSYS2 lockfiles in lockstep: msys2_package_urls.txt
(URL per line, fed to wget -i) and msys2_package_hashes.txt (sha256 +
*msys2_packages/<filename>, fed to sha256sum -c). The URL is derivable
from the filename — `msys2_remember_current_packages.sh` already
encodes the package-type → URL prefix mapping when generating the
URLs file.

Move that same mapping into msys2_download_packages.sh: parse the
hash file's filenames, compute URL = `https://mirror.msys2.org/<prefix>/<filename>`,
write to a temp URL list, and feed that to wget. With this, the
suffixed `msys2_package_urls_clang18.txt` is redundant — drop it.

The unsuffixed `msys2_package_urls.txt` still exists on master (for
historical reasons / the regenerator's output) but is no longer
read by the download script. Cleaning that up is out of scope here.

Cache key (hashFiles(msys2_package_hashes_clang18.txt)) is unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Fedr Fedr added full-ci run all steps test-pip-build Build Python wheels (and discard them) labels May 9, 2026
…iagnostic, restore READMEs

Per review feedback:

  1. Shorten the verbose comments in build-test-windows.yml's cache /
     install steps and in msys2_download_packages.sh /
     msys2_install_packages.sh down to 2-3 lines each.
  2. Rename `Install MRBind toolchain (clang 18.1.8 pinned) into runner
     MSYS2` back to `Install MSYS2 for MRBind` (master's name) — both
     in build-test-windows.yml and pip-build.yml.
  3. Drop the `Diagnostic — Python bindings post-Unit-Tests` step that
     was added during the std_vector_const_Mesh investigation — no
     longer needed, and not present in master.
  4. Restore scripts/mrbind/README-generating.md and
     scripts/mrbind/README-updating-clang.md to master's content.

No behavior change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fedr and others added 6 commits May 10, 2026 00:37
`pacman -S --needed gettext-tools` from the runner's fresh MSYS2 DB
resolves to gettext-tools-1.0 with `libc++ >= 22` as a transitive
dep, and silently upgrades our pinned libc++ 18.1.8-2 → 22.1.4-1.
The next step (Build MRBind) then compiles with clang 18 + libc++
22 — libc++ 22 headers reference `__builtin_clzg/ctzg` which clang
18 doesn't have, so the build fails with screenfuls of `'_Tp' does
not refer to a value` and `use of undeclared identifier
'__builtin_clzg'` errors out of `<bit>` and `<charconv>`.

Master's frozen S3-zip MSYS2 has a stale DB and resolves to an
older gettext-tools that doesn't pull in libc++ 22, which is why
master's distribution flow doesn't trip this.

Pass `--assume-installed mingw-w64-clang-x86_64-libc++=18.1.8-2`
so pacman accepts our pinned libc++ as satisfying gettext-tools'
dep and leaves it alone.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…bc++

`--assume-installed mingw-w64-clang-x86_64-libc++=18.1.8-2` didn't
help — pacman still pulled in libc++ 22.1.4-1 because gettext-tools
1.0's dep is `libc++>=22`, and our --assume-installed at 18.1.8-2
doesn't satisfy that constraint.

`--ignore PKG` is the right knob: it tells pacman not to upgrade
the named package even if a dep requests a newer version. gettext-
tools and gettext-libtextstyle still install; libc++ stays at the
pinned 18.1.8-2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…omment

The "newer clang (>=22) loses the std_vector_const_Mesh registration"
attribution wasn't quite right — that symptom showed up during the
mrbind experiments that motivated this work, not as an inherent
regression in clang 22 itself. Replace with a neutral note that we
pin to the same clang version master ships and that pinning only
clang/llvm causes STATUS_DLL_NOT_FOUND.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…s lockfile

Since msys2_download_packages.sh derives URLs from each filename's
package-type prefix, no script reads msys2_package_urls.txt anymore.
The only remaining consumer was msys2_remember_current_packages.sh
(which writes it). Drop the file and trim the regenerator's URL
echoes to leave just hash-file output.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…file

The URLs lockfile was kept in lockstep with the hashes lockfile, but
each URL is fully determined by the package's filename — the
package-type prefix (`mingw-w64-clang-x86_64-…` → `mingw/clang64/`,
etc.) maps directly to the mirror path. The mapping was already
encoded in `msys2_remember_current_packages.sh` on the writer side.

Move the same mapping into `msys2_download_packages.sh`: parse each
`<sha256> *msys2_packages/<filename>` line of the hashes file,
compute `https://mirror.msys2.org/<prefix>/<filename>`, write to a
temp URL list, and feed that to `wget -i`. Drop the now-unused URL
echoes from the regenerator and delete the 139-line URLs lockfile.

No behavior change for the developer-bootstrap flow
(`install_deps_windows_msys2.bat` calls these scripts and downloads
the same set of packages).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fedr and others added 3 commits May 10, 2026 19:29
Git autocrlf on Windows checks msys2_package_hashes.txt out as CRLF
by default, and `sha256sum -c` doesn't tolerate filenames with a
trailing \r — every entry fails as

  sha256sum: 'msys2_packages/<file>'$'\r': No such file or directory
                                       : FAILED open or read

Hit by anyone running install_deps_windows_msys2.bat manually on a
Windows checkout. Pipe the lockfile through `tr -d '\r'` before
feeding to sha256sum (and the subsequent mapfile read).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… uses

Both scripts now match PR #6060's versions byte-for-byte:

  * msys2_download_packages.sh accepts an optional SUFFIX positional
    arg, reading `msys2_package_hashes<SUFFIX>.txt` (default suffix
    `''`). Switches `wget -c` → `wget -nc` so a cache hit skips
    files already on disk without HEAD-request round-trips; the
    install script's sha256 verify is the safety net for partials.

  * msys2_install_packages.sh accepts the same optional SUFFIX arg
    in addition to the CR-strip already added in this PR.

Master's CI doesn't pass a suffix (the `install_deps_windows_msys2.bat`
local-developer flow stays unchanged, suffix `''`). PR #6060 calls
`msys2_download_packages.sh _clang18` to read its alternate
`msys2_package_hashes_clang18.txt` lockfile. Landing this in master
means PR #6060's diff for these two scripts drops to zero on rebase.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fedr added a commit that referenced this pull request May 11, 2026
…file (#6083)

* ci(mrbind): drop msys2_package_urls.txt; derive URLs from hashes lockfile

The URLs lockfile was kept in lockstep with the hashes lockfile, but
each URL is fully determined by the package's filename — the
package-type prefix (`mingw-w64-clang-x86_64-…` → `mingw/clang64/`,
etc.) maps directly to the mirror path. The mapping was already
encoded in `msys2_remember_current_packages.sh` on the writer side.

Move the same mapping into `msys2_download_packages.sh`: parse each
`<sha256> *msys2_packages/<filename>` line of the hashes file,
compute `https://mirror.msys2.org/<prefix>/<filename>`, write to a
temp URL list, and feed that to `wget -i`. Drop the now-unused URL
echoes from the regenerator and delete the 139-line URLs lockfile.

No behavior change for the developer-bootstrap flow
(`install_deps_windows_msys2.bat` calls these scripts and downloads
the same set of packages).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci(mrbind): strip CR before sha256sum -c in msys2_install_packages.sh

Git autocrlf on Windows checks msys2_package_hashes.txt out as CRLF
by default, and `sha256sum -c` doesn't tolerate filenames with a
trailing \r — every entry fails as

  sha256sum: 'msys2_packages/<file>'$'\r': No such file or directory
                                       : FAILED open or read

Hit by anyone running install_deps_windows_msys2.bat manually on a
Windows checkout. Pipe the lockfile through `tr -d '\r'` before
feeding to sha256sum (and the subsequent mapfile read).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci(mrbind): add SUFFIX support + wget -nc to match the version PR #6060 uses

Both scripts now match PR #6060's versions byte-for-byte:

  * msys2_download_packages.sh accepts an optional SUFFIX positional
    arg, reading `msys2_package_hashes<SUFFIX>.txt` (default suffix
    `''`). Switches `wget -c` → `wget -nc` so a cache hit skips
    files already on disk without HEAD-request round-trips; the
    install script's sha256 verify is the safety net for partials.

  * msys2_install_packages.sh accepts the same optional SUFFIX arg
    in addition to the CR-strip already added in this PR.

Master's CI doesn't pass a suffix (the `install_deps_windows_msys2.bat`
local-developer flow stays unchanged, suffix `''`). PR #6060 calls
`msys2_download_packages.sh _clang18` to read its alternate
`msys2_package_hashes_clang18.txt` lockfile. Landing this in master
means PR #6060's diff for these two scripts drops to zero on rebase.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Fedr Fedr requested a review from adalisk-emikhaylov May 11, 2026 05:52
… pacman -S

Master's `pacman -S gettext-tools` against its frozen pacman DB
resolved to an older gettext-tools that doesn't depend on
`cc-libs`. The runner's fresh DB resolves to gettext-tools 1.0,
which depends on `cc-libs` (an aggregator not in our lockfile) AND
on `libc++>=22` (which would upgrade our pinned libc++ 18). Earlier
attempts:

  * `--assume-installed libc++=18.1.8-2` — pacman ignored it
    because 18 doesn't satisfy `>=22`.
  * `--ignore mingw-w64-clang-x86_64-libc++` — kept libc++ at 18,
    but pacman then bailed on the `cc-libs` dep with
    `warning: cannot resolve "mingw-w64-clang-x86_64-cc-libs"` and
    silently skipped installing gettext-tools at all, so msgfmt was
    missing and translation `.po → .mo` compilation got silently
    skipped by CMake's I18nHelpers.

Pin `mingw-w64-clang-x86_64-gettext-tools-0.22.5-2-any.pkg.tar.zst`
in the clang18 lockfile. Its three deps are all already satisfied:
gettext-runtime (in our lockfile), libiconv (in our lockfile), and
gcc-libs (provided by our pinned libc++-18.1.8-2 via the `provides=`
relationship). pacman -U the lockfile installs gettext-tools cleanly
alongside everything else — no second pacman invocation, no
upload_artifacts-gated branch, msgfmt is present unconditionally.

Lockfile grows from 48 → 49 packages.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Comment thread .github/workflows/pip-build.yml
Fedr and others added 3 commits May 12, 2026 00:20
Mirror the cache step from build-test-windows.yml — same key
(lockfile hash) and same path (scripts/mrbind/msys2_packages), so
the two workflows share cache entries. A build-test-windows run
that populates the cache makes the next pip-build run a cache hit,
and vice versa.

Saves ~60 s on the wget step per pip-build run on cache hit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
build-test-windows.yml and pip-build.yml had identical two-step blocks
(`Cache pinned MSYS2 packages` + `Install MSYS2 for MRBind`). Move them
into `.github/actions/install-msys2-mrbind/action.yml` as a composite
action and have both workflows reference it via `uses:`.

  * build-test-windows.yml: keeps its `if:` guard on the outer step,
    so the composite runs only when mrbind/c-bindings/c-sharp is
    actually needed.
  * pip-build.yml: always invokes the composite (mrbind is always on
    in pip-build).

Same cache key + path as before, so the two workflows continue to
share cache entries. Net: -29 lines in workflows, +24 lines in the
new action.

(The action's path was master's previous composite-action name,
deleted earlier on this branch when the install moved inline. Now
that we have a reusable abstraction again, the path is reused.)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Fedr Fedr merged commit 0638372 into master May 12, 2026
79 checks passed
@Fedr Fedr deleted the ci/use-runner-msys2-pin-clang18 branch May 12, 2026 09:15
Fedr added a commit that referenced this pull request May 12, 2026
Conflicts resolved by taking master throughout. PR #6021's
runner-MSYS2 + pacman-installed-clang approach has been fully
superseded by master (via #6060 / #6082 / #6083), so after the
merge this branch matches master exactly with no remaining
diff to apply.

Files reset to master:
  * .github/actions/install-msys2-mrbind/action.yml (re-added with
    cache + composite layout)
  * .github/workflows/build-test-windows.yml + pip-build.yml +
    build-test-distribute.yml + distro-release.yml +
    prepare-images.yml + release-body-update.yml +
    unity-nuget-test.yml + versioning-release.yml
  * scripts/mrbind/msys2_download_packages.sh (URL derivation +
    suffix support + wget -nc)
  * scripts/mrbind/msys2_install_packages.sh (CR-strip + suffix)
  * scripts/mrbind/msys2_remember_current_packages.sh (no URL
    file write)
  * scripts/mrbind/msys2_package_hashes.txt (master's frozen
    baseline)
  * scripts/mrbind/README-generating.md / README-updating-clang.md
  * source/MRMesh/MRMeshTopology.cpp/h + source/MRViewer/CMakeLists.txt
    (mrbind-experiment fallout reverted)
  * thirdparty/mrbind submodule (master's pin)

Net new: msys2_package_hashes_clang18.txt (added by master via #6060).
Net deleted: msys2_package_urls.txt (removed by master via #6083).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fedr added a commit that referenced this pull request May 12, 2026
Switch the install-msys2-mrbind composite from `pacman -U` against a
sha256-pinned lockfile (master's approach via #6060, ensuring clang
18.1.8) to a plain `pacman -S --needed` against the live MSYS2
mirror — picks up whatever clang/libc++/cmake/etc. is current.

  * Minimal explicit package set: clang, lld, libc++, cmake, ninja,
    gettext-tools, plus msys2 `make`. Pacman resolves transitive
    deps; nothing is downgraded so `--needed` skips anything the
    runner image already ships.
  * No `actions/cache@v5` step at all. The clang-18 cache populated
    by master's composite (key `msys2-mrbind-clang18-<hash>`) is on
    a different path + key and cannot be hit.
  * scripts/mrbind/msys2_package_hashes_clang18.txt deleted — the
    lockfile is no longer consulted on the CI install path.
    Developer-bootstrap path (install_deps_windows_msys2.bat) still
    uses the unsuffixed msys2_package_hashes.txt and is unaffected.

Trade-off vs master: faster (one `pacman -S`, ~30-60 s, no archive
extraction step), but exposed to mrbind regressions on newer clang
that motivated the original clang-18 pin. Whether mrbind tolerates
current clang now is what this PR validates.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants