Linux-vcpkg: keep libz-ng dynamic, strip ZLIB_NG_* DT_VERNEED before auditwheel#5998
Closed
Linux-vcpkg: keep libz-ng dynamic, strip ZLIB_NG_* DT_VERNEED before auditwheel#5998
Conversation
Scoped to the single public function the user requested:
Expected<void> zlibCompressStream( std::istream&, std::ostream&,
const ZlibCompressParams& )
plus its thin int-level overload, which forwards to it. Decompression
(zlibDecompressStream) and every other zlib consumer (libzip's own
deflate path inside compressZip/decompressZip, etc.) stay on stock
zlib -- this PR does NOT swap zlib for zlib-ng globally.
zlib-ng is used in its native "-ng" mode: the library exposes zng_
prefixed symbols and the zlib-ng.h header, the SONAME is libz-ng (not
libz), so it can coexist with stock zlib in the same process without
any ABI overlap or symbol collisions.
Per-platform source of the library:
- Windows vcpkg: zlib-ng port (added to requirements/windows.txt)
- Rocky Linux vcpkg: zlib-ng port (added to requirements/vcpkg-linux.txt)
- macOS Homebrew: zlib-ng formula (added to requirements/macos.txt;
not keg-only, native-mode upstream defaults)
- Ubuntu apt: no standalone package, built from thirdparty/zlib-ng
- Emscripten: no package, built from thirdparty/zlib-ng
thirdparty/CMakeLists.txt gates add_subdirectory(./zlib-ng) on
NOT WIN32 AND NOT APPLE AND NOT MESHLIB_USE_VCPKG, matching the split
above. ZLIB_COMPAT stays OFF so the submodule build also produces
native zng_ / libz-ng / zlib-ng.h, identical to what vcpkg and brew
install.
MRMesh/CMakeLists.txt uses find_library/find_path rather than
find_package(zlib-ng CONFIG) for the same relocatability reason as
libdeflate (upstream config can bake an absolute include path that
breaks under the Ubuntu/Emscripten Docker COPY-shuffle).
Source: split the compress body into a new MRZlibNg.cpp so that
translation unit includes only <zlib-ng.h> and the untouched
zlibDecompressStream in MRZlib.cpp keeps including only <zlib.h>.
Avoids any macro-redefinition ordering question between the two
headers without forcing code to choose one.
Drop the separate MRZlibNg.cpp; fold the zng_-prefixed zlibCompressStream overloads directly into MRZlib.cpp alongside zlibDecompressStream. Both <zlib.h> and <zlib-ng.h> include cleanly in the same TU: the Z_* return-code / flush-mode / strategy constants are defined in both headers with identical values (C preprocessor allows a redefinition with the same replacement list, no warning), and the function names (deflate/inflate vs zng_deflate/zng_inflate) and struct types (z_stream vs zng_stream) are disjoint. The shared zlibToString and windowBitsFor helpers are reused across compress (zng_*) and decompress (stock zlib) since MAX_WBITS equals Z_MAX_WINDOWBITS = 15 and the Z_* code values match. No functional change relative to the previous commit on this branch; just collapses two TUs into one.
This reverts commit e66838a. The merge looked clean under static inspection (Z_* macros identical between zlib.h and zlib-ng.h, deflate/inflate vs zng_deflate/zng_inflate disjoint, z_stream vs zng_stream disjoint) but failed to compile on macOS-arm64 Debug with: /opt/homebrew/include/zlib-ng.h:1079:20: error: typedef redefinition with different types ('uint32_t (*)(void *, const uint8_t **)' vs 'unsigned int (*)(void *, unsigned char **)') Both headers declare callback typedefs in_func and out_func (used by inflateBack) at global scope with the same name and different signatures: zlib.h: typedef unsigned (*in_func)(void*, unsigned char**); zlib-ng.h: typedef uint32_t (*in_func)(void*, const uint8_t**); C++ refuses the redefinition. The typedefs are unconditional in both headers -- there's no feature macro to hide them -- even though MeshLib's code uses neither inflateBack nor the two typedefs. Splitting the compress body into its own TU (MRZlibNg.cpp) so each TU sees at most one of the two headers is the clean fix. Run showing the failure: 24776570063.
zlib-ng's CMakeLists treats an undefined BUILD_SHARED_LIBS as "build
both shared and static targets". Emscripten then demotes the SHARED
one to STATIC, after which both rules emit the same libz-ng.a output
and ninja fails at configure time:
CMake Warning (dev) at zlib-ng/CMakeLists.txt:1165 (add_library):
ADD_LIBRARY called with SHARED option but the target platform does
not support dynamic linking. Building a STATIC library instead.
...
ninja: error: build.ninja:1263: multiple rules generate libz-ng.a
[-w dupbuild=err]
Force BUILD_SHARED_LIBS=OFF only for the zlib-ng add_subdirectory on
Emscripten. Ubuntu apt keeps the default (shared) so MRMesh.so still
picks up libz-ng.so. Local set() -- no CACHE -- keeps the override
scoped to this add_subdirectory and doesn't leak to the later
Emscripten-specific thirdparty libs below.
Seen on run 24776570063, job 72496287138.
The CMake build globs source files automatically, but the MSBuild project is hand-maintained -- the new TU that carries zlibCompressStream against zlib-ng needs to be listed explicitly in both MRMesh.vcxproj (ClCompile) and MRMesh.vcxproj.filters (Source Files\IO, next to MRZlib.cpp).
zlib-ng does not declare Z_MAX_WINDOWBITS -- I hallucinated that name
when writing MRZlibNg.cpp. zlib-ng's zconf-ng.h re-exports MAX_WBITS
(15) under exactly the same spelling stock zlib uses, so the native-
mode header already gives us the constant we need.
Fix the two constexpr initialisers that were using the wrong name,
and tighten the surrounding comment to reflect what the header
actually provides.
Seen on macOS arm64 Debug (run 24777754710, job 72500510814):
/source/MRMesh/MRZlibNg.cpp:25:34: error: use of undeclared
identifier 'Z_MAX_WINDOWBITS'
/source/MRMesh/MRZlibNg.cpp:26:35: error: use of undeclared
identifier 'Z_MAX_WINDOWBITS'
The test pinned stats.compressedSize against sizeof(cRawLevel{1,9})
and sizeof(cWrappedLevel{1,9}), i.e. reference blobs captured from
stock zlib. Each alternative deflate implementation we've tried
produces a valid, lossless output of a slightly different size:
stock zlib: reference (70 / 76 bytes on this corpus)
zlib-ng compat: stock - 1 byte
zlib-ng native: stock + 7 bytes (seen here)
libdeflate: stock ± small amount
All of these are correct. Chasing tolerance windows (+/-4 was enough
for zlib-ng-compat, +/-7 now for native, libdeflate may need more)
is fragile. Replace the exact-size EXPECT_EQ with four engine-
agnostic invariants that still catch any real regression in the
stats API:
- stats.crc32 matches an independent CRC-32 reference (was already)
- stats.uncompressedSize == sizeof( input ) (was already)
- stats.compressedSize == out.str().size() (API internal consistency)
- 0 < stats.compressedSize < sizeof( input ) (non-empty, compressed)
The round-trip check in ZlibCompressTestFixture + ZlibDecompressTest
Fixture still guarantees byte-exact recovery, so any encoder bug that
produces something parseable-but-wrong would still fail there.
Seen on run 24778762984 (feat/zlib-compress-stream-zlib-ng):
macOS arm64 Release, Debug
source/MRTest/MRZlibTests.cpp:160-161:
stats.compressedSize: 77 c.expectedCompSize: 70
stats.compressedSize: 83 c.expectedCompSize: 76
…lib-compress-stream-zlib-ng
…subdirectory The static-only setting I added in d1b60f3 to work around zlib-ng's multiple-rules-generate-libz-ng.a collision on Emscripten leaked to every subsequent add_subdirectory in the same scope. Specifically jsoncpp further down in thirdparty/CMakeLists.txt (line 133) is configured as: set(JSONCPP_WITH_TESTS OFF) set(JSONCPP_WITH_POST_BUILD_UNITTEST OFF) set(BUILD_STATIC_LIBS OFF) add_subdirectory(./jsoncpp) That relies on BUILD_SHARED_LIBS being in its previous (unset or ON) state to produce the shared jsoncpp library. After my leak flipped BUILD_SHARED_LIBS to OFF, jsoncpp had both BUILD_SHARED_LIBS=OFF and BUILD_STATIC_LIBS=OFF -- neither flavour was built. Headers were still installed but the library file was not, so the main Emscripten build failed at configure time with: CMake Error (FindPackageHandleStandardArgs.cmake): Could NOT find JsonCpp (missing: JsonCpp_LIBRARY) (found version "1.9.5") Wrap the set(BUILD_SHARED_LIBS OFF) in save/restore and scope the add_subdirectory(./zlib-ng) call inside that window so the override is visible only to zlib-ng. Same pattern already used for googletest at lines 46-50. Seen on Multithreaded-64Bit, Multithreaded, and Singlethreaded Emscripten legs (run 24781052774).
…estore-from-backup
The save/restore pattern I added in d68fd56b captured "" from an
undefined BUILD_SHARED_LIBS and then "restored" BUILD_SHARED_LIBS to ""
(defined-but-empty). That's a different state than the original
(undefined), and jsoncpp's option(BUILD_SHARED_LIBS ... ON) fallback
does NOT re-initialise an already-defined variable. So the empty value
persisted into jsoncpp's lib_json CMakeLists line 179:
set_target_properties(${OBJECT_LIB} PROPERTIES
OUTPUT_NAME jsoncpp
VERSION ${PROJECT_VERSION}
SOVERSION ${PROJECT_SOVERSION}
POSITION_INDEPENDENT_CODE ${BUILD_SHARED_LIBS} <- expands to ""
)
The empty expansion broke the key/value pairing and produced
"set_target_properties called with incorrect number of arguments",
aborting thirdparty CMake configure on the Emscripten image build.
On every Emscripten MeshLib build before this PR, BUILD_SHARED_LIBS
was undefined when thirdparty/CMakeLists.txt reached the zlib-ng
block (confirmed: zlib-ng is added before libzip/jsoncpp, and nothing
upstream sets the variable). So unset() after the add_subdirectory
is both correct and simpler than the if(DEFINED)/backup/restore
dance, and it matches the state jsoncpp's option() wants to see.
Seen on run 24800811051, job 72582653120.
STATUS_DLL_NOT_FOUND (exit code -1073741515 / 0xC0000135) at
MeshViewer.exe launch kills the Run Start-and-Exit Tests step with
zero useful output — the Windows loader aborts before main() and
CI sees only a bare ##[error]Process completed with exit code ...
Observed on the zlib-ng branch run 24802707812, job 72596857457
(msvc-2019 Debug CMake with the x64-windows-meshlib-iterator-debug
triplet): vcpkg installed zlib-ng successfully but libz-ng.dll
didn't end up next to MeshViewer.exe. Three other Windows legs in
the same run with the default triplet succeeded, so the gap is
specifically in the iterator-debug triplet's DLL-copy chain.
Add two unconditional diagnostic steps so the next run of any
Windows job self-documents the state the loader will see:
1. After vcpkg-integrate-install, "Diagnostic — vcpkg installed
tree": lists the contents of
C:\vcpkg\installed\<TRIPLET>\{bin,debug\bin,lib,debug\lib}
so we see which DLLs vcpkg actually installed and under what
names (e.g. libz-ng.dll vs libz-ngd.dll). Confirms or rules out
"vcpkg didn't install the package" as the cause.
2. Right before "Run Start-and-Exit Tests", "Diagnostic — output
bin DLL inventory + MRMesh imports":
- Lists all .dll/.exe under source\x64\<CONFIG>\ so we see
which DLLs were copied next to MeshViewer.exe
- Runs dumpbin /dependents on MeshViewer.exe, MRMesh.dll,
MRTest.exe so we see which DLL names the loader is actually
looking for at process start
- Dumps the PATH so we can tell whether the vcpkg install
dir would be a fallback lookup location
Together, these three data points are enough to turn any
DLL-not-found failure into a one-line diagnosis.
Both steps have `if: always()` and `continue-on-error: true`, so
they run even when the main Build step failed (often when you need
the diagnostic most) and never mask a real failure themselves.
Net cost: ~30 lines of pwsh output per Windows job when everything
works; ~100 lines when a DLL is missing. Trivial relative to the
multi-gigabyte Windows CI logs already emitted.
…brary
The diagnostic output from run 24834309936 shows two linked problems on
the msvc-2019 Debug CMake x64-windows-meshlib-iterator-debug leg:
1. Debug MRMesh.dll imports from zlib-ng2.dll (the RELEASE variant)
instead of zlib-ngd2.dll (the matching Debug variant) that vcpkg
also installed. Every other MRMesh.dll dependency resolved to its
d-suffixed debug DLL (zlibd1.dll, spdlogd.dll, tbb12_debug.dll,
tiffd.dll, fmtd.dll) -- those use find_package(... CONFIG) which
exports IMPORTED_CONFIGURATIONS=DEBUG;RELEASE with distinct per-
config IMPORTED_LOCATION properties.
2. Neither zlib-ng2.dll nor zlib-ngd2.dll was copied next to
MeshViewer.exe by vcpkg's applocal-copy step. Applocal keys on
imported-target metadata; a raw find_library() result doesn't
carry the Debug/Release-aware config map that applocal expects,
so the copy falls through.
Together: Debug MeshViewer.exe launches, loader walks MRMesh.dll's
import table, looks for zlib-ng2.dll in the output dir and on PATH,
finds neither (vcpkg's install bin isn't on PATH), aborts with
STATUS_DLL_NOT_FOUND (0xC0000135 / exit -1073741515) before main().
Fix both by using find_package(zlib-ng CONFIG REQUIRED) on Windows and
linking the imported target zlib-ng::zlib (vcpkg's zlib-ng port ships
the proper config package). Keep the existing find_library / find_path
path for everything else -- Ubuntu apt and Emscripten build zlib-ng
from thirdparty/zlib-ng and its generated config has the same non-
relocatable-include-path bug we hit with libdeflate under the Docker
multi-stage COPY-shuffle, and those platforms are single-config so the
Debug/Release picking issue doesn't apply.
Only the Windows find is gated; macOS and Linux keep their current
working behaviour.
The find_package(zlib-ng CONFIG) approach from 83e7169 works on vcpkg 2026.03.18 (msvc-2022 legs) but fails on vcpkg 2024.10.21 pinned for the msvc-2019 legs with Could not find a package configuration file provided by "zlib-ng" with any of the following names: zlib-ngConfig.cmake zlib-ng-config.cmake The older vcpkg port ships zlib-ng 2.1.5 which vcpkg installed correctly (diagnostic output confirmed zlib-ng2.dll / zlib-ngd2.dll / zlib-ng.lib / zlib-ngd.lib all present in the install tree) but without a CMake config file. Visible from the port's install announcement: libzip advertises "provides CMake targets:" while zlib-ng only announces "provides pkg-config modules:". Neither find_library (loses Debug/Release selectivity) nor find_package(CONFIG) (unavailable on the older port) works. Manually declare the imported target with per-configuration IMPORTED_IMPLIB and IMPORTED_LOCATION properties, pointing directly at the triplet install tree via the vcpkg-toolchain-provided VCPKG_INSTALLED_DIR / VCPKG_TARGET_TRIPLET variables. That's what find_package(CONFIG) would emit if the port supplied one; doing it by hand makes the find resilient to the port's config availability. Keep the else-branch (Linux / macOS / Emscripten / Rocky Linux vcpkg) on the existing find_library path -- those platforms are single-config, don't have the Debug/Release selection issue, and have no applocal-copy dependency on imported-target metadata. Seen on run 24843744883, jobs 72724909304 (msvc-2019 Release) and 72724909339 (msvc-2019 Debug iterator-debug).
Complements the earlier zlibCompressStream-on-zlib-ng swap on this PR: now both directions of MRMesh's public RFC 1950 / 1951 streaming API run through zlib-ng, not just deflate. Motivation is per upstream zlib-ng benchmarks and the AWS Open Source forks comparison: decompression in zlib-ng is ~2-3x faster than stock zlib at the inflate step (wider margin than the deflate speedup), driven by SIMD-vectorised inflate_chunkcopy (SSE2/SSSE3/AVX2 on x86, NEON on ARM). References in the upstream benchmark discussion github.com/zlib-ng/zlib-ng/discussions/871 and the 2025 re-run at github.com/zlib-ng/zlib-ng/issues/1486. Concrete MeshLib consumers that inherit the speedup: decompressZip, scene loaders that read .mrmesh blobs, and the round-trip verify step in CompressManySmallFilesToZip (~15 MB decompressed per test iteration, half its wallclock was inflate). Implementation notes: - Port the existing Buffer<char> chunk-loop inflate code from MRZlib.cpp to MRZlibNg.cpp, replacing z_stream/inflate*/inflate with zng_stream/zng_inflate*/zng_inflate. Behaviour is identical: same 256 KiB chunks, same Z_NO_FLUSH/Z_STREAM_END handshake, same error codes (Z_* constants are shared between the two headers and have identical numeric values). - MRZlib.cpp had only the two zlibDecompressStream overloads left after the earlier compress move; with decompress now in MRZlibNg.cpp too, the old file has no live content. Delete it (and its ClCompile entries in MRMesh.vcxproj + MRMesh.vcxproj.filters). MRZlib.h stays -- it's the public API. - Round-trip remains compatible across forks. zlib-ng native's inflate reads any valid RFC 1950/1951 stream produced by stock zlib, libdeflate, zlib-ng, or any other DEFLATE encoder, so the parametrised ZlibDecompressTestFixture's pre-captured stock-zlib reference blobs continue to decompress correctly. - Stock zlib header <zlib.h> is no longer included by any MRMesh TU. MRMesh/CMakeLists.txt still does find_package(ZLIB) + link ZLIB::ZLIB because libzip transitively depends on it and the existing link line doesn't hurt; a cleanup PR can drop that link later if desired.
This reverts commit cb9dfb0.
Now that both zlibCompressStream and zlibDecompressStream live in the same TU through zlib-ng (and there's no longer a second zlib-only TU to disambiguate from), rename MRZlibNg.cpp back to MRZlib.cpp -- the file is the stock place readers expect to find MRZlib's implementation. MRMesh.vcxproj and MRMesh.vcxproj.filters updated to match. Also restore source/MRTest/MRZipCompressTests.cpp to master's test- scale constants (1000 sphere vertices, 20+20 files x 6000 bytes). The larger constants were temporarily bumped during the benchmark comparisons on this branch; they're not appropriate for the default MRTest run.
…on script Upstream zlib-ng 2.3.3's CMakeLists.txt defines HAVE_SYMVER and passes -Wl,--version-script=zlib-ng.map on non-Apple, non-AIX Unix. Both together tag every exported symbol in libz-ng.so.2 with ZLIB_NG_2.0.0 / ZLIB_NG_2.1.0 version nodes, which land in DT_VERNEED of anything linking against it. auditwheel's manylinux policy database has no entry for the pair (libz-ng.so.2, ZLIB_NG_*), so the Linux-vcpkg NuGet wheel-repair step fails with "too-recent versioned symbols" even though no actual symbol is too recent — auditwheel's generic phrasing for any (lib, version- tag) pair it can't place in a known policy. This blocks #5959 on the Linux-vcpkg Clang 20 Release leg. Contrast with stock libz.so.1: auditwheel has that library explicitly on the manylinux allowlist with its ZLIB_1.2.0 tags pre-registered, so it passes trivially. zlib-ng is not on any allowlist. We don't exercise zlib-ng's ABI-versioning machinery — MeshLib's consumers rebuild against whatever libz-ng we ship — so neutralizing both knobs is safe. The overlay port replaces the guarding `if(NOT APPLE AND NOT CMAKE_SYSTEM_NAME STREQUAL AIX)` with `if(FALSE)`, skipping the HAVE_SYMVER define and the --version-script linker flag in one edit. Upstream's zlib-ng.map file is left on disk but never wired into the build. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ild for overlay zlib-ng port)
…stream-zlib-ng-stable
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file is only entered by Linux/Emscripten builds (build_thirdparty.sh). Windows and MESHLIB_USE_VCPKG configs never execute thirdparty/CMakeLists.txt at all -- they get zlib-ng from vcpkg directly. The only remaining exclusion is APPLE, which uses the Homebrew copy. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ol version script" This reverts commit 29057ae.
…uditwheel Replaces the reverted vcpkg overlay-port (29057ae) with a consumer-side fix: every .so is rewritten in place to drop its libz-ng.so.2 entry from DT_VERNEED before auditwheel scans it. We do not exercise zlib-ng's ABI versioning, so the ZLIB_NG_2.0.0 / ZLIB_NG_2.1.0 tags upstream's CMake attaches to consumer DT_VERNEED add nothing but break auditwheel's manylinux policy check (no policy entry for libz-ng.so.2). The new helper (scripts/wheel/strip_zlib_ng_symbol_versions.py) zeroes the libz-ng Verneed's vn_cnt -- enough to silence pyelftools/auditwheel -- and rewrites the corresponding .gnu.version (DT_VERSYM) entries to VER_NDX_GLOBAL so the dynamic linker resolves zng_* symbols by name at load time without consulting the now-empty Vernaux set. Other versioned deps (GLIBC_*, GLIBCXX_*, libgcc_s, ...) are not touched. Wired into the two auditwheel call sites: - scripts/wheel/build_wheel.py: strips LIB_DIR + WHEEL_SRC_DIR before `python -m build --wheel`, so both the wheel's own modules and the dep libs auditwheel pulls in during repair are clean. - scripts/nuget_patch/patch_library_deps.py: strips the parent dirs of the input C-API .so before make_fake_whl + auditwheel repair. Linux-only; no-op elsewhere. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ted by pyelftools)
The previous version zeroed the libz-ng Verneed's vn_cnt to silence
auditwheel's symbol-version scan. That assumed pyelftools' iter_versions
would see vn_cnt=0 and yield zero Vernauxes. In pyelftools 0.32 (current
auditwheel transitive dep) the iterator instead hard-asserts vn_cnt > 0
and raises ELFError, crashing `auditwheel repair`:
elftools.common.exceptions.ELFError: Expected number of version
auxiliary entries (vn_cnt) to be > 0 for the following version entry:
Container({'vn_version': 1, 'vn_cnt': 0, 'vn_file': 295589, ...})
Switch to a proper removal: rebuild .gnu.version_r in place without
the libz-ng Verneed (remaining entries repacked contiguously, section
zero-padded to its original size), decrement DT_VERNEEDNUM, and rewrite
the corresponding .gnu.version (DT_VERSYM) entries to VER_NDX_GLOBAL so
the dynamic linker resolves zng_* by name at load time.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fedr
added a commit
that referenced
this pull request
Apr 28, 2026
Wrap every `dnf -y install` (and `dnf config-manager --add-repo`) in both `docker/rockylinux8-vcpkgDockerfile` and `docker/rockylinux9-vcpkgDockerfile` with the existing `retry.sh` helper (3 attempts, 30s backoff). Move the `COPY scripts/retry.sh` to the top of each build/production stage so it is available before the first `dnf install` runs. Run https://github.com/MeshInspector/MeshLib/actions/runs/25045542370/job/73359583376 on PR #5998 failed in the production stage at line 69 with: Error: Failed to download metadata for repo 'baseos': repodata/...-primary.xml.gz - Cannot download, all mirrors were already tried without success repodata/...-filelists.xml.gz - Cannot download, ... repodata/...-updateinfo.xml.gz - Cannot download, ... Every Rocky Linux 8.10 (aarch64) baseos mirror returned 404 for those specific repodata files — the classic "stale repomd" sync gap that clears after a few minutes once mirrors re-sync. #5985 already wraps `./vcpkg install` with retry but does not cover `dnf install`, so this failure mode hits without retry. Same Dockerfile also has the gh-cli install RUN block that hits cli.github.com — wrapped for the same reason.
Restore the layout the merged variant of this work uses on master:
- cmake/Modules/Findzlib-ng.cmake providing the zlib-ng::zlib target
on Ubuntu apt / Emscripten via pkg-config + find_library/find_path.
- thirdparty/vcpkg/ports/zlib-ng/{portfile.cmake,vcpkg.json} as the
standard vcpkg overlay, building zlib-ng@2.3.3 with the upstream
CMake config (no symbol-version stripping inside the port).
- thirdparty/install.bat: pass --overlay-ports so the overlay port is
picked up on Windows.
- source/MRMesh/CMakeLists.txt: a single
find_package(zlib-ng CONFIG REQUIRED)
target_link_libraries(... PRIVATE zlib-ng::zlib)
in place of the previous WIN32/ELSE branch with the manually
constructed SHARED IMPORTED target. CONFIG mode is required on
Windows so the imported target carries IMPORTED_LOCATION_DEBUG /
_RELEASE for applocal-deploy.
- source/MRMesh/MRMeshConfig.cmake.in: propagate find_dependency(zlib-ng).
- thirdparty/CMakeLists.txt: drop now-stale leading comment block;
shorten the Emscripten BUILD_SHARED_LIBS comment.
Differs from 017807d only on the Linux-vcpkg auditwheel workaround:
that commit pins VCPKG_LIBRARY_LINKAGE=static for zlib-ng in both Linux
triplets so libz-ng's symbols absorb into libMRMesh.so. We keep the
Linux-vcpkg zlib-ng build dynamic and instead drop libz-ng.so.2's
DT_VERNEED entry from every consumer .so via
scripts/wheel/strip_zlib_ng_symbol_versions.py just before auditwheel.
Same end result for auditwheel; libMRMesh.so still loads libz-ng.so.2
dynamically at runtime.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The auditwheel/zlib-ng incompatibility (pypa/auditwheel#613) is already handled on this branch by scripts/wheel/strip_zlib_ng_symbol_versions.py, which rewrites every consumer .so so its .gnu.version_r no longer carries the libz-ng entry before auditwheel runs. With that in place the static-linkage workaround is redundant -- restore the default VCPKG_LIBRARY_LINKAGE=dynamic so libz-ng on Linux-vcpkg matches every other platform and libMRMesh.so stays a thin shared library. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fedr
changed the title
Contributor
Author
|
Closing for now. The Linux-vcpkg auditwheel rejection this PR works around is auditwheel's own bug (pypa/auditwheel#613). I've opened a fix for it upstream at pypa/auditwheel#694 — once that lands and ships in a release, the consumer-side ELF rewrite in this PR becomes unnecessary and we can revisit with a much smaller change. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Provenance
Originally opened as a green snapshot of #5959. Now that #5959 has merged to master (017807d04f), this PR's remaining content is just the post-merge follow-up: replace the static-linkage workaround for pypa/auditwheel#613 with a consumer-side ELF rewrite, so libz-ng can stay a dynamic library on Linux-vcpkg like on every other platform.
What changes vs master
Drop the static-linkage pin (
thirdparty/vcpkg/triplets/{x64,arm64}-linux-meshlib.cmake)#5959 worked around auditwheel's missing manylinux policy entry for libz-ng.so.2 by pinning
VCPKG_LIBRARY_LINKAGE=staticfor the zlib-ng port on both Linux triplets, so libz-ng's symbols absorbed into libMRMesh.so directly with no DT_VERNEED reference. This PR removes that pin: libz-ng builds dynamic on Linux-vcpkg again, matching every other platform, and libMRMesh.so stays a thin shared library.scripts/wheel/strip_zlib_ng_symbol_versions.py(new)Surgical ELF rewrite, run on every consumer
.sojust beforeauditwheel repair. For each.so:libz-ng.so.2Verneed in.gnu.version_r. Rebuild the section in place without that entry — remaining Verneed/Vernaux records repacked contiguously, the section zero-padded to its originalsh_sizeso all other section offsets stay valid. (Zeroingvn_cntwould have been a one-line fix, but pyelftools 0.32 — auditwheel's transitive dep — hard-assertsvn_cnt > 0and crashes the repair step.)DT_VERNEEDNUMin the dynamic table..gnu.version(DT_VERSYM) entry that referenced one of the dropped Vernaux indices toVER_NDX_GLOBAL(1, "global, unversioned"), so the dynamic linker resolveszng_*symbols by name only at load time without consulting the now-empty Vernaux set.Other versioned dependencies (
GLIBC_*,GLIBCXX_*,libgcc_s, …) are left untouched, so auditwheel's manylinux policy classification (which keys off GLIBC versions) still works correctly.This is safe because MeshLib does not exercise zlib-ng's ABI versioning — consumers are always rebuilt against whatever libz-ng we ship — so dropping the version requirement only tells the loader "resolve by name, no version check," which is what we want.
Wired into both auditwheel call sites
scripts/wheel/build_wheel.py— stripsLIB_DIR+WHEEL_SRC_DIRbeforepython -m build --wheel, so both the wheel's own modules and the dep libs auditwheel pulls in during repair are clean. Linux-only.scripts/nuget_patch/patch_library_deps.py— strips the parent dirs of the input C-API.sobeforemake_fake_whl+ auditwheel repair. Linux-only.Why a Linux-vcpkg-specific issue at all
Auditwheel's manylinux policy database has no entry for
libz-ng.so.2(zlib-ng has never been part of any manylinux baseline). With dynamic linkage, every consumer.socarrieslibz-ng.so.2 ZLIB_NG_2.0.0/2.1.0in itsDT_VERNEED, and auditwheel rejects the wheel as "too-recent versioned symbols" — the generic phrasing for any (lib, version-tag) pair it can't classify. The Apt + Emscripten + Homebrew + Windows paths don't go through auditwheel and so were never affected; only the Linux-vcpkg NuGet wheel-repair step was. StrippingDT_VERNEEDon the consumer side makes auditwheel see no version requirements for libz-ng at all, which matches reality (we don't depend on a specific libz-ng version) and unblocks the repair step.