Add from source wrapping#50
Conversation
|
Hi! This is the friendly automated conda-forge-linting service. I just wanted to let you know that I linted all conda-recipes in your PR ( I do have some suggestions for making it better though... For recipe/meta.yaml:
This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/27852953943. Examine the logs at this URL for more detail. |
|
@thewtex Can you approve the workflow runs for this PR? |
|
@blowekamp @CavRiley yes, triggered! ✨ |
|
@thewtex @blowekamp thank you! FYI: the build process for these is successful on my forks GHAs but fail at validation due to the new |
Is the second package really needed? Last time I added a second package I did this: |
Depends on the remote modules need or willingness to support conda environments. The second package only holds the build tree, so if the need is not there, then it can be removed |
Most modules are being moved to the main ITK repository right now. Unless you have an immediate need or application for itk-devel, I would delay added it. |
Replace the PyPI wheel repackaging with a from-source ITK build that produces a self-contained `itk` package: the Python module plus the C++ runtime libraries it links against. Target Python 3.10, 3.11 and 3.12 across linux-64, linux-aarch64, osx-64, osx-arm64 and win-64.
The from-source ITK build with Python wrapping fills the ~14 GB GitHub-hosted runner, so the test phase failed extracting the itk package with "No space left on device". Set github_actions.free_disk_space so the rerendered workflow reclaims runner space before building. Re-rendered.
The build requirement `cmake >=3.16.3, <3.30.0` had a space after the comma, which conda-build can misparse (treating `<3.30.0` as a build string). Use `cmake >=3.16.3,<3.30.0`.
Windows and macOS built on Azure-hosted agents (2 and 3 cores), where a full 2;3;4-dim ITK wrapping build did not finish within the 6 h job cap (Windows especially, on 2 cores). Route osx + win to GitHub Actions (windows-latest and macos-15-intel are 4 cores) so the builds fit the same 6 h limit with margin. Re-rendered.
osx_arm64 was cross-compiled on the osx_64 (Intel) runner via build_platform: osx_arm64: osx_64, which skips the arm64 tests and ignores GitHub's native Apple-Silicon runners. Build it natively on macos-latest (arm64) via provider: osx_arm64: github_actions, so the arm64 Python wrapping is import-tested on the target architecture. Re-rendered.
d67502d to
199a610
Compare
|
@thewtex We may need to request more CI compute resources on GHA or Azure runners (seen in conda-forge admin requests). Also should consider which Python versions to target, or whether to build against the stable ABI mirroring the wheels. |
It look to me that the osx intel build is running on an OS ARM build. The emulation is slowing things down. I am not sure how or if GHA can be configured to use the intel image. |
| - CONFIG: osx_64_python3.10.____cpython | ||
| STORE_BUILD_ARTIFACTS: False | ||
| UPLOAD_PACKAGES: True | ||
| os: macos | ||
| runs_on: ['macos-15-intel'] |
There was a problem hiding this comment.
@blowekamp I would think this configuration enforces the intel runner image. But the Set up job step for the osx-64 jobs use the macos-15 image name. Is there a possibility that image name refer to both ARM and intel images?
There was a problem hiding this comment.
Yes, I think you are correct.
I was looking for this string in the "Set up job" step in the action and didn't see it. Look again at the log, I don't see clear information about what the architecture is for the runner.
There was a problem hiding this comment.
On line 732 of the "Build on macOS" step during conda configuration there is this line in a conda info call:
virtual packages : __archspec=1=skylake
As far as I know, the emulator would have reported an "M" architecture if emulated, but I could be wrong
|
With Mac OSX Tahoe 27, intel's are no longer supported. With this legacy configuration being the only one timing out perhaps this PR can move forward with out it? |
I think that is reasonable. @hjmjohnson may be able to provide an Intel mac as a self-hosted runner but I cannot confirm that. |
@CavRiley yes, good ideas. These could be follow-up PR's. From your link, it looks like there may be self-hosted runners that may be useful. It looks like osx_64 could be disabled by specifying |
Checklist
0(if the version changed)conda-smithy(Use the phrase@conda-forge-admin, please rerenderin a comment in this PR for automated rerendering)This PR adds a wrapping output to itk conda packaging (discussed in InsightSoftwareConsortium/ITKPythonPackage#302). This allows ITK remote modules to be packages as conda packages usable in Python conda environments.
I also experimented with using these conda packages as caches for building Python wheels for ITK. While this worked for mac and windows builds, to make manylinux compatible wheels, I believe the conda packages would need to be created within the manylinux docker environments with the correct compilers and toolchains (which may not be possible here).
Also experimented with inclusion of this in the
libitk-feedstockbut ended up moving the work here, seen in this PR.