Skip to content

Add from source wrapping#50

Open
CavRiley wants to merge 9 commits into
conda-forge:mainfrom
CavRiley:add-from-source-wrapping
Open

Add from source wrapping#50
CavRiley wants to merge 9 commits into
conda-forge:mainfrom
CavRiley:add-from-source-wrapping

Conversation

@CavRiley

Copy link
Copy Markdown

Checklist

  • Used a personal fork of the feedstock to propose changes
  • Bumped the build number (if the version is unchanged)
  • Reset the build number to 0 (if the version changed)
  • Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
  • Ensured the license file is being packaged.

This PR adds a wrapping output to itk conda packaging (discussed in InsightSoftwareConsortium/ITKPythonPackage#302). This allows ITK remote modules to be packages as conda packages usable in Python conda environments.

I also experimented with using these conda packages as caches for building Python wheels for ITK. While this worked for mac and windows builds, to make manylinux compatible wheels, I believe the conda packages would need to be created within the manylinux docker environments with the correct compilers and toolchains (which may not be possible here).

Also experimented with inclusion of this in the libitk-feedstock but ended up moving the work here, seen in this PR.

@conda-forge-admin

conda-forge-admin commented May 25, 2026

Copy link
Copy Markdown
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe/meta.yaml:

  • ℹ️ The recipe is not parsable by parser conda-souschef (grayskull). This parser is not currently used by conda-forge, but may be in the future. We are collecting information to see which recipes are compatible with grayskull.
  • ℹ️ The recipe is not parsable by parser conda-recipe-manager. The recipe can only be automatically migrated to the new v1 format if it is parseable by conda-recipe-manager.
  • ℹ️ 'Free Disk Space' is deprecated.
    Deprecated. Use workflow_settings.free_disk_space instead.
    Free up disk space building.
    The following components can be cleaned up: apt, cache, docker.
    When set to true, only apt and cache are cleaned up.
    Set it to the full list to clean up all components.

This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/27852953943. Examine the logs at this URL for more detail.

@blowekamp

Copy link
Copy Markdown

@thewtex Can you approve the workflow runs for this PR?

@thewtex

thewtex commented Jun 9, 2026

Copy link
Copy Markdown
Member

@blowekamp @CavRiley yes, triggered! ✨

@CavRiley

CavRiley commented Jun 9, 2026

Copy link
Copy Markdown
Author

@thewtex @blowekamp thank you!

FYI: the build process for these is successful on my forks GHAs but fail at validation due to the new itk-devel output and/or job timeouts.

@blowekamp

Copy link
Copy Markdown

@thewtex @blowekamp thank you!

FYI: the build process for these is successful on my forks GHAs but fail at validation due to the new itk-devel output and/or job timeouts.

Is the second package really needed?

Last time I added a second package I did this:
conda-forge/admin-requests#2070

@CavRiley

CavRiley commented Jun 9, 2026

Copy link
Copy Markdown
Author

Is the second package really needed?

Depends on the remote modules need or willingness to support conda environments. The second package only holds the build tree, so if the need is not there, then it can be removed

@blowekamp

Copy link
Copy Markdown

Is the second package really needed?

Depends on the remote modules need or willingness to support conda environments. The second package only holds the build tree, so if the need is not there, then it can be removed

Most modules are being moved to the main ITK repository right now. Unless you have an immediate need or application for itk-devel, I would delay added it.

CavRiley added 6 commits June 9, 2026 18:51
Replace the PyPI wheel repackaging with a from-source ITK build that
produces a self-contained `itk` package: the Python module plus the
C++ runtime libraries it links against.

Target Python 3.10, 3.11 and 3.12 across linux-64, linux-aarch64,
osx-64, osx-arm64 and win-64.
The from-source ITK build with Python wrapping fills the ~14 GB
GitHub-hosted runner, so the test phase failed extracting the itk
package with "No space left on device". Set
github_actions.free_disk_space so the rerendered workflow reclaims
runner space before building. Re-rendered.
The build requirement `cmake    >=3.16.3, <3.30.0` had a space after
the comma, which conda-build can misparse (treating `<3.30.0` as a
build string). Use `cmake >=3.16.3,<3.30.0`.
Windows and macOS built on Azure-hosted agents (2 and 3 cores), where a
full 2;3;4-dim ITK wrapping build did not finish within the 6 h job cap
(Windows especially, on 2 cores). Route osx + win to GitHub Actions
(windows-latest and macos-15-intel are 4 cores) so the builds fit the
same 6 h limit with margin. Re-rendered.
osx_arm64 was cross-compiled on the osx_64 (Intel) runner via
build_platform: osx_arm64: osx_64, which skips the arm64 tests and
ignores GitHub's native Apple-Silicon runners. Build it natively on
macos-latest (arm64) via provider: osx_arm64: github_actions, so the
arm64 Python wrapping is import-tested on the target architecture.
Re-rendered.
@CavRiley CavRiley force-pushed the add-from-source-wrapping branch from d67502d to 199a610 Compare June 9, 2026 23:52
@CavRiley

Copy link
Copy Markdown
Author

@thewtex We may need to request more CI compute resources on GHA or Azure runners (seen in conda-forge admin requests).

Also should consider which Python versions to target, or whether to build against the stable ABI mirroring the wheels.

@blowekamp

Copy link
Copy Markdown

@thewtex We may need to request more CI compute resources on GHA or Azure runners (seen in conda-forge admin requests).

Also should consider which Python versions to target, or whether to build against the stable ABI mirroring the wheels.

It look to me that the osx intel build is running on an OS ARM build. The emulation is slowing things down.

I am not sure how or if GHA can be configured to use the intel image.

Comment thread .github/workflows/conda-build.yml Outdated
Comment on lines +61 to +65
- CONFIG: osx_64_python3.10.____cpython
STORE_BUILD_ARTIFACTS: False
UPLOAD_PACKAGES: True
os: macos
runs_on: ['macos-15-intel']

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@blowekamp I would think this configuration enforces the intel runner image. But the Set up job step for the osx-64 jobs use the macos-15 image name. Is there a possibility that image name refer to both ARM and intel images?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think you are correct.

I was looking for this string in the "Set up job" step in the action and didn't see it. Look again at the log, I don't see clear information about what the architecture is for the runner.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On line 732 of the "Build on macOS" step during conda configuration there is this line in a conda info call:

virtual packages : __archspec=1=skylake

As far as I know, the emulator would have reported an "M" architecture if emulated, but I could be wrong

@blowekamp

Copy link
Copy Markdown

With Mac OSX Tahoe 27, intel's are no longer supported. With this legacy configuration being the only one timing out perhaps this PR can move forward with out it?

@CavRiley

Copy link
Copy Markdown
Author

With Mac OSX Tahoe 27, intel's are no longer supported. With this legacy configuration being the only one timing out perhaps this PR can move forward with out it?

I think that is reasonable.

@hjmjohnson may be able to provide an Intel mac as a self-hosted runner but I cannot confirm that.

@CavRiley CavRiley marked this pull request as ready for review June 19, 2026 13:34
@CavRiley CavRiley requested a review from thewtex as a code owner June 19, 2026 13:34
@thewtex

thewtex commented Jun 19, 2026

Copy link
Copy Markdown
Member

@thewtex We may need to request more CI compute resources on GHA or Azure runners (seen in conda-forge admin requests).

Also should consider which Python versions to target, or whether to build against the stable ABI mirroring the wheels.

@CavRiley yes, good ideas. These could be follow-up PR's. From your link, it looks like there may be self-hosted runners that may be useful.

It looks like osx_64 could be disabled by specifying build_platform in conda-forge.yml.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants