pixi environment support#112
Open
davidrich27 wants to merge 10 commits into
Open
Conversation
- Default build type changed to RelWithDebInfo (avoids protobuf Debug/Release ABI mismatch with conda-built libprotobuf) - Added all team members to pixi.toml authors - configure.sh passes Protobuf_INCLUDE_DIR to ensure pixi headers are used instead of system headers - Added deps/libtorch to .gitignore - Added matplotplusplus warning suppression flags for GCC 13 - Updated pixi.lock Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replaced inline defaults in init.sh with larch-build.env.template - Added LARCH_PROTOBUF_PATH option for custom protobuf installs (enables Debug builds by avoiding conda protobuf ABI issue) - Simplified init.sh to copy template, load-config.sh to source and export - Added pixi build instructions to README.md Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- init.sh: env var overrides applied on top of template defaults - configure.sh: use bash array for cmake args (safe with spaces) - configure.sh: platform-aware protobuf lib detection (.so/.dylib) - configure.sh: replace GNU find -printf with portable alternative - configure.sh: properly quote nested dirname calls - load-config.sh: add set -euo pipefail - Extract test setup into scripts/run-tests.sh (DRY) - Template: leave LARCH_BUILD_JOBS empty for auto-detect Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Split pixi.toml deps into shared/Linux/macOS sections - Linux: gcc/gxx, ucx, yasm, libhwloc - macOS: tbb-devel, libprotobuf (compiler from Xcode) - configure.sh auto-detects macOS and enables USE_SYSTEM_TBB, sets Protobuf_ROOT to CONDA_PREFIX - Updated pixi.lock with macOS package resolutions Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a pixi environment for building larch, providing reproducible builds without system dependencies. All compilers, libraries, and tools come from conda-forge via pixi.
Pixi build workflow
Build settings (build dir, build type, USE_USHER, USE_NETAM, libtorch path, extra cmake flags, job count) are stored in
larch-build.envat the project root. This file is gitignored and user-editable betweeninitandconfigure. A copy is saved into the build directory at configure time for reference.What's included
pixi.toml— workspace config with all dependencies (GCC 13, cmake, protobuf, boost, openmpi, pytorch-cpu, etc.) and task definitionsscripts/—init.sh,configure.sh,load-config.sh,fetch-libtorch.sh-std=c++2binstead of-std=c++23)-Wno-stringop-truncation -Wno-array-bounds -Wno-stringop-overread)configure.shpasses-DProtobuf_INCLUDE_DIRto ensure pixi's protobuf headers are used instead of system headersRelWithDebInfo(matches Ognian's recommended build, and avoids a protobuf ABI issue with Debug builds — see below)Known issue: protobuf Debug/Release ABI mismatch
When building in Debug mode with conda/pixi protobuf, linking fails with an undefined reference to
InternalMetadata::~InternalMetadata(). This is because conda builds protobuf in Release mode (withNDEBUG), which inlines and doesn't export that symbol. Debug builds expect it inlibprotobuf.so.RelWithDebInfo and Release builds work fine (they define
NDEBUG). The default build type is set toRelWithDebInfoto avoid this.Possible solutions if Debug builds are needed in the future:
assert()globally (larch uses its ownAssert()macro so impact is minimal)Test results
Tested on both local machine and remote compute cluster — results match.
Pre-existing failures (all 3 builds): dagbin simultaneous write, dagbin seeded, ambiguous VCF with MatOptimize.
Regression from v0.1.0 → main:
MATConversion: Check SampleIds(still fails on this branch — not introduced by pixi).Fixed on this branch:
RF distance: counter(viafix-macos-buildbase:RFDistance→SumRFDistance).New tests: 51 netam/ML tests (Netam Likelihood, KmerEncoder, Model, ML SPR, S5F) — all pass.
Test plan
🤖 Generated with Claude Code