Architecture: LinkML-Map CLI Migration
The core transformation engine has been completely rewritten. map_data.py and the StreamWriter/MultiStreamWriter infrastructure have been removed (~1,300 lines deleted) and replaced with direct calls to linkml-map derive from the Makefile. This is the culmination of #275 and the most significant architectural change since v1.0.
- Transformation specs are now composed at build time via
compose_specs.pyand passed directly to thelinkml-mapCLI pipeline.Makefilegained significant new orchestration logic for spec composition and target schema handlingbdc-workflow.shrewritten to support the new CLI-based flow, including--trans-specfor alternate spec repos- Integration tests rewritten with expanded assertions covering field values, cross-table derivations, unit conversions, and error handling
- New unit tests for spec composition (
test_compose_specs.py)
(#290, #296, closes #275, supersedes #253, #256)
Pipeline Behavior
- Non-zero exit on mapping errors — pipeline now exits non-zero even in non-strict mode when mapping errors occur, making failures visible in CI and automation (#286)
- Alternate trans-spec support —
--trans-specCLI flag allows specifying a different trans-spec repo/branch/path for testing against development specs (#296)
BDC Deployment
- Three-tier container deployment — formalized dev → test → prod container registry separation.
docker-devfor pipeline testing,docker-push-7bridgesfor data validation,bdc-v*tags for production releases (#298) - Seven Bridges API scripts — complete Python rewrite of task submission, monitoring, status checking, and log retrieval. Replaces ad-hoc bash scripts (#293)
- Release process documented in
RELEASING.md - Removed unused PyPI publish workflow (#298)
Toolchain & Dependencies
- linkml-map 0.5.2 (from 0.4.0) — major version bump enabling CLI-based transformations
- schema-automator 0.5.5 (from 0.5.4)
- linkml 1.10.0 / linkml-runtime 1.10.0
- Removed stale
[tool.black]config and unusedrstfmt/blackdependencies
Testing & Data
- MeasurementObservationSet added to toy data (#285)
- Blood pressure transformation spec added to toy data for broader derivation coverage
- Error spec test data for validating strict/non-strict error handling
- Target schema added to toy data for CLI-based mapping