chore(ci): migrate Python dependency locking from pip-tools to uv

[yogarajalakshmi-s]

What this PR does / why we need it

Migrates Python dependency locking from pip-tools (pip-compile) to uv, aligning kubeflow/trainer with kubeflow/sdk tooling.

• Add pyproject.toml for dataset and model initializers
• Replace requirements-lock.txt with uv.lock for all 3 components
• Update osv-scanner.yaml to use uv lock --upgrade-package
• Update validate-lockfile.yaml to use uv lock --check
• Add helper scripts adapted from kubeflow/sdk
• Remove hack/scripts/update-source-dep.py

Testing

• uv lock --check passes for all 3 components locally
• osv-scanner reads all 3 uv.lock files successfully (didn't get any issues)
• Full workflow testing will happen in CI after /ok-to-test is applied

Note

All changes were written, reviewed and tested locally by me. (AI tooling was used to understand the codebase structure)

Which issue this PR fixes:
Part of #3535

Checklist:

• Docs included if any changes are user facing

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign tenzen-y for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[github-actions]

🎉 Welcome to the Kubeflow Trainer! 🎉

Thanks for opening your first PR! We're happy to have you as part of our community 🚀

Here's what happens next:

• If you haven't already, please check out our Contributing Guide for repo-specific guidelines and the Kubeflow Contributor Guide for general community standards.
• Our team will review your PR soon! cc @kubeflow/kubeflow-trainer-team

Join the community:

• Slack: Join our #kubeflow-trainer Slack channel.
• Meetings: Attend the Kubeflow AutoML and Training Working Group bi-weekly meetings.

Feel free to ask questions in the comments if you need any help or clarification!
Thanks again for contributing to Kubeflow! 🙏

[Copilot]

Pull request overview

Migrates Python dependency locking in the repo from pip-tools (pip-compile-generated requirements-lock.txt) to uv (uv.lock), and updates CI workflows (lockfile validation + OSV auto-fix) to operate on uv.lock across the dataset initializer, model initializer, and Python API components.

Changes:

• Added pyproject.toml + uv.lock for the dataset/model initializers and replaced existing requirements-lock.txt lockfiles with uv.lock across all Python components.
• Updated GitHub workflows to validate lockfiles via uv lock --check and run OSV scanning/fixing against uv.lock.
• Added helper scripts for OSV auto-fix flows and removed the legacy hack/scripts/update-source-dep.py.

Reviewed changes

Copilot reviewed 11 out of 14 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
.github/workflows/validate-lockfile.yaml	Switches lockfile sync validation from pip-compile output comparison to uv lock --check.
.github/workflows/osv-scanner.yaml	Updates Python OSV scan inputs to uv.lock and rewires the auto-fix logic to use uv lock --upgrade-package + optional overrides.
.github/scripts/update_overrides.py	Adds a script to manage [tool.uv].override-dependencies entries in pyproject.toml.
.github/scripts/extract_version.py	Parses uv tree output to extract an upgraded package version.
.github/scripts/compare_versions.py	Compares versions (PEP 440) to decide whether an override is needed.
cmd/initializers/dataset/pyproject.toml	Introduces uv-compatible dependency declaration for dataset initializer.
cmd/initializers/dataset/uv.lock	New uv lockfile for dataset initializer.
cmd/initializers/dataset/requirements-lock.txt	Removed pip-tools lockfile.
cmd/initializers/model/pyproject.toml	Introduces uv-compatible dependency declaration for model initializer.
cmd/initializers/model/uv.lock	New uv lockfile for model initializer.
cmd/initializers/model/requirements-lock.txt	Removed pip-tools lockfile.
api/python_api/uv.lock	New uv lockfile for Python API.
api/python_api/requirements-lock.txt	Removed pip-tools lockfile.
hack/scripts/update-source-dep.py	Removes the legacy dependency-updater used by the old pip-tools OSV auto-fix flow.

[Fiona-Waters]

Thanks for this contribution @yogarajalakshmi-s
Looks good so far. One thing I notice is a missing test file. The kubeflow/sdk repo includes a test_scripts.py with pytest coverage for the helper scripts. Since the scripts in this PR are adapted from there, it would be worth bringing over the test file too — these run in a nightly security workflow where silent failures are hard to notice.

[yogarajalakshmi-s]

Thanks for this contribution @yogarajalakshmi-s Looks good so far. One thing I notice is a missing test file. The kubeflow/sdk repo includes a test_scripts.py with pytest coverage for the helper scripts. Since the scripts in this PR are adapted from there, it would be worth bringing over the test file too — these run in a nightly security workflow where silent failures are hard to notice.

Thank you for your review, @Fiona-Waters! I will include the test script.

[andreyvelich]

/ok-to-test