Version 2.3.0

To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here or via UI here

Install python SDK (python 3.7 above) by running:

python3 -m pip install kfp kfp-server-api --upgrade

See the Change Log

What's Changed

• chore: Update RELEASE.md by @chensun in #10767
• fix(components): Add input param autorater_prompt_parameters to online_evaluation_pairwise component by @copybara-service in #10759
• feat(components): create infer preprocessor component by @copybara-service in #10765
• feat(components): create utility class for preprocessors and use it in rlhf preprocessor and infer preprocessor by @copybara-service in #10768
• chore(sdk): drop Python 3.7 for kfp sdk by @rickyxie0929 in #10750
• chore: Promote @rimolive as a backend approver by @chensun in #10780
• feat(components): use preprocessor utility methods for the upload model graph by @copybara-service in #10774
• chore: upgrade node to v18 by @HumairAK in #10794
• feat(GH workflow): migrate periodic functional tests to GH actions by @shruti2522 in #10751
• chore: Add Container Engine and Image Tag parameters to backend Makefiles by @gmfrasca in #10725
• chore: add meaningful logs to client initialization. by @HumairAK in #10801
• feat(kubernetes_platform): Update kubernetes_platform go package to include EnabledSharedMemory by @hsteude in #10703
• feat(components): Retry on batch prediction internal errors in AutoSxS by @copybara-service in #10799
• fix(components): Add staging and temp locations to prophet trainer component by @copybara-service in #10803
• fix(components): Remove unused import function_based from infer pipeline by @copybara-service in #10802
• docs(components): Bump the python version to 3.8 by @copybara-service in #10764
• chore(components): GCPC 2.14.1 Release by @copybara-service in #10813
• chore(components): Rollback GCPC 2.14.1 Release due to b/339911077 by @copybara-service in #10818
• chore(components): Add TaskError proto by @copybara-service in #10823
• chore(components): GCPC 2.14.1 Release by @copybara-service in #10821
• feat(components): Create the write_user_defined_error function by @copybara-service in #10835
• test: fix CI failure for periodic functional tests by @tmvfb in #10817
• chore(docs): Fix link by @haoxins in #7533
• chore: Update Periodic Functional Tests to run once a day by @DharmitD in #10842
• test: fix version conflict for functional tests by @tmvfb in #10837
• chore: server should skip sample load if path DNE by @DharmitD in #10833
• fix(frontend): reduce list run latency by @droctothorpe in #10797
• feat(components): Support parsing Gemini BP outputs in AutoSxS pipeline by @copybara-service in #10853
• feat(components): Use GetModel integration test to manually test write_user_defined_error function by @copybara-service in #10843
• docs: modified the ConfigMap as Env variable example in README.md with right key_name within use_config_map_as_env() block. by @vamsi-01 in #10855
• fix(sdk): Add required auth scopes to RegistryClient for GCP service accounts credentials by @PChambino in #10819
• chore(deps): bump requests from 2.31.0 to 2.32.2 in /backend/src/apiserver/visualization by @dependabot in #10858
• chore: Change command to apply manifests in the docs by @rimolive in #10864
• fix(manifests): Move metacontroller to the top in kustmization.yaml by @codablock in #10669
• test: Migrate Backend tests to a GHA workflow by @DharmitD in #10871
• feat(backend): mount EmptyDir volumes for launcher write locations by @HumairAK in #10857
• docs(components): internal by @copybara-service in #10874
• feat(components): Add role_field_name and model_name as input parameters to llm_evaluation_preprocessor component to support gemini model's input and output schema by @copybara-service in #10875
• fix(components): add check and add log to call out the fallback to the default model checkpoint and remove the model checkpoint check condition in RLHF GCPC by @copybara-service in #10828
• test: migrate kubeflow-pipeline-e2e-test to GitHub Actions by @hbelmiro in #10887
• feat(components): Support dynamic machine type paramters in CustomTrainingJobOp by @KevinGrantLee in #10883
• fix(kubernetes_platform): fix api-generator docker mount for SELinux by @gregsheremeta in #10890
• feat(components): Add Starry Net forecasting pipeline to public preview by @copybara-service in #10888
• chore(components): Update AutoSxS and RLHF image tags by @copybara-service in #10906
• feat(components): Update Starry Net image tags by @copybara-service in #10920
• chore: Updated Python image to 3.8 in backend/Dockerfile by @hbelmiro in #10941
• Expose starry_net yaml to GitHub by @chensun in #10943
• chore(sdk): release KFP SDK 2.8.0 by @chensun in #10940
• chore: Upgrade Python version to 3.8 in readthedocs config by @chensun in #10945
• feat(frontend&backend): Add UI support for object store customization and prefixes by @HumairAK in #10787
• chore(components): Update AutoSxS and RLHF image tags by @copybara-service in #10958
• chore(components): GCPC 2.15.0 Release by @copybara-service in #10951
• docs(backend): Remove deprecated v2 compatibility mode docs by @droctothorpe in #10956
• Upgrade go version to 1.21 by @DharmitD in #10911
• chore: Add ability to mount self-signed certs to kfp by @DharmitD in #10849
• feat(components): add persistent_resource_id to v1 GCPC custom job components/utils by @copybara-service in #10954
• chore: Added @rimolive, @hbelmiro and @DharmitD as approvers and reviewers for GitHub Actions by @hbelmiro in #10963
• test: Migrate KFP SDK YAPF test to a GHA Workflow by @DharmitD in #10961
• test: Created a new GitHub action to reuse the creation of a KFP cluster by @hbelmiro in #10946
• test: kubeflow-pipeline-upgrade-test migrated to GitHub Actions by @hbelmiro in #10932
• test: Migrate Frontend tests to a GHA Workflow by @DharmitD in #10972
• test: install kfp on github periodic functional tests workflow by @tmvfb in #10859
• chore: Add an option to re-run GHA workflows by @rimolive in #10930
• chore(components): Update Starry Net description and add nan- and zero-threshold args by @copybara-service in #10986
• test: Moved KFP backend visualization tests to GitHub Actions by @hbelmiro in #10984
• test: Migrate KFP SDK Upgrade Test to GHA by @DharmitD in #10989
• test: Migrate KFP SDK Runtime Code Test to GHA by @DharmitD in #10991
• chore(components): Bump image version for Structured Data pipelines by @copybara-service in #11000
• chore(components): Fix bug due to protobuf library being upgraded by pinning protobuf version by @copybara-service in #11007
• chore(components): Add target_field_name as input parameters to llm_evaluation_preprocessor component to support gemini model's input and output schema by @copybara-service in #11028
• chore(components): Bump Starry Net images and enforce that TF Record generation always runs before test set generation to speed up pipelines runs by @copybara-service in #11026
• chore(components): Add check that component in preview.custom_job.utils.create_custom_training_job_from_component doesn't have any parameters that share names with any custom job fields by @copybara-service in #11033
• feat(internal): Adding proto field to support regional quota by @copybara-service in #11001
• feat(kubernetes_platform): Add empty dir mount by @gregsheremeta in #10892
• test: Migrate KFP SDK Execution Tests to GHA by @DharmitD in #10975
• test: Removed mkp tests by @hbelmiro in #10924
• fix(sdk): Throw 'exit_task cannot depend on any other tasks.' error when an ExitHandler has a parameter dependent on other task by @hbelmiro in #11005
• fix(docs): add note about protoc dependency and version by @gregsheremeta in #10895
• chore: Adding @gregsheremeta to SDK reviewers. by @chensun in #11042
• chore: Adding @HumairAK to backend and frontend reviewers by @chensun in #11043
• fix(docs): IR -> Pipeline Spec by @gregsheremeta in #11031
• test: Migrate miscellaneous SDK tests to GHA Workflow by @DharmitD in #11032
• chore: Adding @gregsheremeta to SDK reviewers (kubernetes_platform) by @gregsheremeta in #11044
• test: Migrate KFP SDK Tests to GHA by @DharmitD in #11037
• chore: adding @hbelmiro to kubernetes_platform/OWNERS by @hbelmiro in #11045
• test: Moved kfp-kubernetes-execution-tests to GitHub Actions by @hbelmiro in #10926
• add dev docs for kfp/sdk by @HumairAK in #11046
• feat(components): Support dynamic machine parameters in preview.custom_job.utils.create_custom_training_job_from_component by @copybara-service in #10974
• test: Moved kubeflow-pipelines-samples-v2 to GitHub Actions by @hbelmiro in #11048
• feat(components): internal by @copybara-service in #11050
• feat(component): internal by @copybara-service in #11049
• test: Add script and GitHub Action for kfp-kubernetes-library by @diegolovison in #11006
• chore(components): GCPC 2.16.0 Release by @copybara-service in #11065
• fix: Incorrect typing in samples/cores/loop_parallism and fixing loop_parameter examples by @papagala in #11062
• feat(components): internal by @copybara-service in #11070
• chore: Fixed e2e tests by @hbelmiro in #11072
• chore(components): Bump image version for Structured Data pipelines by @copybara-service in #11073
• feat(components): internal by @copybara-service in #11074
• feat(components): internal by @copybara-service in #11076
• feat(components): internal by @copybara-service in #11081
• feat(api): Add new ResourceSpec proto fields to support dynamic values. by @KevinGrantLee in #11075
• feat(components): internal by @copybara-service in #11082
• feat(components): internal by @copybara-service in #11083
• docs: Fix environment variables names by @diegolovison in #11013
• feat(backend): move comp logic to workflow params by @zazulam in #10979
• feat(component): internal by @copybara-service in #11085
• feat(components): internal by @copybara-service in #11087
• fix(components): Fix to model batch explanation component for Structured Data pipelines by @copybara-service in #11088
• feat(components): Support dynamic values for boot_disk_type, boot_disk_size in preview.custom_job.utils.create_custom_training_job_from_component by @copybara-service in #11055
• chore: Upgrade Argo to v3.4.17 (fixes #10977) by @gmfrasca in #10978
• test: Moved kubeflow-pipelines-manifests to GitHub Actions by @VaniHaripriya in #11066
• fix: re-enable exit hanler test. by @liavweiss in #11100
• fix(frontend): retrieve archived logs from correct location by @droctothorpe in #11010
• feat(component): internal by @copybara-service in #11107
• feat(component): internal by @copybara-service in #11110
• chore(components): GCPC 2.16.1 Release by @copybara-service in #11111
• test: Fail fast when image build fails on tests #11102 by @ElayAharoni in #11115
• fix(components): Use instance.target_field_name format for text-bison models only, use target_field_name for gemini models by @copybara-service in #11118
• chore: Renamed GitHub workflows from *.yaml to *.yml for consistency by @hbelmiro in #11126
• Fix view edit cluster roles by @papagala in #11067
• fix(components): Pass moddel name to eval_runner to process batch prediction's output as per the output schema of model used by @copybara-service in #11119
• feat(components): release LLM Model Evaluation image version v0.7 by @copybara-service in #11128
• chore: Adding @DharmitD to SDK reviewers by @DharmitD in #11131
• test: Kubeflow Pipelines V2 integration Tests by @diegolovison in #11125
• chore: Add make targets for building driver and launcher images by @gmfrasca in #11103
• feat(Backend + SDK): Update kfp backend and kubernetes sdk to support EmptyDir by @gregsheremeta in #10913
• docs:fixing broken links in readme by @Fiona-Waters in #11108
• chore(deps): bump micromatch from 4.0.5 to 4.0.8 in /test/frontend-integration-test by @dependabot in #11132
• Fix: Basic sample tests - sequential is flaky by @diegolovison in #11138
• chore: Wrapped "Failed GetContextByTypeAndName" error for better troubleshooting by @hbelmiro in #11098
• chore(components): Update AutoSxS and RLHF image tags by @copybara-service in #11116
• test: Improvements to wait_for_pods function by @hbelmiro in #11162
• fix(frontend): fixes filter pipeline text box shows error when typing anything in it. Fixes #10241 by @ElayAharoni in #11096
• correct artifact preview behavior in UI by @HumairAK in #11059
• chore: Added DCO link to PR template by @hbelmiro in #11176
• chore(backend): Update driver and launcher licenses by @chensun in #11177
• chore(backend): update driver and launcher default images by @chensun in #11178
• chore: Add instructions for releasing driver and launcher images by @chensun in #11179
• test: Fixed kfp-runtime-tests to run on master branch by @hbelmiro in #11158
• (fix): reduce executor logs by @HumairAK in #11169
• chore: add PaulinaPacyna and ouadakarim as reviewers by @chensun in #11180
• test: Move run-all-gcpc-modules to GitHub Actions by @aman23bedi in #11157
• fix(sdk): Kfp support for pip trusted host by @diegolovison in #11151
• chore(sdk): Loosening kubernetes dependency constraint by @egeucak in #11079
• chore: Remove unwanted Frontend test files by @DharmitD in #10973
• fix(ui): fixes empty string value in pipeline parameters by @jan-stanek in #11175
• chore(backend): update driver and launcher default images by @chensun in #11182

New Contributors

• @shruti2522 made their first contribution in #10751
• @tmvfb made their first contribution in #10817
• @vamsi-01 made their first contribution in #10855
• @PChambino made their first contribution in #10819
• @codablock made their first contribution in #10669
• @diegolovison made their first contribution in #11006
• @papagala made their first contribution in #11062
• @zazulam made their first contribution in #10979
• @VaniHaripriya made their first contribution in #11066
• @liavweiss made their first contribution in #11100
• @ElayAharoni made their first contribution in #11115
• @Fiona-Waters made their first contribution in #11108
• @aman23bedi made their first contribution in #11157
• @egeucak made their first contribution in #11079
• @jan-stanek made their first contribution in #11175

Full Changelog: 2.2.0...2.3.0