build(deps-dev): bump delta-spark from 4.2.0 to 4.3.0

[dependabot]

Bumps delta-spark from 4.2.0 to 4.3.0.

Release notes

Sourced from delta-spark's releases.

Delta Lake 4.3.0

We are excited to announce the release of Delta Lake 4.3.0, which delivers new features, performance improvements, and protocol updates across Delta Spark, Kernel, UniForm, Sharing, and Flink. See the highlights below for the marquee changes.

Highlights

• [Spark] Unity Catalog Delta REST API integration: Spark now supports the UC Delta API, using Unity Catalog as the source of truth for managed Delta tables. With server-side commit validation, server-advertised table features, and intent-based metadata updates, this integration provides consistent, safe access for Spark today and sets the foundation for future support across Flink, Trino, and other engines.
• [Spark] Selective data replacement with replaceOn and replaceUsing DataFrame APIs: Spark now supports selectively replacing table data with the result of a DataFrame. Use replaceUsing to replace rows that match on specified columns, or replaceOn to replace rows that satisfy a user-defined condition.
• [UniForm] Atomic + incremental Iceberg conversion and Spark 4.1 support: UniForm now writes Iceberg metadata atomically with the Delta commit, incrementally converts only the changed log range.
• [Sharing] Streaming and CDF Support: Delta Sharing in 4.3.0 improves Spark Structured Streaming and Batch CDF support with automatic Delta response resolution, Parquet-to-Delta streaming conversion, shared DeltaFormattable streaming CDF support, and Trigger.AvailableNow support for shared tables.

Delta Spark

Delta Spark 4.3.0 is built on Apache Spark 4.1.0 and Apache Spark 4.0.1. As with Apache Spark, we publish Maven artifacts for Scala 2.13.

• Maven artifacts for Spark 4.1.0:
  • delta-spark_4.1_2.13
  • delta-connect-client_4.1_2.13
  • delta-connect-common_4.1_2.13
  • delta-connect-server_4.1_2.13
  • delta-contribs_2.13
  • delta-storage
  • delta-storage-s3-dynamodb
• Maven artifacts for Spark 4.0.1:
  • delta-spark_4.0_2.13
  • delta-connect-client_4.0_2.13
  • delta-connect-common_4.0_2.13
  • delta-connect-server_4.0_2.13
  • delta-contribs_2.13
  • delta-storage
  • delta-storage-s3-dynamodb
• Backward-compatibility artifacts (no Spark version in the name; default to Spark 4.1.0):
  • delta-spark_2.13
  • delta-connect-client_2.13
  • delta-connect-common_2.13
  • delta-connect-server_2.13
• Python artifact:
  • delta-spark

The key features of this release are:

• Unity Catalog Delta REST API integration : Delta Spark now uses the new UC Delta REST API by default for UC-managed Delta tables. Managed Delta operations, including table loads, CREATE / CTAS, REPLACE, and all other metadata-changing writes such as DML, schema evolution, auto-merge, and supported ALTER TABLE updates, are routed through the new API. Non-Delta tables and external tables, including name-based and path-based access, continue to use the legacy delegate.
• Delta DSv2 Connector with Delta Kernel (Experimental) : adds new features like support for batch writes to Catalog-Managed tables, Streaming source support including all read options and Catalog-driven batch CDC (SELECT … CHANGES FROM VERSION/TIMESTAMP, DV-aware) - gated behind spark.databricks.delta.changelogV2.enabled.
• V2 Checkpoint performance hardening for large tables: V2 checkpoints now default to 50,000 actions per sidecar, so sidecar files are automatically split into multiple parts and checkpoint writes parallelize better out of the box.
• Selectively replace data with 'replaceOn' and 'replaceUsing' DataFrame APIs: Use these options to replace part of the table with the result of a DataFrame. ‘replaceOn’ replaces rows that match a user-defined condition. ‘replaceUsing’ replaces rows where specified columns are equal. See Delta Lake API doc.
• Implicit casting for DataFrame by-name writes: DataFrame writes that match by column name, except save() and saveAsTable().mode(“overwrite”), now apply Spark's implicit casts to align source values with the target schema, matching SQL INSERT BY NAME behavior.
• Variant column statistics on write: Delta Spark now collects min/max statistics for Variant columns at write time, enabling data skipping on Variant-shredded tables.
• REPLACE TABLE / RTAS / DPO production hardening: concurrency, source-materialization, and operational-metric coverage now spans the full set of REPLACE-style DataFrame writes introduced in Delta 4.2.

Other notable features and bug-fixes include:

... (truncated)

Commits

• 76c12a2 Enable changelog support (#7007)
• 219977b [4.3] Support arbitrary refs and explicit version override in UC setup script...
• e1004f1 [4.3][Spark] Propagate clustering DomainMetadata through commitLarge (#6992) ...
• bd4ba66 [4.3][Spark] Carry forward unrecognized delta.* properties on catalog-managed...
• 2e41166 [4.3][build]Fix UC setup cache key collisions and local Spark-version switchi...
• 21aa263 [Spark][4.3] Fix Parquet schema converter compatibility across Spark patches ...
• c435979 [Spark][4.3] Read-time CDF via table_changes() in DSv2 (#6857) (#7004)
• 2cef58b [Release] Prepare Delta 4.3.0: remove Spark 4.2 preview, set release version ...
• 295bb77 [Spark] Store and use sidecar file schema in V2 Checkpoint metadata tags (#6912)
• 3be366e [Doc] Add CheckpointMetadata tags in PROTOCOL.md (#6982)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)