[Spark] Support TRUNCATE TABLE for DeltaTableV2

[foss-contributor]

Which Delta project/connector is this regarding?

• Spark
• Standalone
• Flink
• Kernel
• Other (fill in here)

Description

This PR adds TRUNCATE TABLE support for Delta tables backed by DeltaTableV2.

DeltaTableV2 already advertises the TRUNCATE table capability, but it did not implement Spark's
TruncatableTable interface. This change wires truncate through the Delta transaction path so the
operation removes active files, records a TRUNCATE operation in Delta history, preserves row
tracking metadata tags, and keeps table metadata intact.

The PR also adds coverage for core Delta tables and Unity Catalog Delta tables:

• Adds DeltaTruncateTableSuite for Spark/Delta truncate behavior.
• Adds UCDeltaTableDDLTest as a UC DDL integration suite with truncate coverage for both external
  and managed UC Delta tables.

How was this patch tested?

Added unit and integration tests covering TRUNCATE TABLE cases:

• Covering Delta tables referenced by catalog, by path, as well as UC managed and external tables
• Delta unit tests cover truncating non-empty tables, empty-table no-op behavior, repeated/idempotent truncate tokens, append-only atomic failure, partitioned-table metadata preservation, and partition-spec rejection.
• UC integration tests cover truncate across managed and external UC Delta tables
• Metadata preservation checks verify truncate keeps protocol/features, schema, partition metadata, stable DESCRIBE DETAIL fields, UC table id, and stable SHOW TBLPROPERTIES values unchanged while removing active files.
• Metrics are emitted with relevant data for TRUNCATE TABLE operation

Test commands run:

build/sbt 'spark/testOnly org.apache.spark.sql.delta.DeltaTruncateTableSuite org.apache.spark.sql.delta.DeltaTruncateTableWithCatalogOwnedBatch1Suite'

build/sbt "spark/testOnly org.apache.spark.sql.delta.TruncateMetricsSuite"

build/sbt sparkUnityCatalog/'testOnly io.sparkuctest.UCDeltaTableDDLTest'

build/sbt 'spark/testOnly org.apache.spark.sql.delta.catalog.DeltaTableV2Suite'

build/sbt 'spark/testOnly org.apache.spark.sql.delta.CustomCatalogSuite'

build/sbt 'sparkUnityCatalog/test'

Does this PR introduce any user-facing changes?

Yes.

Delta tables backed by DeltaTableV2 now support Spark SQL TRUNCATE TABLE. The operation removes
all active data files through a Delta transaction while preserving table metadata.

Before change on spark-sql error:

spark-sql ()> TRUNCATE TABLE main.test.table;
Table does not support truncates: main.test.table

After change:

spark-sql ()> TRUNCATE TABLE main.test.table;
Time taken: 13.308 seconds

[openinx]

Generally looks good to me, pls address the remaining comments. I will merge it once all are ready. thanks @foss-contributor for the great work!