Skip to content

fix(dataset): fail-fast casting for columns with attached indices#7158

Open
WenDing-Y wants to merge 4 commits into
lance-format:mainfrom
WenDing-Y:fix/cast-with-index-fail-fast
Open

fix(dataset): fail-fast casting for columns with attached indices#7158
WenDing-Y wants to merge 4 commits into
lance-format:mainfrom
WenDing-Y:fix/cast-with-index-fail-fast

Conversation

@WenDing-Y

Copy link
Copy Markdown
Contributor

Summary

Currently, calling alter_columns with a cast on a column that has an attached
index silently drops the index without any warning or error. This has caused
production incidents where vector search silently regressed to brute-force scan.

This PR adds a fail-fast check before any cast is performed: if a target
column has any index attached, the operation is rejected with a clear error
message that names the affected column(s), the index(es), and the required
remediation (drop_index() → cast → rebuild index).

Changes

  • Added index-awareness check in alter_columns that scans cast target columns
    for attached indices
  • Returns a descriptive Error::invalid_input with column names, index names,
    and remediation steps when a conflict is detected
  • Updated existing tests that previously relied on silent index-drop behavior
    to explicitly drop_index() before casting
  • Added new test test_alter_columns_cast_fails_with_attached_index covering
    the IVF_PQ index path: verifies the error message, schema integrity, index
    preservation, and that the cast succeeds after a manual drop

Breaking Change ⚠️

Callers that previously depended on the silent index-drop behavior must now
explicitly call drop_index() before alter_columns with a cast, then rebuild
the index afterward.

Implement fail-fast behavior for casting columns with attached indices to
prevent silent index drops. Update tests to ensure proper error messages and
dataset integrity when attempting to cast indexed columns.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@github-actions github-actions Bot added the bug Something isn't working label Jun 8, 2026
@codecov

codecov Bot commented Jun 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.42857% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
rust/lance/src/dataset/schema_evolution.rs 96.42% 0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

Drop the existing index before altering the column data type.
@github-actions github-actions Bot added the A-python Python bindings label Jun 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-python Python bindings bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant