feat: populate enriched IndexContent fields in dir namespace ListTableIndices#7109
Draft
wjones127 wants to merge 1 commit into
Draft
feat: populate enriched IndexContent fields in dir namespace ListTableIndices#7109wjones127 wants to merge 1 commit into
wjones127 wants to merge 1 commit into
Conversation
…eIndices Previously, the directory namespace's list_table_indices discarded everything from describe_indices except name/uuid/columns. This populates the enriched IndexContent fields (index_type, type_url, num_indexed_rows, size_bytes, num_segments, created_at, index_version, index_details) from the IndexDescription already in hand, avoiding an N+1 DescribeTableIndexStats call per index. dir.rs is the only backend in this repo that constructs IndexContent. Depends on lance-format/lance-namespace#349, which adds these fields to the generated reqwest client. Until that publishes, the dependency points at the PR branch; bump to the published version before merging. Closes lance-format#7101 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
github-actions
Bot
added
A-deps
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Previously, the directory namespace's
list_table_indicesdiscarded everything fromdescribe_indicesexceptname/uuid/columns. This populates the enrichedIndexContentfields from theIndexDescriptionalready in hand, avoiding an N+1DescribeTableIndexStatscall per index:index_type←index_type()type_url←type_url()num_indexed_rows←rows_indexed()size_bytes←total_size_bytes()(staysNonefor legacy indices without file tracking)num_segments←segments().len()created_at← mincreated_atacross segments, RFC3339-formatted (staysNonefor legacy indices)index_version← first segment'sindex_versionindex_details←details().ok()dir.rsis the only backend in this repo that constructsIndexContent; the REST adapter lives in lance-namespace and is covered by lance-format/lance-namespace#349.test_list_table_indicesis extended to assert the new fields for both a scalar (BTree) and an IVF_FLAT vector index.Blocked on
Depends on lance-format/lance-namespace#349, which adds these fields to the generated
lance-namespace-reqwest-client. The latest published version (0.8.0) lacks them, so the dependency currently points at the PR branch via git. Before merging: swap back to a published version pin once #349 merges and publishes (see theTODOinCargo.toml).Closes #7101
🤖 Generated with Claude Code