feat(index): support raw-query ivf rq search #7078
Merged
Claude / Claude Code Review
completed
Jun 8, 2026 in 33m 38s
Code review found 1 important issue
Found 3 candidates, confirmed 3. See review comments for details.
Details
| Severity | Count |
|---|---|
| 🔴 Important | 1 |
| 🟡 Nit | 2 |
| 🟣 Pre-existing | 0 |
| Severity | File:Line | Issue |
|---|---|---|
| 🔴 Important | rust/lance-linalg/src/simd/dist_table.rs:149-158 |
avx512: sum_4bit_dist_table reads past buffer for odd code_len |
| 🟡 Nit | python/python/tests/test_vector_index.py:1083-1112 |
test: multi-bit IVF_RQ tests assert num_rows, not recall |
| 🟡 Nit | docs/src/format/index/vector/index.md:347-350 |
docs: __error_factors column missing from RQ format spec |
Annotations
Check failure on line 158 in rust/lance-linalg/src/simd/dist_table.rs
claude / Claude Code Review
avx512: sum_4bit_dist_table reads past buffer for odd code_len
The AVX2 fast path here got a bounds-check (`if i + 32 >= codes.len() { continue; }`) so the second 32-byte load is skipped for odd `code_len` — added by commit 266b784 to handle dim=8/24 (code_len=1/3) IVF_RQ. The matching AVX-512 C implementation in `rust/lance-linalg/src/simd/dist_table.c` was not updated: it still unconditionally executes `_mm512_loadu_si512(&codes[i])` and `_mm512_loadu_si512(&dist_table[i])` (64-byte loads) in a `for (i = 0; i < code_length; i += 64)` loop. The Rust dispat
Check warning on line 1112 in python/python/tests/test_vector_index.py
claude / Claude Code Review
test: multi-bit IVF_RQ tests assert num_rows, not recall
The new multi-bit IVF_RQ tests `test_create_ivf_rq_multi_bit_searches_l2_and_cosine` (python/python/tests/test_vector_index.py:1070-1112) and `test_build_ivf_rq_multi_bit_persists_split_codes_and_searches` (rust/lance/src/index/vector/ivf/v2.rs:4208-4251) only assert that search returns `num_rows == 10` and never compute recall against a ground-truth k-NN. This violates CLAUDE.md:111 ("Vector index tests must assert recall metrics (>=0.5 threshold), not just verify creation succeeds") for the ce
Check warning on line 350 in docs/src/format/index/vector/index.md
claude / Claude Code Review
docs: __error_factors column missing from RQ format spec
The PR persists a new `__error_factors` float32 column for raw-query IVF_RQ indexes (default for all newly created indexes) but the format spec docs are not updated to mention it. The RQ auxiliary column table (around lines 195-203) and the Appendix 2 `pa.schema(...)` example (around lines 355-361) still list only `_rabit_codes`, `__add_factors`, `__scale_factors`, and the ex-code columns — anyone implementing a reader from this spec would miss `__error_factors`. Please add a row for it to both
Loading