perf(migrations): skippable extension reconcile + drop unused global vector index#2309
Merged
Conversation
The post-migration step (ensure_vector_extension / ensure_text_search_extension) runs per tenant schema, each opening a fresh connection and probing the catalog. Across tens of thousands of tenant schemas this dominates the migration job, even though the step only does real work when the configured backend (HINDSIGHT_API_VECTOR_EXTENSION / HINDSIGHT_API_TEXT_SEARCH_EXTENSION) differs from a schema's existing index/column shape — a rare, operator-driven change that, on populated tables, refuses and asks you to re-embed anyway. `run_migrations_for_schemas` already accepts `ensure_extensions`; this just exposes it on the CLI as `--skip-extension-reconcile` (default off, so behavior is unchanged) so an operator who has NOT changed the backend can skip the reconcile on a routine re-migration over many tenants. A backend change still needs a normal run to reshape the indexes. Test: parametrized check that the flag threads ensure_extensions through to run_migrations_for_schemas.
2b792b8 to
bbde4ea
Compare
…index For per-bank vector backends (pgvector / pgvectorscale / vchord) every vector search is bank + fact_type scoped and served by the per-(bank, fact_type) partial indexes created at bank-creation time (bank_utils.create_bank_vector_indexes). The global idx_memory_units_embedding is never chosen by the planner when bank_id is in the WHERE clause — which is exactly why migration d5e6f7a8b9c0 drops it for these backends. Verified via EXPLAIN: the semantic query uses idx_mu_emb_* whether or not the global index exists. ensure_vector_extension nonetheless recreated that global index on a fresh/empty schema (the "No embedding index found, will create it" path), adding an unused, write-amplifying index back to every schema. Skip creating it for per-bank backends. scann (which genuinely uses a global filtered index) is unaffected. Test: a fresh pgvector schema run through migrate + ensure_vector_extension ends up with no idx_memory_units_embedding (fails without the fix).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two changes that trim the post-migration extension reconcile
(
ensure_vector_extension/ensure_text_search_extension), which runs pertenant schema and dominates the migration job at large tenant counts.
1.
--skip-extension-reconcileonrun-db-migrationThe reconcile only does real work when the configured backend
(
HINDSIGHT_API_VECTOR_EXTENSION/HINDSIGHT_API_TEXT_SEARCH_EXTENSION) differsfrom a schema's existing index/column shape — a rare, operator-driven change (and
on populated tables it refuses and asks you to re-embed anyway). On a routine
no-change re-migration over many tenants it's pure per-tenant overhead.
run_migrations_for_schemasalready takes anensure_extensionsparameter; thisexposes it on the CLI as
--skip-extension-reconcile(default off → behaviorunchanged). A backend change still needs a normal run to reshape the indexes —
the flag is an explicit "I haven't changed the backend" assertion.
2. Stop creating the unused global
memory_unitsvector indexFor per-bank backends (pgvector / pgvectorscale / vchord), every vector search is
bank + fact_type scoped and served by the per-(bank, fact_type) partial indexes
created at bank-creation time (
bank_utils.create_bank_vector_indexes). Theglobal
idx_memory_units_embeddingis never chosen by the planner whenbank_idis in the
WHEREclause — which is exactly why migrationd5e6f7a8b9c0drops itfor these backends.
ensure_vector_extensionnonetheless recreated that global index on a fresh/emptyschema (the "No embedding index found, will create it" path), putting an unused,
write-amplifying index back on every schema. This skips creating it for per-bank
backends.
scann(which genuinely uses a global filtered index) is unaffected.Verified via
EXPLAINon a real DB (3000 rows, both indexes present): thesemantic query uses the per-bank
idx_mu_emb_*index, and dropping the globalindex leaves the plan identical.
Tests
--skip-extension-reconcilethreadsensure_extensionsthrough to
run_migrations_for_schemas.migrate +
ensure_vector_extensionends up with no globalidx_memory_units_embedding(fails without the fix).