Currently, our search APIs assume all queries in a batch should share the same search parameters.
This makes it challenging for databases that might already have an optimizer or heuristics for selecting parameters based on other properties of the vector space or metadata distributions and want to specify different search parameters for each query. In short, this forces these users to assume they will always be doing an online query (batch size of 1) even if they have several 10s or hundreds of queries that can be batched together.
Users have requested the ability to specify separate search params for each query vector, and for cuVS to handle this case efficiently no matter what params are being varied.
Currently, our search APIs assume all queries in a batch should share the same search parameters.
This makes it challenging for databases that might already have an optimizer or heuristics for selecting parameters based on other properties of the vector space or metadata distributions and want to specify different search parameters for each query. In short, this forces these users to assume they will always be doing an online query (batch size of 1) even if they have several 10s or hundreds of queries that can be batched together.
Users have requested the ability to specify separate search params for each query vector, and for cuVS to handle this case efficiently no matter what params are being varied.