[Bug]: Mixed bool and non-bool values in JSON IN/OR filters cause QueryNode crash or silent data loss

[Nqn7m11]

Is there an existing issue for this?

• I have searched the existing issues

Environment

Milvus version: v2.6.20
- Deployment mode(standalone or cluster): standalone
- MQ type(rocksmq, pulsar or kafka): rocksmq
- SDK version(e.g. pymilvus v2.0.0rc2): pymilvus latest
- OS(Ubuntu or CentOS): macOS Darwin 25.3.0 (Apple Silicon) / Docker
- CPU/Memory: Apple M-series / 16GB
- GPU: N/A
- Others: Docker Compose deployment, FLAT index

Current Behavior

When a JSON field filter expression mixes bool values with any other type (int, float, or string), both $in and $or paths produce crashes or silent data loss depending on the specific values and their order.

This manifests through two paths:

1. Direct $in path: meta["b"] in [true, 1] — mixed-type list reaches segcore directly
2. $or optimizer merge path: (meta["b"] == true) or (meta["b"] == 1) or (meta["b"] == 0) — optimizer merges 3+ branches into a mixed-type internal in list

Both paths trigger the same root cause: the segcore C++ executor determines the expected value type from the first element in the in list and either asserts (crash) or silently ignores mismatched values (data loss).

Unique bool characteristics:

• true (internal value 1) crashes; false (internal value 0) causes silent data loss — an asymmetry caused by the zero-value code path
• $or merge threshold is 3 branches for bool (same as str+int), lower than float+int's 4 branches
• Value ordering changes results: same set of values in different order produces different result sets

---

Path 1: Bool mixed-type $in

# Data: d0,d1: b=True(bool)  d2,d3: b=False(bool)  d4: b=0(int)  d5: b=1(int)

# CRASH -- true + int
col.query(expr='meta["b"] in [true, 1]')
# -> segcore assertion crash at Utils.h:228

# SILENT DATA LOSS -- false + int
col.query(expr='meta["b"] in [false, 1]')
# Expected: d2,d3,d5 (false=d2,d3 + int1=d5)
# Actual:   d2,d3 -- int match d5 silently dropped!

# ORDER MATTERS
col.query(expr='meta["b"] in [true, false, 0, 1]')
# -> d0,d1,d2,d3 (only bool matches, ints dropped)

col.query(expr='meta["b"] in [0, 1, true, false]')
# -> d4,d5 (only int matches, bools dropped!)
# Same 4 values, different order, completely different results!

Complete $in crash/data-loss matrix:

First value	Second value	Result	Assertion
true	int (1/0)	Crash	Utils.h:228
true	string	Crash	Utils.h:243
true	float	Crash	Utils.h:277
false	int (1/0)	Data loss (int matches dropped)	--
false	string	Data loss	--
false	float	Data loss	--
int(1)	true/false	Crash	Utils.h:224
int(0)	true/false	Data loss (bool matches dropped)	--
string	true/false	Crash	Utils.h:224
float(1.0)	true	Crash	Utils.h:224
float(0.0)	false	Data loss	--

Pattern: true (internal value 1) and non-zero values -> crash; false (internal value 0) and zero values -> silent data loss. The first element's type determines the matching type for the entire list.

---

Path 2: Bool $or optimizer merge (3+ branches)

# Bool first: SILENT DATA LOSS
col.query(expr='(meta["b"] == true) or (meta["b"] == 1) or (meta["b"] == 0)')
# Expected: d0,d1 + d5 + d4 = 4 docs
# Actual:   d4,d5 only -- bool matches d0,d1 silently dropped!

# Int first: CORRECT
col.query(expr='(meta["b"] == 1) or (meta["b"] == true) or (meta["b"] == 0)')
# Returns: d0,d1,d4,d5 = 4 docs (correct)

# Bool+str: CRASH
col.query(expr='(meta["b"] == true) or (meta["b"] == "yes") or (meta["b"] == "no")')
# -> segcore assertion crash at Utils.h:224

2 branches of any combination work fine (optimizer does not merge <= 2 branches).

Summary of $or behaviors with 3+ branches:

$or branches	Result	Detail
true | 1 | 0 (bool first)	Data loss	Bool matches dropped
1 | true | 0 (int first)	Correct	4 results
0 | true | 1 (int first)	Correct	4 results
true | false | 1 | 0 (4br)	Data loss	Int matches dropped
true | 1 | 0 | false (4br)	Data loss	Bool matches dropped
true | "yes" | "no"	CRASH	Utils.h:224
true | "yes" | "no" | false	CRASH	Utils.h:243
true | 1.0 | 0.0	Data loss	2 results
true | 1.0 | 0.0 | false	Data loss	2 results

---

Expected Behavior

Mixed-type $in and $or on a JSON field should either:

1. Return correct results matching all specified values regardless of type or order, or
2. Return a clear error (code=1100) at the expression parsing/planning stage if mixed types are not supported

It should never crash the QueryNode, and should never silently return incomplete results.

Steps To Reproduce

from pymilvus import (
    Collection, CollectionSchema, DataType, FieldSchema,
    connections, utility,
)
import time

connections.connect("default", host="localhost", port="19530")

if utility.has_collection("repro_bool_mixed"):
    Collection("repro_bool_mixed").drop()

fields = [
    FieldSchema("id", DataType.VARCHAR, is_primary=True, max_length=64),
    FieldSchema("vec", DataType.FLOAT_VECTOR, dim=8),
    FieldSchema("meta", DataType.JSON),
]
col = Collection("repro_bool_mixed", CollectionSchema(fields))
col.create_index("vec", {"index_type": "FLAT", "metric_type": "L2", "params": {}})

col.insert([
    ["d0", "d1", "d2", "d3", "d4", "d5"],
    [[(i * 7 + j) % 100 / 100.0 for j in range(8)] for i in range(6)],
    [
        {"b": True},   # d0
        {"b": True},   # d1
        {"b": False},  # d2
        {"b": False},  # d3
        {"b": 0},      # d4 (int)
        {"b": 1},      # d5 (int)
    ],
])
col.flush()
col.load()
time.sleep(1)

# =============================================
# Control: same-type $in works correctly
# =============================================

r = col.query(expr='meta["b"] in [true, false]')
print("Bool-only $in:", sorted(x["id"] for x in r))
# -> ['d0', 'd1', 'd2', 'd3'] (correct)

r = col.query(expr='meta["b"] in [0, 1]')
print("Int-only $in:", sorted(x["id"] for x in r))
# -> ['d4', 'd5'] (correct)

# =============================================
# Path 1: $in CRASH -- true + int
# =============================================

try:
    col.query(expr='meta["b"] in [true, 1]')
except Exception as e:
    print(f"$in [true, 1]: CRASH! {str(e)[:100]}")

try:
    col.query(expr='meta["b"] in [true, 0]')
except Exception as e:
    print(f"$in [true, 0]: CRASH! {str(e)[:100]}")

# =============================================
# Path 1: $in SILENT DATA LOSS -- false + int
# =============================================

r = col.query(expr='meta["b"] in [false, 1]')
print("$in [false, 1]:", sorted(x["id"] for x in r))
# Expected: ['d2', 'd3', 'd5']
# Actual:   ['d2', 'd3'] -- int match d5 silently dropped!

r = col.query(expr='meta["b"] in [false, 0]')
print("$in [false, 0]:", sorted(x["id"] for x in r))
# Expected: ['d2', 'd3', 'd4']
# Actual:   ['d2', 'd3'] -- int match d4 silently dropped!

# =============================================
# Path 1: $in ORDER DEPENDENCE
# =============================================

r = col.query(expr='meta["b"] in [true, false, 0, 1]')
print("$in [T,F,0,1]:", sorted(x["id"] for x in r))
# -> ['d0', 'd1', 'd2', 'd3'] -- int docs d4,d5 silently dropped!

r = col.query(expr='meta["b"] in [0, 1, true, false]')
print("$in [0,1,T,F]:", sorted(x["id"] for x in r))
# -> ['d4', 'd5'] -- bool docs d0-d3 silently dropped!
# Same values, different order, completely different results!

# =============================================
# Path 1: $in CRASH -- true + string, true + float
# =============================================

try:
    col.query(expr='meta["b"] in [true, "yes"]')
except Exception as e:
    print(f'$in [true, "yes"]: CRASH! {str(e)[:100]}')

try:
    col.query(expr='meta["b"] in [true, 1.0]')
except Exception as e:
    print(f"$in [true, 1.0]: CRASH! {str(e)[:100]}")

# =============================================
# Path 2: $or 3br bool-first -- SILENT DATA LOSS
# =============================================

r = col.query(expr='(meta["b"] == true) or (meta["b"] == 1) or (meta["b"] == 0)')
print("$or 3br true|1|0:", sorted(x["id"] for x in r))
# Expected: ['d0', 'd1', 'd4', 'd5']
# Actual:   ['d4', 'd5'] -- bool matches d0,d1 silently dropped!

# =============================================
# Path 2: $or 3br int-first -- CORRECT
# =============================================

r = col.query(expr='(meta["b"] == 1) or (meta["b"] == true) or (meta["b"] == 0)')
print("$or 3br 1|true|0:", sorted(x["id"] for x in r))
# -> ['d0', 'd1', 'd4', 'd5'] (correct)

# =============================================
# Path 2: $or bool+str -- CRASH
# =============================================

try:
    col.query(expr='(meta["b"] == true) or (meta["b"] == "yes") or (meta["b"] == "no")')
except Exception as e:
    print(f"$or bool+str 3br: CRASH! {str(e)[:100]}")

# =============================================
# Control: 2 branches -- below merge threshold, works fine
# =============================================

r = col.query(expr='(meta["b"] == true) or (meta["b"] == 1)')
print("$or 2br true|1:", sorted(x["id"] for x in r))
# -> ['d0', 'd1', 'd5'] (correct -- no optimizer merge)

# =============================================
# Empty collection immunity
# =============================================

if utility.has_collection("repro_bool_empty"):
    Collection("repro_bool_empty").drop()

col2 = Collection("repro_bool_empty", CollectionSchema(fields))
col2.create_index("vec", {"index_type": "FLAT", "metric_type": "L2", "params": {}})
col2.load()
time.sleep(1)

r = col2.query(expr='meta["b"] in [true, 1]')
print("Empty collection $in [true,1]:", len(r), "results (no crash)")
# -> 0 results, no crash

col2.insert([["d0"], [[0.1]*8], [{"b": True}]])
col2.flush()
col2.load()
time.sleep(1)

try:
    col2.query(expr='meta["b"] in [true, 1]')
except Exception as e:
    print(f"1-doc collection: CRASH! {str(e)[:80]}")
    # -> CRASH -- inserting just 1 doc triggers it

col.drop()
col2.drop()


**All test cases verified 5/5 stable (100% reproducible).**

Milvus Log

Assert "(value_proto.val_case() == milvus::proto::plan::GenericValue::kInt64Val)"
at ../../../internal/core/src/exec/expression/Utils.h:228

Assert "(value_proto.val_case() == milvus::proto::plan::GenericValue::kBoolVal)"
at ../../../internal/core/src/exec/expression/Utils.h:224

Assert "(value_proto.val_case() == milvus::proto::plan::GenericValue::kStringVal)"
at ../../../internal/core/src/exec/expression/Utils.h:243

Assert "(value_proto.val_case() == milvus::proto::plan::GenericValue::kFloatVal)"
at ../../../internal/core/src/exec/expression/Utils.h:277

Anything else?

Root cause analysis:

The segcore C++ executor (PhyFilterBitsNode) processes in lists by determining the expected value type from the first element, then iterates over remaining elements assuming the same type:

1. true first (internal value = 1, non-zero): Asserts remaining elements have kBoolVal type -> non-bool elements fail the assertion -> crash
2. false first (internal value = 0, zero): The zero-value code path does not trigger the assertion -> non-bool elements are silently skipped -> data loss
3. int(1) first (non-zero): Asserts remaining have kInt64Val -> bool elements fail -> crash
4. int(0) first (zero): Zero-value path -> bool elements silently skipped -> data loss

This true/false asymmetry is unique to bool values — it doesn't occur with other type combinations where both crash and data-loss cases don't correlate with the value itself being zero.

For the $or path, the query optimizer merges 3+ equality branches on the same field into an internal in list, inheriting the same type-inconsistency issue. The merge threshold for bool is 3 branches (the optimizer does not merge <= 2 branches).

Empty collection immunity: Confirmed — crashes only occur when data exists (segcore data scanning layer), not in the Go expression planning layer.

Impact:

• Crash: Any application that stores mixed bool/int/string values in a JSON field and queries with $in or builds $or expressions dynamically can be crashed (DoS vector)
• Silent data loss: The false/zero-value path is arguably more dangerous — queries complete successfully but with incomplete results. An application querying $in [false, 0, 1] would silently miss all int(0) and int(1) documents
• Order dependence: $in [true, false, 0, 1] and $in [0, 1, true, false] return completely different result sets — this violates the mathematical property that set membership is order-independent

Workaround:

Never mix bool and int/float/string values in $in or $or on JSON fields. Query each type separately:

# Instead of (BROKEN):
'meta["b"] in [true, 1, 0]'

# Use separate queries and union in application code:
bool_results = col.query(expr='meta["b"] in [true, false]')
int_results = col.query(expr='meta["b"] in [0, 1]')
all_results = bool_results + int_results

Suggested fix:

1. Expression planner (Go): Add type-consistency validation for $in value lists and $or optimizer merge on JSON fields. If types differ and are not compatible, either reject with code=1100 or skip the $or merge optimization.

2. Segcore fallback (C++): Replace assert at Utils.h:224,228,243,277 with graceful error return, so type mismatches produce a query error instead of crashing.

Related: This is the bool-specific variant of the broader mixed-type $in/$or issue on JSON fields. The true/false asymmetry and the 3-branch $or threshold are unique to bool values.

Versions verified:

Version	Crash	Silent data loss
v2.6.20 (latest)	Reproduced	Reproduced