sanitiseRow drops null fields but misaligns remaining values with schema

[piotrszul]

SingleInstanceEvaluator.sanitiseRow() strips null-valued fields from a Row when building the sanitised output. However, it creates a new GenericRowWithSchema where the remaining values are positionally misaligned with the filtered schema.

Example:

Given a Period struct with schema [id, start, end] and values [null, "2000", "2002"]:

1. sanitiseRow filters out id (null) → filtered schema is [start, end]
2. The remaining values are ["2000", "2002"] → assigned to [start, end] ✓

But when the id field is non-null and an intermediate field is null, the positional assignment shifts:

Given schema [id, extension, start, end] with values [null, null, "2000", "2002"]:

1. Filter out id (null) and extension (null) → filtered schema is [start, end]
2. Remaining values ["2000", "2002"] → correctly [start, end] ✓

The actual bug manifests when synthetic fields are stripped alongside null fields. Given a struct like:

schema: [_fid, id, start, end]
values: [123,  null, "2000", "2002"]

1. _fid is stripped as synthetic, id is stripped as null
2. Filtered schema: [start, end], filtered values: ["2000", "2002"] ✓

But in practice, when Row.json() is called on the sanitised row, the output shows field values shifted — e.g., {"id":"2000","start":"2002"} instead of {"start":"2000","end":"2002"}.

Observed in: SingleInstanceEvaluator.evaluate() results and trace output, when complex FHIR types contain null fields that get stripped.

Reproduction:

Evaluate name.period against a Patient with name[0].period = {start: "2000", end: "2002"} via PathlingContext.evaluateFhirPath(). The returned JSON for the Period will have misaligned field names.

Impact: Affects all complex type values returned by evaluateFhirPath, including both main results and trace output.