fix(meta): gate snapshot backfill merge on pending barriers

[wenym1]

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

Snapshot-backfill jobs currently merge into the database graph as soon as every backfill executor starts consuming upstream. If the independent partial graph still has a large replay-barrier backlog, replacement actors can then spend a long time waiting for their state-table epochs to commit and stall database streaming.

This PR delays the merge until the creating job's pending barrier count is within a configurable limit. The count includes in-flight barriers, collected barriers awaiting completion, and the checkpoint currently completing. The pending count is carried with collected-barrier events so a checkpoint is forced only when the job is actually eligible to merge; the merge path rechecks the live count at the checkpoint opportunity.

The new snapshot_backfill_finish_max_lagged_barriers streaming option defaults to 100. CI overrides it to 5 to exercise the gated path more frequently.

Checklist

• I have written necessary rustdoc comments.
• I have added necessary unit tests and integration tests.
• I have added test labels as necessary.
• I have added fuzzing tests or opened an issue to track them.
• My PR contains breaking changes.
• My PR changes performance-critical code, so I will run (micro) benchmarks and present the results.
• I have checked the Release Timeline and Currently Supported Versions to determine which release branches I need to cherry-pick this PR into.

Documentation

• My PR needs documentation updates.

Release note

Snapshot-backfill jobs now wait for their independent barrier backlog to fall below a configurable limit before merging into the database streaming graph. This reduces long streaming stalls during snapshot-backfill completion. The limit can be configured with streaming.snapshot_backfill_finish_max_lagged_barriers and defaults to 100.

[wenym1]

• fix(meta): gate snapshot backfill merge on pending barriers #26288 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.