Skip to content

bench: pipeline-fair cluster-mode throughput methodology (memtier --cluster-mode collapses the pipeline) #568

Description

@ELares

The c7g.4xlarge re-bench (2026-07-07) PROVED #517 zero-hop (shard-owners hops_sent = 0 vs 93.7% hop single-endpoint), but could NOT cleanly measure the THROUGHPUT benefit of hop-elimination, because the benchmark client confounds it.

The confound: memtier_benchmark --cluster-mode with --pipeline=16 cannot actually pipeline: consecutive keys in a pipeline batch route to DIFFERENT shard-owner ports (that is the whole point of cluster routing), so the per-connection pipeline depth collapses toward 1. Evidence: shard-owners avg latency was 9.79ms vs 0.76ms single-endpoint (a 13x jump consistent with pipeline depth dropping from 16 to ~1), and throughput 1.18M vs 2.09M ops/sec. The single-endpoint leg keeps the full pipeline=16 (all ops to one port, then hop internally), so the 2.09M-vs-1.18M gap is mostly a client pipeline-depth artifact, NOT a fair hop-vs-no-hop comparison.

What a fair measurement needs: pipeline-matched methodology. Options:

  • --pipeline=1 for BOTH single-endpoint and cluster-mode (removes the pipeline variable; measures per-request hop cost directly).
  • A per-node-pipelining cluster benchmark (pipeline within each node's key subset).
  • redis-benchmark per-shard-port runs summed, vs single-endpoint.

Until then, the honest claim is: #517 eliminates the hop (proven by the metric), and the throughput benefit is un-measured pending a pipeline-fair harness. The zero-hop property itself is not in question.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions