Question: Clarification on Ping-Pong Benchmark Methodology

[Venkat2811]

Hello,

First, thank you for this impressive high-performance IPC library. The reported latencies are excellent.

I've been studying your benches/bench.rs to understand the performance characteristics, and I have a question about the methodology used in the ping-pong benchmark.

I noticed two things:

1. The reported latency is nearly constant across a wide range of message sizes (e.g., ~1.25µs for both 64KB and 512KB).
2. Looking at the benchmark implementation, it appears the write operations use write_empty_slice, which increments a pointer rather than writing the message payload, and the read operations use s.discard(), which seems to skip bytes rather than reading them from memory.

This leads me to believe that the benchmark is measuring the round-trip notification latency (the time to signal and coordinate the transfer) rather than the data transfer latency (the time to actually move the payload).

Could you please confirm if this understanding is correct? Is this intentional ?

If so, it might be helpful to clarify this in the benchmark's documentation to distinguish it from benchmarks that measure full data transfer.

Thanks,
Venkat

[GuangmingLuo]

@Millione please take a look

[Millione]

Hello Venkat,

Sorry for the late reply.

Thank you for your kind words about the library and for taking the time to delve into the benchmark code. It's great to see such a detailed level of engagement.

Regarding your question about the ping-pong benchmark, I appreciate you bringing this up as it's a common point of inquiry. However, I believe there might be a slight misunderstanding about what the write_empty_slice and discard() operation does.

Your observation is very astute, but the conclusion is not entirely correct. It's important to clarify the behavior of both functions:

1. write_empty_slice: It is not a no-op. This function actively allocates the shared memory buffer of the requested size (sz) and commits to writing a message of that length. The memory is secured and made available for the reading process. The key point is that it does this without the need to copy data from a pre-existing source buffer, which is the optimization. The act of allocation and making the memory "valid" for reading is a form of commitment and is part of the transfer cost.

2. s.discard(sz): This is the crucial part that ensures data is actually transferred. As you suspected, it does not just skip bytes. The discard(sz) call must read the entire allocated payload (all sz bytes) from the shared memory before it can advance the read pointer. This is what ensures the full cost of moving the data across processes is accounted for.

Therefore, the benchmark is measuring the full round-trip data transfer latency. The combination of these operations means that for a message of size sz:

1. The writing process writes sz bytes in shared memory.

2. The reading process reads sz bytes from shared memory.

The nearly constant latency for large messages is a result of the library's efficient zero-copy design, but it is absolutely measuring the time to write and then read the entire payload.

I hope this clarifies the internal mechanics. You've raised an excellent point, and I will make a note to add a comment in the benchmark code to explain this behavior more explicitly for future readers. Thank you again for the valuable feedback!

Best regards,
Millione

[Venkat2811]

Hello @Millione ,

No problem, and thank you for your detailed explanation & positive discussion.

Based on your pointers, I did deeper analysis of the benchmark code and considering hardware fundamentals, I'd like to respectfully share my findings. The evidence suggests the benchmark, in its current form, measures IPC coordination overhead rather than data transfer costs, which explains the constant low latency across message sizes.

1. Code Analysis: Pointer Arithmetic vs. Memory Access

   The core benchmark functions operate on buffer indices without accessing the underlying memory:

   • Write Path: write_empty_slice only increments the write_index. It does not write any data to the buffer.

    // benches/bench.rs:260-268
    fn write_empty_slice(slice: &mut BufferSlice, size: u32) -> u32 {
        slice.write_index += size as usize;  // Only increments index
        // ... bounds check ...
    }

   • Read Path: s.discard() calls LinkedBuffer::discard() , which in turn calls BufferSlice::skip(). This function only increments the read_index.

    // src/buffer/slice.rs:246-254
    pub fn skip(&mut self, size: usize) -> usize {
        // ...
            self.read_index += size;  // Only increments index
       // ...
     }

2. Physical Hardware Constraints

   Assuming the numbers in readme are from same machine used in: https://github.com/cloudwego/shmipc-go , Intel Xeon E5-2630 v4 Specs: ark.intel.com

   The benchmark reports a latency of ~1.25µs for a 512KB transfer. This timing is inconsistent with the physical limits of memory access.

    Intel Xeon E5-2630 v4 has a DRAM latency of ~85ns per 64-byte cache line.
   
    Calculation: A 512KB transfer requires accessing 8,192 cache lines.
   
    Minimum Theoretical Latency: 8,192 cache lines × 85 ns/line ≈ 696µs.

   Even with ideal L3 cache performance (~18ns/line), the minimum latency would be ~147µs. The reported 1.25µs is physically impossible, if the 512KB of data were actually being read from memory.

3. What the Benchmark Measures

   The benchmark provides a valuable measurement of the control plane overhead, including:

   • IPC signaling (futex/eventfd).
   • Shared memory metadata updates (atomic pointer increments).
   • Context switching and synchronization costs.

   It does not, however, measure the data plane costs, such as memory bandwidth utilization or cache miss penalties.

Recommendation: Enhance the Benchmark

To provide a more complete performance picture, I suggest adding an optional mode to the benchmark that forces memory access. This would allow users to distinguish between pure IPC overhead and true data transfer latency.

A simple implementation could touch each cache line:

// Example: Force a read of each cache line in the slice
fn touch_memory(slice: &BufferSlice, size: usize) {
    const CACHE_LINE_SIZE: usize = 64;
    let base_ptr = slice.data.add(slice.read_index);

    for offset in (0..size).step_by(CACHE_LINE_SIZE) {
        unsafe {
            // Volatile read to ensure the compiler doesn't optimize it away
            let _ = std::ptr::read_volatile(base_ptr.add(offset));
        }
    }
}

Adding a --touch-memory flag would make this excellent library even more useful for performance-critical applications by clearly delineating coordination costs from data movement costs.

Thank you again for creating such an impressive zero-copy library. I hope this analysis is helpful.

Thanks,
Venkat

[Venkat2811]

@Millione

As a follow-up, I also wanted to discuss a subtle but critical aspect of latency benchmarking: Coordinated Omission.

The current benchmark measures throughput by timing a tight loop of operations. This is a valid way to measure maximum throughput, but it can be misleading when interpreted as single-operation latency due to two factors:

• Amortization: Fixed costs are spread across many operations, hiding the true cost of any single one.
• Coordinated Omission: When the system under test cannot keep up, requests start to queue (CPU queue). The benchmark measures the time from when an operation starts to when it ends, ignoring the time it spent waiting in the queue. This omits the largest parts of the latency profile under load and artificially lowers the reported results.

For a true latency measurement under sustained load, each operation should be scheduled at a fixed rate, and the latency measured from its intended start time. Eg: generating constant load of 100k - 1M req/sec, incrementally to understand max load system can sustain for desired latency profile.

I'm also researching and building things in this space and made similar mistakes :)

Reference:

Gil Tene's talk on Coordinated Omission: "How NOT to Measure Latency"

I'm very impressed with this elegant lib and, am learning a lot by studying it. I know benchmarks can be a sensitive subject, so please know this analysis is offered respectfully and in the spirit of collaboration.

Thanks,
Venkat