Multi-stream wfb_tx: per-stream FEC profiles, priority drain, waybeam SHM input (one process for all streams)

[josephnef]

Summary

Multi-stream wfb_tx: one TX process serving several streams with per-stream FEC profiles, per-stream radiotap, strict drain priority, per-stream runtime control, and direct waybeam_venc SHM ring input.

This is the follow-up promised in OpenIPC/waybeam_venc#85 ("single wfb-ng TX process consuming all waybeam streams with per-stream FEC profiles"). waybeam_venc 0.17.0 splits SVC-T layered video onto two transport channels so wfb-ng can protect each temporal layer differently — but with stock wfb-ng that costs one wfb_tx process per stream, and each FEC encoder schedules blind to the others.

Note on upstream: this PR intentionally lives in this fork. Upstream wfb-ng contribution policy asks for Telegram-first discussion of features and bans AI-generated code — taking this work upstream (rewritten/owned by a human contributor, with bench numbers) is a separate, later step via t.me/wfb_ng. Until then the fork builds drop-in binaries: without -y, behavior is identical to upstream master (verified by the unchanged upstream test suite).

What it does

wfb_tx -K drone.key -i 7669206 -C 7003 \
       -y "u=5600,p=0,k=8,n=12,T=15" \      # base video    — priority 0, strong FEC
       -y "u=5605,p=1,k=8,n=9,T=15"  \      # enhance video — sheds load first
       wlan0

• -y stream spec (repeatable): u=<udp_port> or shm=<venc_ring name> (input), p=<radio_port> (unique), k/n/T/F (FEC), mcs/bw/gi/stbc/ldpc/vht/nss (per-stream radiotap). Unset keys inherit the global options.
• One Transmitter instance per stream — own channel_id on the shared link_id, own FEC encoder/block state, own session key + announce timer, own fec_timeout, own radiotap header. Exactly what N processes do today, but in one event loop. RX side is unchanged: one wfb_rx -p <radio_port> per stream, as always.
• Strict drain priority = spec order. On every wakeup, ready inputs are serviced lowest-index-first and drained fully; under radio saturation the kernel socket buffers of later streams overflow first. That's the unequal-protection behavior layered video wants: base keeps flowing, droppable enhancement sheds first. (Caveat documented: a continuously saturating high-priority stream can starve lower ones — order streams by importance.)
• shm= input: attaches to a waybeam_venc shared-memory packet ring (/dev/shm/<name>; src/venc_ring.{h,c} vendored byte-identical from waybeam_venc, MIT, attribution headers). One detached reader thread per ring (futex wait → socketpair → main poll loop, so all Transmitter state stays single-threaded); producer respawn handled via epoch-based re-attach. Pairs with waybeam's outgoing.server=shm://venc_wfb + outgoing.enhancePort (which creates venc_wfb_enh).
• Per-stream runtime control: wfb_tx_cmd <port> {set,get}_{fec,radio}_stream -r <radio_port> … — the hook for closed-loop per-layer FEC adaptation. Legacy commands keep addressing the first stream; unknown radio_port → ENODEV.
• Stats: per-stream PKT_S/TX_ANT_S stdout lines tagged with radio_port + the legacy aggregate PKT/TX_ANT lines. Existing parsers (wfb_ng/protocols.py) ignore unknown tags — verified by running the new tests under the stock TXProtocol parser.

Commits (each independently buildable)

1. tx: extract control-socket handling, add stream-taggable stats — zero-behavior refactor
2. tx: multi-stream local mode with per-stream FEC and radiotap (-y) — core
3. tx: waybeam venc_ring SHM input for multi-stream mode (shm= key)
4. tx: per-stream control commands for multi-stream mode
5. tests + docs: multi-stream coverage, changelog and README section

Verification

• Build: make all_bin clean (the only two warnings are pre-existing on master — identical count); fec_test + libsodium_test pass.
• Python suite: twisted.trial wfb_ng.tests → 37 passed, 1 skipped (pre-existing tuntap skip), 0 failed — that's all 32 upstream tests (regression: legacy single-stream mode byte-compatible) plus 6 new test_multistream.py cases:
  • per-stream delivery with exact packet accounting per FEC profile (2/4 vs 1/2)
  • per-stream FEC recovery + loss isolation (drops on one stream never affect the other)
  • cross-stream session isolation (stream A packets fed to stream B's aggregator decode to nothing)
  • per-stream cmd set/get fec + radio; legacy cmd → first stream; ENODEV for unknown radio_port
• Manual loopback (no radio, -D debug mode + socat relays): two UDP streams with different FEC, independent per-stream FEC-timeout behavior observed in PKT_S lines; set_fec_stream -r 17 -k 4 -n 8 live retune while stream 16 stayed at 2/4.
• SHM smoke: ad-hoc venc_ring producer → -y "shm=venc_test,p=16,k=2,n=4" → ~20 pkt/s delivered; kill -9 + restart of the producer → shm ring venc_test recreated (epoch X -> Y), re-attached and flow resumed.
• Pending hardware bench (waybeam devices offline): real-radio run with waybeam shm://venc_wfb + enhancePort, tcpdump radiotap verification of both channel_ids and per-stream MCS, and the full SVC-T per-layer FEC end-to-end (tracked as the POC for any future upstream discussion).

Future work

• Aggregated airtime scheduler beyond strict priority (weighted/token-based).
• wfb-server/services.py opt-in to spawn one multi-stream TX per profile group (left out deliberately — this PR keeps the Python layer untouched).
• RX-side consolidation (one wfb_rx for several channel_ids).

🤖 Generated with Claude Code