Skip to content
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
bb0500d
contrib/net-profiler
xzyaoi May 2, 2026
71fe7a4
add mesh-bench design proposal
xzyaoi May 2, 2026
2486df3
add --config-dir flag for per-instance opentela state
xzyaoi May 2, 2026
a611fc4
fix initConfig seed-write path under --config-dir
xzyaoi May 2, 2026
146aee6
otela init pre-generates libp2p identity key
xzyaoi May 2, 2026
eab617f
fix Task 2 review: avoid host.go round-trip and reset viper in test
xzyaoi May 2, 2026
a9ad35b
add otela peer-id command
xzyaoi May 2, 2026
a43daca
register libp2p /ipfs/ping/1.0.0 protocol
xzyaoi May 2, 2026
ef584c9
add /v1/probe/echo responder endpoint
xzyaoi May 2, 2026
dfd43e3
fix Task 5 review: flush per chunk and verify echo body content
xzyaoi May 2, 2026
4d989f5
add /v1/probe/run originator endpoint
xzyaoi May 2, 2026
637ae65
fix Task 6 review: bandwidth-weighted throughput and stateless ping
xzyaoi May 3, 2026
8089d52
add otela probe CLI command
xzyaoi May 3, 2026
0c4ed38
add bench config rendering for mesh-bench
xzyaoi May 3, 2026
fa9b6a7
add bench phases: init and discover
xzyaoi May 3, 2026
0547701
add bench phases: configure and push
xzyaoi May 3, 2026
a9df2c9
add bench phases: start and converge
xzyaoi May 3, 2026
f9256bf
add bench phase: sweep
xzyaoi May 3, 2026
385784f
add bench phase: teardown
xzyaoi May 3, 2026
f8a5282
wire net-profiler bench subcommand
xzyaoi May 3, 2026
2d45fa7
add bench end-to-end smoke test and per-machine port overrides
xzyaoi May 3, 2026
45d374d
update net-profiler
xzyaoi May 3, 2026
4635652
network-profiler
xzyaoi May 10, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,4 @@ website/.source/
.omx/
.local
.pista
.claude/
4 changes: 4 additions & 0 deletions contrib/network-profiler/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
__pycache__/
*.py[cod]
.pytest_cache/
results/
8 changes: 8 additions & 0 deletions contrib/network-profiler/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
.PHONY: test bench-smoke

test:
python3 -m pytest tests/ -v

bench-smoke:
cd ../../src && $(MAKE) build
python3 -m pytest tests/test_bench_smoke.py -v -s
72 changes: 72 additions & 0 deletions contrib/network-profiler/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Network Profiler

Pairwise network profiler for machines managed through
[`ResearchComputer/remote-cluster-controller`](https://github.com/ResearchComputer/remote-cluster-controller).

It runs commands on every source machine, targets every other machine, records latency with `ping`, optionally records bandwidth with `iperf3`, and renders an HTML heatmap.

## Configure machines

Copy `examples/machines.example.json` and update:

- `machines[].name`: label used in output.
- `machines[].address`: address reachable from the other remote machines.
- `machines[].rcc_host`: host identifier understood by `remote-cluster-controller`.
- `remote_command`: argv template for running a shell command on a remote host.

The default remote command template is:

```json
[
"remote-cluster-controller",
"exec",
"--host",
"{host}",
"--",
"bash",
"-lc",
"{command}"
]
```

If your local checkout exposes a different CLI, change only `remote_command`.
The template supports `{host}`, `{name}`, `{address}`, and `{command}`.

## Run

Show what will be executed:

```bash
python3 -m network_profiler plan --config machines.json
```

Collect latency only:

```bash
python3 -m network_profiler collect --config machines.json --output results/network.jsonl
```

Collect latency and bandwidth:

```bash
python3 -m network_profiler collect --config machines.json --iperf --output results/network.jsonl
```

Render heatmaps:

```bash
python3 -m network_profiler heatmap --input results/network.jsonl --output results/ping.html
python3 -m network_profiler heatmap --input results/network.jsonl --kind iperf3 --output results/iperf.html
```

## Remote requirements

Each remote machine needs:

- `bash`
- `ping`
- `iperf3` only when `--iperf` is used
- network paths that allow source machines to reach target `address` values
- TCP access to `iperf_port` between remote machines when collecting bandwidth

The JSONL output preserves every command, success flag, parsed metrics, and failure text so failed links can be inspected without rerunning the whole sweep.
5 changes: 5 additions & 0 deletions contrib/network-profiler/network_profiler/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
"""Network profiling utilities for remote clusters."""

__all__ = ["__version__"]

__version__ = "0.1.0"
4 changes: 4 additions & 0 deletions contrib/network-profiler/network_profiler/__main__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
from .cli import main

if __name__ == "__main__":
raise SystemExit(main())
Loading
Loading