Skip to content

Commit 30267e8

Browse files
[mini] Improve documentation for nsys usage (#684)
When running with nsys, stalling was observed in running more sophisticated runs. They come from the cuda events. Disabling the tracing of the cuda events fixes the problem, hence the documentation should reflect that.
1 parent a392417 commit 30267e8

1 file changed

Lines changed: 5 additions & 2 deletions

File tree

docs/user/profiling.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -47,16 +47,19 @@ The iteration range is half-open: `[start, stop)`. For example,
4747
iterations 50 through 149. Invalid or negative values are ignored and treated as
4848
the default `0`.
4949

50-
Run `nsys` with the CUDA profiler API capture range:
50+
For profiling longer, multithreaded runs, such as hundreds of ttbar events, the CUDA-event tracing needs to be disabled, as it was observed create stalls in `cudaEventRecord`. Furthermore, writing periodically the buffered CUDA trace data can help with memory issues. This can be done with the following options:
5151

5252
```console
5353
nsys profile --capture-range=cudaProfilerApi --capture-range-end=stop \
54-
--trace=cuda,nvtx --sample=none --cpuctxsw=none \
54+
--trace=cuda,nvtx --cuda-event-trace=false --cuda-flush-interval=10000 \
55+
--sample=none --cpuctxsw=none \
5556
--stats=true --export=sqlite --force-overwrite=true \
5657
--output adept_transport_profile \
5758
<application command>
5859
```
5960

61+
Note that older Nsight Systems versions do not support the options `cuda-event-trace` and `cuda-flush-interval`. In that case, it is advised to use a more recent Nsight Systems version (tested with 2025.5.2).
62+
6063
Open the report with:
6164

6265
```console

0 commit comments

Comments
 (0)