Caliper: Annotate kernels separately

[michaelmckinsey1]

• Depends on Jacobi 2D Seq, OMP, and OMPTarget launch twice #700

Summary

• This PR is additional Caliper regions for RAJAPerf kernels that have multiple kernel launches.
• It does the following (modify list as needed):
  • Adds additional Caliper regions at the request of @pearce8
• There is currently no synchronize for these regions for asynchronous GPU kernels. But these should be profiled with CUDA/HIP events anyway, which do not need CPU synchronization to measure GPU time.
• RAJA::launch or RAJA::forall kernels are annotated using the RAJA Caliper plugin
• add function types to be able to filter out kernels instead of type=function -> type=subkernel [1]
• Update documentation for naming convention

Examples

Polybench_JACOBI_1D has one launch per rep for poly_jacobi_1D_1 and one for poly_jacobi_1D_2. So its tree will now profile each separately:

POLYBENCH_FLOYD_WARSHALL, HALO_PACKING, HALO_EXCHANGE all have variable amount of launches, so we will append _k instead of adding k regions to the tree

All kernels with only 1 launch per rep are unchanged.

[1] how to filter subkernels at runtime

raja-perf.exe --atcc "exclude_attributes=subkernel"

Then subkernels will be excluded:

Caveat for this method is that if profiling API functions, like HIP or MPI the subkernel regions will no longer be excluded if they have API functions.

[michaelmckinsey1]

made a draft because need to add different types to these regions so we can filter them with caliper

[rhornung67]

Thank you @michaelmckinsey1

[michaelmckinsey1]

@MrBurmark @artv3 @rhornung67
I added one new commit since this was approved adebe6c. I noticed while testing for many different variants there were missing annotations. For example, the RAJA variant would be annotated but not Base and Lambda. This commit did not annotate any new kernels. This happened because I was using an LLM to apply the annotations and apparently, despite giving manual examples and describing that there are different variants, it has a really difficult time identifying that the annotations should be applied uniformly across the variants (this surprised me since I thought it would be trivial).

Also it is now rebased on #700