Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,10 @@
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = [os.path.join(conf_directory, 'sphinx/_static')]

html_css_files = [
'raja.css',
]

# Add any extra paths that contain custom files (such as robots.txt or
# .htaccess) here, relative to this directory. These files are copied
# directly to the root of the documentation.
Expand Down
8 changes: 8 additions & 0 deletions docs/sphinx/_static/raja.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
/* Use the available browser width for wide reference tables. */
.wy-nav-content {
max-width: none;
}

.rst-content table.docutils {
width: 100%;
}
130 changes: 130 additions & 0 deletions docs/sphinx/user_guide/feature/policies.rst
Original file line number Diff line number Diff line change
Expand Up @@ -730,6 +730,136 @@ GPU Policies for SYCL
configuration. SYCL dimension 2 always exists and should be used as
one would use the x dimension in CUDA and HIP.

Device policy aliases
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

To simplify transitions between GPU backends (CUDA/HIP/SYCL) and reduce
downstream preprocessor conditionals, RAJA provides a small set of
``device_*`` policy aliases that resolve to the *active* GPU backend.

In particular, the following aliases are available when building with a GPU
device backend (i.e., when ``ENABLE_CUDA``, ``ENABLE_HIP``, or
``RAJA_ENABLE_SYCL`` is enabled):

.. list-table:: ``device_*`` alias coverage
:widths: 34 14 14 14 24
:header-rows: 1

* - Alias family
- CUDA
- HIP
- SYCL
- Notes
* - ``device_exec*``
- yes
- yes
- partial
- ``device_exec`` and ``device_exec_async`` exist for SYCL; the CUDA/HIP
occupancy and reduction variants do not.
* - ``device_atomic`` and ``device_atomic_explicit<host_policy>``
- yes
- yes
- yes
- Backend atomic policy aliases are available on all three backends.
* - ``device_reduce``
- yes
- yes
- yes
- Backend default reduce policy alias.
* - ``device_reduce_atomic`` and ``device_reduce_base<with_atomic>``
- yes
- yes
- no
- CUDA/HIP expose tuning and base reduce aliases; SYCL currently only
exposes ``device_reduce``.
* - ``device_multi_reduce_atomic`` and
``device_multi_reduce_atomic_low_performance_low_overhead``
- yes
- yes
- no
- CUDA/HIP expose ``multi_reduce`` aliases; SYCL does not currently
provide a backend-equivalent ``device_*`` mapping.
* - ``device_launch_t``
- yes
- yes
- yes
- Backend launch policy alias.
* - ``device_global_size_{x,y,z}_{direct,direct_unchecked,loop}<N>``
- yes
- yes
- partial
- SYCL currently exposes only ``device_global_size_x_direct``.
* - ``device_global_thread_{x,y,z}``
- yes
- yes
- yes
- Launch-space global thread mapping.
* - ``device_thread_{x,y,z}_{direct,loop}``
- yes
- yes
- yes
- Single-dimension thread mapping.
* - ``device_thread_{xy,xz,yx,yz,zx,zy,xyz,xzy,yxz,yzx,zxy,zyx}_{direct,loop}``
- yes
- yes
- no
- Multi-dimension thread permutations are not defined for SYCL.
* - ``device_thread_syncable_loop`` and named permutations
- yes
- yes
- no
- Synchronizable thread permutations are CUDA/HIP only.
* - ``device_thread_size_*`` families
- yes
- yes
- no
- Includes direct, direct_unchecked, and loop variants for 1D and
permuted multi-dimension forms.
* - ``device_block_{x,y,z}_{direct,loop}``
- yes
- yes
- yes
- Single-dimension block mapping.
* - ``device_block_{xy,xz,yx,yz,zx,zy,xyz,xzy,yxz,yzx,zxy,zyx}_{direct,loop}``
- yes
- yes
- no
- Multi-dimension block permutations are not defined for SYCL.
* - ``device_block_syncable_loop`` and named permutations
- yes
- yes
- no
- Synchronizable block permutations are CUDA/HIP only.
* - ``device_block_size_*`` families
- yes
- yes
- no
- Includes direct, direct_unchecked, and loop variants for 1D and
permuted multi-dimension forms.
* - ``device_flatten_block_threads_{xy,...,zyx}_{direct,loop}``
- yes
- yes
- yes
- Flattened block/thread permutations are available across all backends.
* - ``device_flatten_thread_size_*``, ``device_flatten_block_size_*``,
``device_flatten_global_size_*``
- yes
- yes
- no
- These flattened size aliases exist for CUDA/HIP but are not yet
provided through ``device_*`` for SYCL.

.. important::
For SYCL, these aliases use CUDA-like ``(x,y,z)`` naming with the standard
RAJA mapping described above: ``x`` corresponds to SYCL dimension 2,
``y`` to dimension 1, and ``z`` to dimension 0. These build options enable
the corresponding internal ``RAJA_*_ACTIVE`` compile-time macros used by
the implementation.
Device aliases that have no SYCL equivalent are intentionally not defined
under SYCL, so using them will fail at compile time.

See also the example ``examples/device-policy-aliases.cpp``.

======================================== ============= ==============================
SYCL Execution Policies Works with Brief description
======================================== ============= ==============================
Expand Down
14 changes: 14 additions & 0 deletions docs/sphinx/user_guide/tutorial/launch_basic.rst
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,20 @@ then enclosed by a host/device lambda which takes a
``RAJA::LaunchContext`` object, which may be used to control the flow
within the kernel, for example by creating thread-team synchronization points.

.. note::
RAJA treats ``Teams(i,j,k)`` and ``Threads(i,j,k)`` as an (x,y,z) ordering.
For users who prefer SYCL's (dim0, dim1, dim2) ordering, RAJA provides
``Teams::sycl_order(dim0, dim1, dim2)`` and
``Threads::sycl_order(dim0, dim1, dim2)``, which map SYCL dimensions to
RAJA coordinates as ``x = dim2``, ``y = dim1``, and ``z = dim0``. In other
words, ``Teams::sycl_order(dim0, dim1, dim2)`` is equivalent to
``Teams(dim2, dim1, dim0)``, and similarly for ``Threads``. For example::

RAJA::LaunchParams(RAJA::Teams::sycl_order(g0, g1, g2),
RAJA::Threads::sycl_order(l0, l1, l2))

See also the example ``examples/launch-device-policy-aliases.cpp``.

Inside the execution space, developers write a kernel using nested
``RAJA::loop`` methods. The manner in which each loop is executed
is determined by a template parameter type, which
Expand Down
8 changes: 8 additions & 0 deletions examples/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,14 @@ raja_add_executable(
NAME dynamic-forall
SOURCES dynamic-forall.cpp)

raja_add_executable(
NAME device-policy-aliases
SOURCES device-policy-aliases.cpp)

raja_add_executable(
NAME launch-device-policy-aliases
SOURCES launch-device-policy-aliases.cpp)

if (RAJA_ENABLE_JIT)
raja_add_executable(
NAME forall-jit
Expand Down
114 changes: 114 additions & 0 deletions examples/device-policy-aliases.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~//
// Copyright (c) Lawrence Livermore National Security, LLC and other
// RAJA Project Developers. See top-level LICENSE and COPYRIGHT
// files for dates and other details. No copyright assignment is required
// to contribute to RAJA.
//
// SPDX-License-Identifier: (BSD-3-Clause)
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~//

//
// Device policy aliases example
//
// This example demonstrates:
// - Backend-generic GPU execution policies via RAJA::device_*
// - CUDA-like x/y/z naming for SYCL (x->2, y->1, z->0) in those aliases
// - RAJA::Teams::sycl_order / RAJA::Threads::sycl_order for expressing a
// launch grid in SYCL's (dim0,dim1,dim2) ordering.
//
// Notes:
// - RAJA::device_* aliases are defined only in a GPU device compile pass
// (RAJA_GPU_ACTIVE). If you compile this file as host-only, it will fall
// back to a sequential path.
//

#include <iostream>

#include "RAJA/RAJA.hpp"
#include "RAJA/util/resource.hpp"

static void printDims(const char* label, const int v[3])
{
std::cout << label << " (x,y,z)=(" << v[0] << "," << v[1] << "," << v[2]
<< ")\n";
}

int main(int, char**)
{
constexpr int N = 1024;

// Demonstrate the (dim0,dim1,dim2)->(x,y,z) mapping helpers for RAJA::launch.
{
const auto threads_xyz = RAJA::Threads(8, 4, 2);
const auto threads_sycl = RAJA::Threads::sycl_order(/*dim0=*/2,
/*dim1=*/4,
/*dim2=*/8);
printDims("Threads(x,y,z)", threads_xyz.value);
printDims("Threads::sycl_order(dim0,dim1,dim2)", threads_sycl.value);
}

RAJA::resources::Host host {};
int* a = host.allocate<int>(N);
int* b = host.allocate<int>(N);
int* c = host.allocate<int>(N);

for (int i = 0; i < N; ++i)
{
a[i] = i;
b[i] = 2 * i;
c[i] = 0;
}

#if defined(RAJA_GPU_ACTIVE)
// Use RAJA::device_exec to select the active GPU backend (CUDA/HIP/SYCL).
//
// The matching resource type is inferred from the execution policy.
auto device_res = RAJA::resources::get_default_resource<RAJA::device_exec<256>>();

int* d_a = device_res.allocate<int>(N);
int* d_b = device_res.allocate<int>(N);
int* d_c = device_res.allocate<int>(N);

device_res.memcpy(d_a, a, sizeof(int) * N);
device_res.memcpy(d_b, b, sizeof(int) * N);

RAJA::forall<RAJA::device_exec<256>>(device_res, RAJA::RangeSegment(0, N),
[=] RAJA_DEVICE(int i) {
d_c[i] = d_a[i] + d_b[i];
});

device_res.memcpy(c, d_c, sizeof(int) * N);
device_res.wait();

device_res.deallocate(d_a);
device_res.deallocate(d_b);
device_res.deallocate(d_c);

std::cout << "Computed on device using RAJA::device_exec<256>\n";
#else
RAJA::forall<RAJA::seq_exec>(host, RAJA::RangeSegment(0, N), [=](int i) {
c[i] = a[i] + b[i];
});
std::cout << "Computed on host (no RAJA_GPU_ACTIVE)\n";
#endif

// Quick correctness check.
bool ok = true;
for (int i = 0; i < N; ++i)
{
if (c[i] != a[i] + b[i])
{
ok = false;
std::cerr << "Mismatch at i=" << i << " got " << c[i]
<< " expected " << (a[i] + b[i]) << "\n";
break;
}
}

host.deallocate(a);
host.deallocate(b);
host.deallocate(c);

return ok ? 0 : 1;
}

Loading
Loading