[lldb][test] Add support for building Wasm test inferiors#192872
Conversation
This PR adds support for building the test inferiors to WebAssembly. Specifically, it allows you to configure a sysroot and resource dir (pointing at the WASI SDK). The Wasm runtime can be configured through the `LLDB_TEST_USER_ARGS`. ``` LLDB_TEST_TRIPLE:STRING=wasm32-wasip1 LLDB_TEST_SYSROOT:PATH=/path/to/wasi-sdk-32.0-arm64-macos/share/wasi-sysroot LLDB_TEST_RESOURCE_DIR:PATH=/path/to/wasi-sdk-32.0-arm64-macos/lib/clang/22/ LLDB_TEST_USER_ARGS:STRING=--setting;platform.plugin.wasm.runtime-path=/path/to/iwasm;--setting;platform.plugin.wasm.runtime-args=--heap-size=1048576;--setting;platform.plugin.wasm.port-arg=-g= ``` With the configuration listed above I was able to confirm that I could build and run a handful of C and C++ tests. To set expectations: lots of tests are unsupported because they rely on things not available in Wasm (e.g. shared libraries) or they use features currently unsupported in LLDB (most notably: expression evaluation).
|
CC @MaxDesiatov |
|
@llvm/pr-subscribers-lldb Author: Jonas Devlieghere (JDevlieghere) ChangesThis PR adds support for building the test inferiors to WebAssembly. Specifically, it allows you to configure a sysroot and resource dir (pointing at the WASI SDK). The Wasm runtime can be configured through the With the configuration listed above I was able to confirm that I could build and run a handful of C and C++ tests. To set expectations: lots of tests are unsupported because they rely on things not available in Wasm (e.g. shared libraries) or they use features currently unsupported in LLDB (most notably: expression evaluation). Full diff: https://github.com/llvm/llvm-project/pull/192872.diff 11 Files Affected:
diff --git a/lldb/packages/Python/lldbsuite/test/builders/builder.py b/lldb/packages/Python/lldbsuite/test/builders/builder.py
index 03c1af579b018..1c557a64bd195 100644
--- a/lldb/packages/Python/lldbsuite/test/builders/builder.py
+++ b/lldb/packages/Python/lldbsuite/test/builders/builder.py
@@ -247,6 +247,11 @@ def getLibCxxArgs(self):
def getLLDBObjRoot(self):
return ["LLDB_OBJ_ROOT={}".format(configuration.lldb_obj_root)]
+ def getResourceDirArgs(self):
+ if configuration.resource_dir:
+ return ["RESOURCE_DIR={}".format(configuration.resource_dir)]
+ return []
+
def _getDebugInfoArgs(self, debug_info):
if debug_info is None:
return []
@@ -298,6 +303,7 @@ def getBuildCommand(
self.getModuleCacheSpec(),
self.getLibCxxArgs(),
self.getLLDBObjRoot(),
+ self.getResourceDirArgs(),
self.getCmdLine(dictionary),
]
command = list(itertools.chain(*command_parts))
diff --git a/lldb/packages/Python/lldbsuite/test/configuration.py b/lldb/packages/Python/lldbsuite/test/configuration.py
index d1c933b35fcdf..e1189f0f31d03 100644
--- a/lldb/packages/Python/lldbsuite/test/configuration.py
+++ b/lldb/packages/Python/lldbsuite/test/configuration.py
@@ -50,6 +50,9 @@
# Allow specifying a triple for cross compilation.
triple = None
+# Clang resource directory for cross compilation.
+resource_dir = None
+
# The overriden dwarf verison.
# Don't use this to test the current compiler's
# DWARF version, as this won't be set if the
diff --git a/lldb/packages/Python/lldbsuite/test/dotest.py b/lldb/packages/Python/lldbsuite/test/dotest.py
index 7f73fce14bcdd..45a0d68e5364b 100644
--- a/lldb/packages/Python/lldbsuite/test/dotest.py
+++ b/lldb/packages/Python/lldbsuite/test/dotest.py
@@ -276,6 +276,8 @@ def parseOptionsAndInitTestdirs():
configuration.dsymutil = seven.get_command_output(
"xcrun -find -toolchain default dsymutil"
)
+ if args.resource_dir:
+ configuration.resource_dir = args.resource_dir
if args.llvm_tools_dir:
configuration.llvm_tools_dir = args.llvm_tools_dir
configuration.filecheck = shutil.which("FileCheck", path=args.llvm_tools_dir)
diff --git a/lldb/packages/Python/lldbsuite/test/dotest_args.py b/lldb/packages/Python/lldbsuite/test/dotest_args.py
index f3b0837bdc4ab..a3d6db2e2d53a 100644
--- a/lldb/packages/Python/lldbsuite/test/dotest_args.py
+++ b/lldb/packages/Python/lldbsuite/test/dotest_args.py
@@ -106,6 +106,13 @@ def create_parser():
dest="dsymutil",
help=textwrap.dedent("Specify which dsymutil to use."),
)
+ group.add_argument(
+ "--resource-dir",
+ metavar="dir",
+ dest="resource_dir",
+ default="",
+ help=textwrap.dedent("Specify the clang resource directory for cross-compiling test inferiors."),
+ )
group.add_argument(
"--llvm-tools-dir",
metavar="dir",
diff --git a/lldb/packages/Python/lldbsuite/test/lldbplatformutil.py b/lldb/packages/Python/lldbsuite/test/lldbplatformutil.py
index a3fab6e49c2a7..cccba1b6f31db 100644
--- a/lldb/packages/Python/lldbsuite/test/lldbplatformutil.py
+++ b/lldb/packages/Python/lldbsuite/test/lldbplatformutil.py
@@ -108,7 +108,22 @@ def finalize_build_dictionary(dictionary):
if dictionary is None:
dictionary = {}
- if target_is_android():
+
+ if configuration.triple:
+ # When cross-compiling with an explicit triple, derive OS from it
+ # rather than from the selected platform.
+ triple_os = configuration.triple.split("-")[1] if "-" in configuration.triple else ""
+ if triple_os.startswith("wasi"):
+ dictionary["OS"] = "Wasm"
+ elif triple_os == "linux" or triple_os.startswith("linux"):
+ dictionary["OS"] = "Linux"
+ elif triple_os == "windows" or triple_os.startswith("windows"):
+ dictionary["OS"] = "Windows_NT"
+ elif triple_os == "apple":
+ dictionary["OS"] = "Darwin"
+ else:
+ dictionary["OS"] = triple_os
+ elif target_is_android():
dictionary["OS"] = "Android"
dictionary["PIE"] = 1
elif platformIsDarwin():
diff --git a/lldb/packages/Python/lldbsuite/test/lldbtest.py b/lldb/packages/Python/lldbsuite/test/lldbtest.py
index f2a9f3bba1993..5289a5ef0a189 100644
--- a/lldb/packages/Python/lldbsuite/test/lldbtest.py
+++ b/lldb/packages/Python/lldbsuite/test/lldbtest.py
@@ -822,7 +822,7 @@ def setUpCommands(cls):
# Set any user-overridden settings.
for setting, value in configuration.settings:
- commands.append("setting set %s %s" % (setting, value))
+ commands.append("settings set -- %s %s" % (setting, value))
# Make sure that a sanitizer LLDB's environment doesn't get passed on.
if (
diff --git a/lldb/packages/Python/lldbsuite/test/make/Makefile.rules b/lldb/packages/Python/lldbsuite/test/make/Makefile.rules
index 5cff3f5d91539..44063a47b56ae 100644
--- a/lldb/packages/Python/lldbsuite/test/make/Makefile.rules
+++ b/lldb/packages/Python/lldbsuite/test/make/Makefile.rules
@@ -118,6 +118,10 @@ ifeq "$(OS)" "Android"
include $(THIS_FILE_DIR)/Android.rules
endif
+ifeq "$(OS)" "Wasm"
+ include $(THIS_FILE_DIR)/Wasm.rules
+endif
+
ifeq "$(TRIPLE)" ""
ifeq "$(ARCH)" ""
# No triple, no arch: query the compiler for its default triple.
@@ -172,7 +176,11 @@ ifeq "$(OS)" "Darwin"
else
ifneq "$(SDKROOT)" ""
SYSROOT_FLAGS := --sysroot "$(SDKROOT)"
- GCC_TOOLCHAIN_FLAGS := --gcc-toolchain="$(SDKROOT)/usr"
+ ifeq "$(OS)" "Wasm"
+ GCC_TOOLCHAIN_FLAGS :=
+ else
+ GCC_TOOLCHAIN_FLAGS := --gcc-toolchain="$(SDKROOT)/usr"
+ endif
else
# Do not set up these options if SDKROOT was not specified.
# This is a regular build in that case (or Android).
diff --git a/lldb/packages/Python/lldbsuite/test/make/Wasm.rules b/lldb/packages/Python/lldbsuite/test/make/Wasm.rules
new file mode 100644
index 0000000000000..a28521391d410
--- /dev/null
+++ b/lldb/packages/Python/lldbsuite/test/make/Wasm.rules
@@ -0,0 +1,14 @@
+USE_SYSTEM_STDLIB = 1
+
+ifneq "$(RESOURCE_DIR)" ""
+ARCH_CFLAGS += -resource-dir $(RESOURCE_DIR)
+endif
+
+ARCH_CXXFLAGS += \
+ -fno-exceptions
+
+ARCH_LDFLAGS += \
+ $(if $(RESOURCE_DIR),-resource-dir $(RESOURCE_DIR)) \
+ -Wl,--export-all \
+ -Wl,--no-entry \
+ -Wl,--allow-undefined
diff --git a/lldb/test/API/CMakeLists.txt b/lldb/test/API/CMakeLists.txt
index 0738278c63a9c..bff3bac438d6b 100644
--- a/lldb/test/API/CMakeLists.txt
+++ b/lldb/test/API/CMakeLists.txt
@@ -92,6 +92,8 @@ set(LLDB_TEST_EXECUTABLE "${LLDB_DEFAULT_TEST_EXECUTABLE}" CACHE PATH "lldb exec
set(LLDB_TEST_COMPILER "${LLDB_DEFAULT_TEST_COMPILER}" CACHE PATH "C Compiler to use for building LLDB test inferiors")
set(LLDB_TEST_DSYMUTIL "${LLDB_DEFAULT_TEST_DSYMUTIL}" CACHE PATH "dsymutil used for generating dSYM bundles")
set(LLDB_TEST_MAKE "${LLDB_DEFAULT_TEST_MAKE}" CACHE PATH "make tool used for building test executables")
+set(LLDB_TEST_RESOURCE_DIR "" CACHE PATH "Clang resource directory for cross-compiling test inferiors")
+set(LLDB_TEST_SYSROOT "" CACHE PATH "Sysroot for cross-compiling test inferiors")
if ("${LLDB_TEST_COMPILER}" STREQUAL "")
message(FATAL_ERROR "LLDB test compiler not specified. Tests will not run.")
diff --git a/lldb/test/API/lit.cfg.py b/lldb/test/API/lit.cfg.py
index c92b104c9227c..f58e1e5ace5a3 100644
--- a/lldb/test/API/lit.cfg.py
+++ b/lldb/test/API/lit.cfg.py
@@ -323,6 +323,8 @@ def delete_module_cache(path):
dotest_cmd += ["--platform-working-dir", config.lldb_platform_working_dir]
if is_configured("cmake_sysroot"):
dotest_cmd += ["--sysroot", config.cmake_sysroot]
+if is_configured("test_resource_dir"):
+ dotest_cmd += ["--resource-dir", config.test_resource_dir]
if is_configured("dotest_user_args_str"):
dotest_cmd.extend(config.dotest_user_args_str.split(";"))
diff --git a/lldb/test/API/lit.site.cfg.py.in b/lldb/test/API/lit.site.cfg.py.in
index b7cd281425d7e..b1070c24bbb72 100644
--- a/lldb/test/API/lit.site.cfg.py.in
+++ b/lldb/test/API/lit.site.cfg.py.in
@@ -43,6 +43,7 @@ config.libcxx_libs_dir = "@LIBCXX_LIBRARY_DIR@"
config.libcxx_include_dir = "@LIBCXX_GENERATED_INCLUDE_DIR@"
config.libcxx_include_target_dir = "@LIBCXX_GENERATED_INCLUDE_TARGET_DIR@"
config.lldb_launcher = "@LLDB_LAUNCHER@"
+config.test_resource_dir = "@LLDB_TEST_RESOURCE_DIR@"
config.lldb_enable_mte = @LLDB_ENABLE_MTE@
config.lldb_enable_arm64e_debugserver = @LLDB_USE_ARM64E_DEBUGSERVER@
# The API tests use their own module caches.
|
|
✅ With the latest revision this PR passed the Python code formatter. |
Do runtimes support multiple processes? I tried the test suite against qemu-user and that was the main issue there. Which is my way of saying: I think it would be ok if we end up with a few more categories for things that don't/are never going to work in a wasm runtime generally. As long as we avoid too many |
| endif | ||
|
|
||
| ARCH_CXXFLAGS += \ | ||
| -fno-exceptions |
There was a problem hiding this comment.
Is this a global thing like WASM in general doesn't support this or is this something connected to the particular SDK being used at the moment?
Either is fine, just wondering.
There was a problem hiding this comment.
The WASI SDK I'm iterating with didn't have C++ exception support, but it seems like it was very recently added to WASI so we may be able to start using it in the (near) future: WebAssembly/wasi-sdk#565
There was a problem hiding this comment.
Not all runtimes have support for exception handling BTW. IIRC C++ exception are compiled down by Clang to Wasm exceptions? For WasmKit we would have to merge this one first swiftwasm/WasmKit#311
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/141/builds/17603 Here is the relevant piece of the build log for the reference |
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/211/builds/7704 Here is the relevant piece of the build log for the reference |
|
All Windows buildbots are broken. |
…feriors" (#193493) Reverts llvm/llvm-project#192872 Buildbot failures reported.
…feriors" (#193493) Reverts llvm/llvm-project#192872 Buildbot failures reported.
This PR adds support for building the test inferiors to WebAssembly. Specifically, it allows you to configure a sysroot and resource dir (pointing at the WASI SDK). The Wasm runtime can be configured through the `LLDB_TEST_USER_ARGS`. ``` LLDB_TEST_TRIPLE:STRING=wasm32-wasip1 LLDB_TEST_SYSROOT:PATH=/path/to/wasi-sdk-32.0-arm64-macos/share/wasi-sysroot LLDB_TEST_RESOURCE_DIR:PATH=/path/to/wasi-sdk-32.0-arm64-macos/lib/clang/22/ LLDB_TEST_USER_ARGS:STRING=--setting;platform.plugin.wasm.runtime-path=/path/to/iwasm;--setting;platform.plugin.wasm.runtime-args=--heap-size=1048576;--setting;platform.plugin.wasm.port-arg=-g= ``` With the configuration listed above I was able to confirm that I could build and run a handful of C and C++ tests. To set expectations: lots of tests are unsupported because they rely on things not available in Wasm (e.g. shared libraries) or they use features currently unsupported in LLDB (most notably: expression evaluation).
…lvm#193493) Reverts llvm#192872 Buildbot failures reported.
…feriors" (#193493) Reverts llvm/llvm-project#192872 Buildbot failures reported.
This PR adds support for building the test inferiors to WebAssembly. Specifically, it allows you to configure a sysroot and resource dir (pointing at the WASI SDK). The Wasm runtime can be configured through the `LLDB_TEST_USER_ARGS`. ``` LLDB_TEST_TRIPLE:STRING=wasm32-wasip1 LLDB_TEST_SYSROOT:PATH=/path/to/wasi-sdk-32.0-arm64-macos/share/wasi-sysroot LLDB_TEST_RESOURCE_DIR:PATH=/path/to/wasi-sdk-32.0-arm64-macos/lib/clang/22/ LLDB_TEST_USER_ARGS:STRING=--setting;platform.plugin.wasm.runtime-path=/path/to/iwasm;--setting;platform.plugin.wasm.runtime-args=--heap-size=1048576;--setting;platform.plugin.wasm.port-arg=-g= ``` With the configuration listed above I was able to confirm that I could build and run a handful of C and C++ tests. To set expectations: lots of tests are unsupported because they rely on things not available in Wasm (e.g. shared libraries) or they use features currently unsupported in LLDB (most notably: expression evaluation).
…lvm#193493) Reverts llvm#192872 Buildbot failures reported.
This PR adds support for building the test inferiors to WebAssembly. Specifically, it allows you to configure a sysroot and resource dir (pointing at the WASI SDK). The Wasm runtime can be configured through the `LLDB_TEST_USER_ARGS`. ``` LLDB_TEST_TRIPLE:STRING=wasm32-wasip1 LLDB_TEST_SYSROOT:PATH=/path/to/wasi-sdk-32.0-arm64-macos/share/wasi-sysroot LLDB_TEST_RESOURCE_DIR:PATH=/path/to/wasi-sdk-32.0-arm64-macos/lib/clang/22/ LLDB_TEST_USER_ARGS:STRING=--setting;platform.plugin.wasm.runtime-path=/path/to/iwasm;--setting;platform.plugin.wasm.runtime-args=--heap-size=1048576;--setting;platform.plugin.wasm.port-arg=-g= ``` With the configuration listed above I was able to confirm that I could build and run a handful of C and C++ tests. To set expectations: lots of tests are unsupported because they rely on things not available in Wasm (e.g. shared libraries) or they use features currently unsupported in LLDB (most notably: expression evaluation).
…lvm#193493) Reverts llvm#192872 Buildbot failures reported.
This PR adds support for building the test inferiors to WebAssembly. Specifically, it allows you to configure a sysroot and resource dir (pointing at the WASI SDK). The Wasm runtime can be configured through the
LLDB_TEST_USER_ARGS.With the configuration listed above I was able to confirm that I could build and run a handful of C and C++ tests. To set expectations: lots of tests are unsupported because they rely on things not available in Wasm (e.g. shared libraries) or they use features currently unsupported in LLDB (most notably: expression evaluation).