diff --git a/.gitmodules b/.gitmodules index ac76d6693..d1f766c2a 100644 --- a/.gitmodules +++ b/.gitmodules @@ -2,3 +2,6 @@ path = crates/moonutil/resources/error_codes url = https://github.com/moonbitlang/moonbit-docs.git branch = markdown-build +[submodule "third_party/moonbitlang_async"] + path = third_party/moonbitlang_async + url = https://github.com/moonbitlang/async.git diff --git a/CONTEXT.md b/CONTEXT.md new file mode 100644 index 000000000..440452a54 --- /dev/null +++ b/CONTEXT.md @@ -0,0 +1,60 @@ +# MoonBuild Async Wasm Runtime + +This context names the concepts used to describe `moonrun` support for running `moonbitlang/async` on the wasm backend. + +## Language + +**Semantic Stub Boundary**: +The operation-level contract exposed by `moonbitlang/async` native stubs, independent of native pointer layout or runtime object representation. +_Avoid_: Raw C ABI boundary + +**Mapped Parity**: +A compatibility goal where wasm host behavior is tracked against native async stub operations without requiring a literal translation of the C files. +_Avoid_: Rewrite, line-by-line port + +**Host Handle**: +An integer resource identity that the wasm guest can store and pass back while the host owns the underlying resource. +_Avoid_: Raw fd, raw HANDLE, externref + +**Guest Owner Struct**: +A wasm-side value that keeps MoonBit-owned data reachable while the host has a pending operation referring to its guest-memory range. +_Avoid_: Pinned guest pointer + +**Guest String Path**: +An async path argument passed from wasm to `moonrun` as a borrowed MoonBit `String` pointer plus a length measured in UTF-16 code units. +The guest must not pre-encode these paths as UTF-8 `Bytes`; `moonrun` converts the UTF-16 units into `OsString`, using the host's native path representation. +_Avoid_: UTF-8 path bytes, C string path + +**Source Provenance Import**: +A `moonbit_v0` import declared together with the async C-stub source file and native symbol it tracks. +_Avoid_: Untraceable host helper + +**Ported Symbol Origin**: +A V8-free Rust host implementation entry that records the async C-stub file and native symbol it is ported from. Active mapped imports must have both registry provenance and implementation provenance. +_Avoid_: Source comments that tests cannot verify + +**Async Sys Module**: +A V8-free, source-shaped Rust module that owns the behavior of a native async C-stub operation. It may use `AsyncHost` for shared runtime state, resource tables, guest-memory helpers, or shared ABI representation types, but the operation logic belongs in the sys module. +_Avoid_: Thin wrappers around behavior hidden in `AsyncHost` + +**Current Guest Memory**: +The `WebAssembly.Memory` object exposed as `moonbit_v0.memory` by the JS glue after instance creation or imported-memory discovery. +Host calls reacquire the current backing store for each import and never retain borrowed guest slices. +_Avoid_: Cached raw wasm pointer + +**Async Monotonic Time**: +The wasm host value returned for async `ms_since_epoch`. It has millisecond precision and is monotonic from an arbitrary process-local origin; callers may compare values by subtraction, but the absolute value is not meaningful. +_Avoid_: Wall-clock epoch + +## Boundary Decisions + +- `moonrun` keeps V8 as the first adapter, but async host state remains outside V8 types. +- `AsyncHost` owns shared runtime state, resource tables, guest-memory helpers, and shared ABI representation types. `async_sys` owns ported operation behavior. +- `moonbit_v0` imports strip the native `moonbitlang_async_` prefix and do not add an `async_` prefix. +- Native C stubs are the semantic reference. Rust code should stay structurally close to the source files, but it does not link against `moonbit.h` object layouts. +- Wasm async time uses a monotonic host clock from an unspecified origin. Native C stubs currently use platform wall-clock APIs, but async timer semantics only require elapsed millisecond differences. +- The async wasm host currently supports only Unix-family and Windows hosts. Other host families are compile-time unsupported. +- Variable-length data crosses the boundary through guest offsets and explicit lengths. Async jobs store host-owned buffers plus guest offsets, then copy into freshly reacquired guest memory during a later host call. +- Async path arguments are the exception to byte-buffer transport: they cross as Guest String Paths so Windows reaches `OsString`/wide OS calls without a guest UTF-8 encode followed by host UTF-16 re-encode. +- V8 memory growth can replace the observable memory backing store. The runtime must not lend guest pointers to OS APIs that need pinned buffers across `memory.grow`; use host-owned pinned buffers and copy to/from wasm memory instead. +- Windows APIs that require stable buffers should receive host-owned memory, not raw wasm memory. This includes overlapped IO and other APIs where the OS may retain a pointer until asynchronous completion. diff --git a/TODO.md b/TODO.md new file mode 100644 index 000000000..48ecb8313 --- /dev/null +++ b/TODO.md @@ -0,0 +1,41 @@ +# Async Wasm Runtime Audit TODO + +## Narrative + +And PR #1772 adds the `moonrun` host runtime needed for `moonbitlang/async` on the wasm backend, with a declared MVP surface around filesystem, fd, c_buffer, os_error, env, time, thread_pool, and limited process support. + +But upstream async wasm tests have been failing, and serializing timing-sensitive tests can hide scheduler or ABI bugs rather than proving the Rust host runtime faithfully preserves the native C-stub semantics. + +Therefore prioritize ABI, ownership, and completion correctness before relying on test serialization as evidence of stability. + +## Priority List + +1. [ ] Fix ABI and guest-memory lifetime safety. + And wasm jobs pass borrowed guest pointers that must remain valid until host completion. + But `Job::file_time_by_path` does not retain the output `FileTime`, and several copy-out paths are not transactional. + Therefore retain every guest owner needed by pending jobs and make `run_job`, `fetch_completion`, and `pipe` validate or roll back on copy-out failure. + +2. [ ] Prevent hangs and lost completions. + And the wasm event loop depends on every running worker job eventually publishing a completion. + But stale or freed job handles can currently make a worker return without queueing a completion. + Therefore make worker failure paths publish a completion or surface a deterministic error so the event loop cannot wait forever. + +3. [ ] Audit supported-surface parity only. + And the PR intentionally supports an MVP subset of async host imports. + But poll, direct IO, sockets, TLS, named pipes, and some spawn-job APIs are registered as unsupported. + Therefore verify parity for the supported surface and document unsupported imports as explicit scope boundaries. + +4. [ ] Check process behavior separately. + And wasm process support is partly custom glue rather than a direct C-stub port. + But Unix signaled child status, Windows argv quoting, Windows handle inheritance, and silently ignored spawn options can diverge from native behavior. + Therefore either match native semantics or fail loudly for unsupported process options. + +5. [ ] Stabilize tests after correctness fixes. + And upstream async tests include timing-sensitive cases. + But `--no-parallelize` or `--max-concurrent-tests` only reduces scheduling pressure. + Therefore apply serialization as a stability measure after ABI and completion bugs are addressed. + +6. [ ] Add focused regression coverage. + And upstream package success gives useful end-to-end confidence. + But it does not stress malformed guest ranges, too-small logical buffers, failed copy-outs, lost completions, or ownership mistakes. + Therefore add targeted tests for invalid guest buffers, file-time copy-out, completion fetch failure, pipe output failure, `file_time_by_path` ownership, and process edge cases. diff --git a/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_fs/app/main/fs_smoke.mbt b/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_fs/app/main/fs_smoke.mbt new file mode 100644 index 000000000..53fb2d25b --- /dev/null +++ b/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_fs/app/main/fs_smoke.mbt @@ -0,0 +1,18 @@ +///| +async test "write and read file through async fs" { + let path = "async-fs-smoke.txt" + @fs.write_file(path, b"wasm async fs", create_mode=CreateOrTruncate) + let content = @fs.read_file(path).text() + inspect(content, content="wasm async fs") + { + let file = @fs.open(path, mode=ReadWrite) + defer file.close() + file.write_at(b"IO", position=5) + let buf = FixedArray::make(2, b'\x00') + let n = file.read_at(buf, position=5) + inspect(n, content="2") + json_inspect(buf.unsafe_reinterpret_as_bytes()[:n], content="IO") + } + let content = @fs.read_file(path).text() + inspect(content, content="wasm IOync fs") +} diff --git a/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_fs/app/main/moon.pkg.json b/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_fs/app/main/moon.pkg.json new file mode 100644 index 000000000..0fddaf4c9 --- /dev/null +++ b/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_fs/app/main/moon.pkg.json @@ -0,0 +1,3 @@ +{ + "import": [ "moonbitlang/async", "moonbitlang/async/fs" ] +} diff --git a/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_fs/app/moon.mod.template b/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_fs/app/moon.mod.template new file mode 100644 index 000000000..0366cae3b --- /dev/null +++ b/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_fs/app/moon.mod.template @@ -0,0 +1,11 @@ +name = "moon/async_fs_workspace" + +version = "0.1.0" + +import { + "moonbitlang/async@0.19.1", +} + +options( + source: ".", +) diff --git a/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_fs/moon.work.template b/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_fs/moon.work.template new file mode 100644 index 000000000..c20ece352 --- /dev/null +++ b/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_fs/moon.work.template @@ -0,0 +1,5 @@ +members = [ + "./app", + "@@ASYNC_MEMBER@@", +] +preferred_target = "wasm" diff --git a/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_timer/app/main/moon.pkg.json b/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_timer/app/main/moon.pkg.json new file mode 100644 index 000000000..effc40e52 --- /dev/null +++ b/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_timer/app/main/moon.pkg.json @@ -0,0 +1,3 @@ +{ + "import": [ "moonbitlang/async" ] +} diff --git a/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_timer/app/main/timer.mbt b/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_timer/app/main/timer.mbt new file mode 100644 index 000000000..2053a485f --- /dev/null +++ b/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_timer/app/main/timer.mbt @@ -0,0 +1,5 @@ +///| +async test "timer sleep resumes" { + @async.sleep(1) + println("timer resumed") +} diff --git a/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_timer/app/moon.mod.template b/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_timer/app/moon.mod.template new file mode 100644 index 000000000..9d19b4853 --- /dev/null +++ b/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_timer/app/moon.mod.template @@ -0,0 +1,11 @@ +name = "moon/async_timer_workspace" + +version = "0.1.0" + +import { + "moonbitlang/async@0.19.1", +} + +options( + source: ".", +) diff --git a/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_timer/moon.work.template b/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_timer/moon.work.template new file mode 100644 index 000000000..c20ece352 --- /dev/null +++ b/crates/moon/tests/test_cases/moon_test/async_wasm_workspace_timer/moon.work.template @@ -0,0 +1,5 @@ +members = [ + "./app", + "@@ASYNC_MEMBER@@", +] +preferred_target = "wasm" diff --git a/crates/moon/tests/test_cases/moon_test/mod.rs b/crates/moon/tests/test_cases/moon_test/mod.rs index 27b7279a8..3443a32f9 100644 --- a/crates/moon/tests/test_cases/moon_test/mod.rs +++ b/crates/moon/tests/test_cases/moon_test/mod.rs @@ -1,3 +1,21 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + mod patch; #[cfg(unix)] mod use_cc_for_native_release; @@ -9,6 +27,115 @@ use crate::dry_run_utils::assert_lines_in_order; use super::*; +fn repo_root() -> std::path::PathBuf { + std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .parent() + .and_then(|path| path.parent()) + .unwrap() + .to_path_buf() +} + +// Upstream async has tick-sensitive tests; keep wasm package runs isolated +// from the Rust test harness's package-level concurrency. +static ASYNC_WASM_TEST_LOCK: std::sync::Mutex<()> = std::sync::Mutex::new(()); + +fn prepare_async_wasm_workspace(dir: &TestDir) -> std::path::PathBuf { + let repo_root = repo_root(); + let async_dir = repo_root.join("third_party/moonbitlang_async"); + let async_member = async_dir + .to_string_lossy() + .replace('\\', "\\\\") + .replace('"', "\\\""); + + std::fs::write( + dir.join("moon.work"), + crate::util::read(dir.join("moon.work.template")) + .replace("@@ASYNC_MEMBER@@", &async_member), + ) + .unwrap(); + std::fs::copy(dir.join("app/moon.mod.template"), dir.join("app/moon.mod")).unwrap(); + + async_dir +} + +fn build_moonrun() -> std::path::PathBuf { + let repo_root = repo_root(); + let cargo = std::env::var_os("CARGO").unwrap_or_else(|| "cargo".into()); + let mut build_moonrun = std::process::Command::new(cargo); + build_moonrun + .current_dir(&repo_root) + .args(["build", "--quiet", "-p", "moonrun"]); + if !cfg!(debug_assertions) { + build_moonrun.arg("--release"); + } + let status = build_moonrun + .status() + .expect("failed to spawn cargo build for moonrun"); + assert!(status.success(), "failed to build moonrun"); + + let mut path = std::env::current_exe().unwrap(); + path.pop(); + if path.ends_with("deps") { + path.pop(); + } + path.join(format!("moonrun{}", std::env::consts::EXE_SUFFIX)) +} + +fn run_async_wasm_package(dir: &TestDir, package: &str) -> String { + let _guard = ASYNC_WASM_TEST_LOCK + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + let moonrun = build_moonrun(); + let output = moon_cmd(dir) + .env("MOON_OVERRIDE", moon_bin()) + .env("MOONRUN_OVERRIDE", &moonrun) + .args([ + "-C", + "app/main", + "test", + "--target", + "wasm", + "--package", + package, + "--sort-input", + "--no-parallelize", + ]) + .assert() + .success() + .get_output() + .stdout + .clone(); + + std::str::from_utf8(&output).unwrap().to_owned() +} + +fn run_upstream_async_wasm_package(package: &str) -> String { + let _guard = ASYNC_WASM_TEST_LOCK + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + let moonrun = build_moonrun(); + let async_dir = repo_root().join("third_party/moonbitlang_async"); + let output = moon_cmd(&async_dir) + .env("MOON_OVERRIDE", moon_bin()) + .env("MOONRUN_OVERRIDE", &moonrun) + .args([ + "test", + "--target", + "wasm", + "--package", + package, + "--sort-input", + "--no-parallelize", + ]) + .assert() + .success() + .get_output() + .stdout + .clone(); + + std::str::from_utf8(&output).unwrap().to_owned() +} + #[test] fn test_moon_test_succ() { // TODO: Audit that the environment access only happens in single-threaded code. @@ -532,6 +659,83 @@ fn test_async_test() { check(last_line, expect!["Total tests: 1, passed: 0, failed: 1."]) } +#[test] +fn test_async_wasm_workspace_timer() { + let dir = TestDir::new("moon_test/async_wasm_workspace_timer"); + prepare_async_wasm_workspace(&dir); + + check( + run_async_wasm_package(&dir, "moon/async_timer_workspace/main"), + expect![[r#" + timer resumed + Total tests: 1, passed: 1, failed: 0. + "#]], + ); +} + +#[test] +fn test_async_wasm_workspace_fs_smoke() { + let dir = TestDir::new("moon_test/async_wasm_workspace_fs"); + prepare_async_wasm_workspace(&dir); + + check( + run_async_wasm_package(&dir, "moon/async_fs_workspace/main"), + expect![[r#" + Total tests: 1, passed: 1, failed: 0. + "#]], + ); +} + +#[test] +fn test_async_wasm_upstream_src_package() { + check( + run_upstream_async_wasm_package("moonbitlang/async"), + expect![[r#" + Total tests: 91, passed: 91, failed: 0. + "#]], + ); +} + +#[test] +fn test_async_wasm_upstream_aqueue_package() { + check( + run_upstream_async_wasm_package("moonbitlang/async/aqueue"), + expect![[r#" + Total tests: 52, passed: 52, failed: 0. + "#]], + ); +} + +#[test] +fn test_async_wasm_upstream_cond_var_package() { + check( + run_upstream_async_wasm_package("moonbitlang/async/cond_var"), + expect![[r#" + Total tests: 8, passed: 8, failed: 0. + "#]], + ); +} + +#[test] +fn test_async_wasm_upstream_semaphore_package() { + check( + run_upstream_async_wasm_package("moonbitlang/async/semaphore"), + expect![[r#" + Total tests: 12, passed: 12, failed: 0. + "#]], + ); +} + +#[test] +fn test_async_wasm_upstream_fs_package() { + check( + run_upstream_async_wasm_package("moonbitlang/async/fs"), + expect![[r#" + Total tests: 43, passed: 43, failed: 0. + "#]], + ); +} + #[test] fn test_max_concurrent_tests() { let dir = TestDir::new("moon_test"); diff --git a/crates/moonbuild/template/test_driver_project/template_async.mbt b/crates/moonbuild/template/test_driver_project/template_async.mbt index 0d4ad324d..f349ca72b 100644 --- a/crates/moonbuild/template/test_driver_project/template_async.mbt +++ b/crates/moonbuild/template/test_driver_project/template_async.mbt @@ -25,7 +25,18 @@ impl MoonBit_Async_Test_Driver for MoonBit_Async_Test_Driver_Impl with fn run_as } ///| -#cfg(not(any(target="wasm", target="wasm-gc"))) +#cfg(target="wasm") +impl MoonBit_Async_Test_Driver for MoonBit_Async_Test_Driver_Impl with fn run_async_tests( + tests, +) { + for f in tests { + @moonbitlang/async.run_async_main(f) + } + // {COVERAGE_END} +} + +///| +#cfg(not(target="wasm-gc")) impl MoonBit_Async_Test_Driver for MoonBit_Async_Test_Driver_Impl with fn is_being_cancelled() { @moonbitlang/async.is_being_cancelled() } diff --git a/crates/moonrun/CONTEXT.md b/crates/moonrun/CONTEXT.md new file mode 100644 index 000000000..4fc18c33f --- /dev/null +++ b/crates/moonrun/CONTEXT.md @@ -0,0 +1,38 @@ +# Moonrun + +Moonrun executes MoonBit wasm programs and provides host services that wasm code cannot perform directly. + +## Language + +**Job**: +A host operation requested by guest code whose result is observed later by the guest coroutine. +_Avoid_: Task, request + +**Worker**: +A host execution unit that runs a job outside the guest coroutine loop. +_Avoid_: Executor thread, background task + +**Completion**: +The host-owned result of a finished job that is ready to wake or resume guest code. +_Avoid_: Callback, event + +**Completion Queue**: +A host-owned queue of completed job identifiers that the guest event loop drains to resume waiting coroutines. +_Avoid_: Notify pipe, callback queue + +**Guest Memory**: +The wasm linear memory owned by the guest program. +_Avoid_: Wasm buffer, V8 memory + +**Guest String Path**: +A MoonBit `String` pointer plus UTF-16 code-unit length used for async path arguments crossing `moonbit_v0`. +Moonrun converts this directly into `OsString`; guest code must not send UTF-8 `Bytes` for paths. +_Avoid_: Guest UTF-8 path buffer + +**Host Buffer**: +Memory owned by moonrun while servicing guest jobs. +_Avoid_: Native buffer, temporary buffer + +**Native-Shaped Async Boundary**: +The wasm async host boundary that keeps MoonBit-facing concepts aligned with `moonbitlang/async` native concepts even when moonrun uses different host representations. +_Avoid_: Wasm-specific async API, shortcut API diff --git a/crates/moonrun/Cargo.toml b/crates/moonrun/Cargo.toml index 484a865c3..eb3133b53 100644 --- a/crates/moonrun/Cargo.toml +++ b/crates/moonrun/Cargo.toml @@ -40,8 +40,16 @@ libc.workspace = true version = "0.59.0" features = [ "Win32_Foundation", + "Win32_Networking_WinSock", + "Win32_Security", + "Win32_Storage_FileSystem", "Win32_System_Console", + "Win32_System_IO", + "Win32_System_Ioctl", + "Win32_System_Pipes", + "Win32_System_SystemServices", "Win32_System_Threading", + "Win32_System_WindowsProgramming", ] [build-dependencies] diff --git a/crates/moonrun/docs/adr/0001-copy-worker-results-through-host-completions.md b/crates/moonrun/docs/adr/0001-copy-worker-results-through-host-completions.md new file mode 100644 index 000000000..9575c97ed --- /dev/null +++ b/crates/moonrun/docs/adr/0001-copy-worker-results-through-host-completions.md @@ -0,0 +1,3 @@ +# Copy Worker Results Through Host Completions + +Moonrun will not let async workers retain guest-memory pointers or write directly into guest memory after an import returns. Worker jobs produce host-owned completions, and guest-visible result data is copied into current guest memory only when the guest resumes and calls back into moonrun. This adds a copy, but keeps wasm memory growth/replacement and worker-thread execution separated. diff --git a/crates/moonrun/docs/adr/0002-preserve-native-shaped-worker-boundary.md b/crates/moonrun/docs/adr/0002-preserve-native-shaped-worker-boundary.md new file mode 100644 index 000000000..037b124e7 --- /dev/null +++ b/crates/moonrun/docs/adr/0002-preserve-native-shaped-worker-boundary.md @@ -0,0 +1,3 @@ +# Preserve Native-Shaped Worker Boundary + +Moonrun will preserve the `moonbitlang/async` native-shaped `Worker` and `Job` boundary for wasm instead of exposing a separate wasm-specific scheduler API to MoonBit. In wasm, worker handles may represent scheduler resources rather than raw OS threads, but the MoonBit-facing structure should stay close to the native async implementation to reduce maintenance drift. diff --git a/crates/moonrun/docs/adr/0003-use-native-shaped-worker-threads-first.md b/crates/moonrun/docs/adr/0003-use-native-shaped-worker-threads-first.md new file mode 100644 index 000000000..f09986c33 --- /dev/null +++ b/crates/moonrun/docs/adr/0003-use-native-shaped-worker-threads-first.md @@ -0,0 +1,3 @@ +# Use Native-Shaped Worker Threads First + +Moonrun will implement the first wasm async executor with one host thread per active native-shaped worker handle, subject to the same worker-count policy as `moonbitlang/async`. This favors behavioral parity and easier cancellation reasoning over starting with a more abstract shared Rust executor pool. diff --git a/crates/moonrun/docs/adr/0004-use-host-completion-queue-for-wasm-workers.md b/crates/moonrun/docs/adr/0004-use-host-completion-queue-for-wasm-workers.md new file mode 100644 index 000000000..12e373269 --- /dev/null +++ b/crates/moonrun/docs/adr/0004-use-host-completion-queue-for-wasm-workers.md @@ -0,0 +1,7 @@ +# Use Host Completion Queue For Wasm Workers + +Moonrun will expose a native-shaped completion-drain import for wasm async workers rather than forcing an OS notify fd into the V8 adapter. The MoonBit side keeps the `fetch_completion` shape, while moonrun can implement the queue with host synchronization primitives and fill guest buffers only during the drain call. + +Worker threads must not write directly into wasm guest memory. They do not run inside a V8 import callback, and the current memory view must be reacquired after potential memory growth. Worker jobs therefore copy borrowed inputs into host-owned values at job creation, compute host-owned result payloads, publish a completion record, and let the guest thread copy output bytes while draining completions. + +This delayed copy-out is part of the boundary design for output buffers such as read, readdir, and file-time results. The MoonBit wasm wrapper treats jobs as opaque handles as the native backend does; `fetch_completion` is the single readiness-and-copy-out boundary. Fixed-size portable records should still avoid unnecessary intermediate structure: wasm `FileTime` is a 48-byte little-endian record, so completion draining should encode that record directly into guest memory and the MoonBit wasm wrapper should read its fields directly instead of calling back into host accessors for each field. diff --git a/crates/moonrun/docs/adr/0005-suspend-coroutines-for-wasm-worker-jobs.md b/crates/moonrun/docs/adr/0005-suspend-coroutines-for-wasm-worker-jobs.md new file mode 100644 index 000000000..12d099fed --- /dev/null +++ b/crates/moonrun/docs/adr/0005-suspend-coroutines-for-wasm-worker-jobs.md @@ -0,0 +1,3 @@ +# Suspend Coroutines For Wasm Worker Jobs + +Moonrun will make wasm worker jobs suspend the waiting coroutine once the real executor is implemented. The current synchronous `run_job` path is only a temporary runnable slice; leaving it as observable behavior would block the guest event loop and diverge from `moonbitlang/async` native semantics. diff --git a/crates/moonrun/docs/adr/0006-let-workers-own-running-jobs.md b/crates/moonrun/docs/adr/0006-let-workers-own-running-jobs.md new file mode 100644 index 000000000..46829dc23 --- /dev/null +++ b/crates/moonrun/docs/adr/0006-let-workers-own-running-jobs.md @@ -0,0 +1,3 @@ +# Let Workers Own Running Jobs + +Moonrun will follow `moonbitlang/async` by treating a worker as owning its current running job while the blocking operation executes. The guest-visible job handle remains stable, but moonrun should not require central job-table locks around blocking syscalls; completion publishes the finished job state back for the guest event loop to observe. diff --git a/crates/moonrun/docs/adr/0007-split-read-job-guest-destination-from-host-buffer.md b/crates/moonrun/docs/adr/0007-split-read-job-guest-destination-from-host-buffer.md new file mode 100644 index 000000000..b582f5863 --- /dev/null +++ b/crates/moonrun/docs/adr/0007-split-read-job-guest-destination-from-host-buffer.md @@ -0,0 +1,3 @@ +# Split Read Job Guest Destination From Host Buffer + +Moonrun read jobs will keep guest destination metadata on the stable job state while workers read into host-owned buffers. Completion records the produced byte count, and moonrun copies the host buffer into current guest memory only when the guest resumes. This preserves the native read-job contract without retaining guest-memory pointers in worker threads. diff --git a/crates/moonrun/src/async_api.rs b/crates/moonrun/src/async_api.rs new file mode 100644 index 000000000..adab0b995 --- /dev/null +++ b/crates/moonrun/src/async_api.rs @@ -0,0 +1,52 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +mod c_buffer; +mod context; +mod env_util; +mod event_loop; +mod fd_util; +mod fs; +mod memory; +mod os_error; +mod process; +mod registry; +mod runtime; +mod socket; +mod thread_pool; +mod time; +mod tls; +mod unsupported; + +use std::any::Any; + +use crate::async_host::AsyncHost; + +pub(crate) use registry::MOONBIT_V0_MODULE; + +pub(crate) fn init_env<'s>( + obj: v8::Local<'s, v8::Object>, + scope: &mut v8::HandleScope<'s>, + dtors: &mut Vec>, +) { + let context = Box::new(context::AsyncContext::new(scope, obj, AsyncHost::default())); + let context_ptr = &*context as *const context::AsyncContext; + dtors.push(context); + + registry::register_imports(obj, scope, context_ptr); +} diff --git a/crates/moonrun/src/async_api/c_buffer.rs b/crates/moonrun/src/async_api/c_buffer.rs new file mode 100644 index 000000000..678b342ba --- /dev/null +++ b/crates/moonrun/src/async_api/c_buffer.rs @@ -0,0 +1,163 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_host::{AsyncHostError, AsyncHostResult, checked_mut_range, checked_range}; +use crate::async_sys::internal::c_buffer::stub; + +use super::context::{ + AsyncContext, ImportArgs, callback_context, finish_bool, throw_import_error, with_memory_mut, +}; + +pub(super) fn blit_to_c( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + if let Err(error) = blit_to_c_impl(scope, &args, context) { + throw_import_error(scope, "c_buffer/blit_to_c", error); + return; + } + ret.set_undefined(); +} + +pub(super) fn blit_from_c( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + if let Err(error) = blit_from_c_impl(scope, &args, context) { + throw_import_error(scope, "c_buffer/blit_from_c", error); + return; + } + ret.set_undefined(); +} + +pub(super) fn c_buffer_get( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + match c_buffer_get_impl(scope, &args, context) { + Ok(byte) => ret.set_int32(i32::from(byte)), + Err(error) => throw_import_error(scope, "c_buffer/c_buffer_get", error), + } +} + +pub(super) fn strlen( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + match strlen_impl(scope, &args, context) { + Ok(len) => ret.set_int32(len), + Err(error) => throw_import_error(scope, "c_buffer/strlen", error), + } +} + +pub(super) fn null_pointer( + _scope: &mut v8::HandleScope, + _args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + ret.set_int32(stub::null_pointer()); +} + +pub(super) fn pointer_is_null( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let mut args = ImportArgs::new(scope, &args); + match args.i32(0) { + Ok(ptr) => finish_bool(&mut ret, stub::pointer_is_null(ptr)), + Err(error) => throw_import_error(scope, "c_buffer/pointer_is_null", error), + } +} + +fn blit_to_c_impl( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, +) -> AsyncHostResult<()> { + let mut args = ImportArgs::new(scope, args); + let dst = args.i32(0)?; + let src = args.i32(1)?; + let offset = args.i32(2)?; + let len = args.i32(3)?; + let src_len = checked_add_i32(offset, len)?; + with_memory_mut(scope, context, |memory| { + let src = checked_range(memory, src, src_len)?.to_vec(); + let dst = checked_mut_range(memory, dst, len)?; + stub::blit_to_c(dst, &src, offset, len) + }) +} + +fn blit_from_c_impl( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, +) -> AsyncHostResult<()> { + let mut args = ImportArgs::new(scope, args); + let src = args.i32(0)?; + let dst = args.i32(1)?; + let offset = args.i32(2)?; + let len = args.i32(3)?; + let dst_len = checked_add_i32(offset, len)?; + with_memory_mut(scope, context, |memory| { + let src = checked_range(memory, src, len)?.to_vec(); + let dst = checked_mut_range(memory, dst, dst_len)?; + stub::blit_from_c(&src, dst, offset, len) + }) +} + +fn c_buffer_get_impl( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, +) -> AsyncHostResult { + let mut args = ImportArgs::new(scope, args); + let buf = args.i32(0)?; + let index = args.i32(1)?; + let len = checked_add_i32(index, 1)?; + with_memory_mut(scope, context, |memory| { + let buf = checked_range(memory, buf, len)?; + stub::c_buffer_get(buf, index) + }) +} + +fn strlen_impl( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, +) -> AsyncHostResult { + let mut args = ImportArgs::new(scope, args); + let buf = args.i32(0)?; + let offset = usize::try_from(buf).map_err(|_| AsyncHostError::Fault)?; + with_memory_mut(scope, context, |memory| { + let buf = memory.get(offset..).ok_or(AsyncHostError::Fault)?; + stub::strlen(buf) + }) +} + +fn checked_add_i32(lhs: i32, rhs: i32) -> AsyncHostResult { + lhs.checked_add(rhs).ok_or(AsyncHostError::Fault) +} diff --git a/crates/moonrun/src/async_api/context.rs b/crates/moonrun/src/async_api/context.rs new file mode 100644 index 000000000..277ebb93b --- /dev/null +++ b/crates/moonrun/src/async_api/context.rs @@ -0,0 +1,141 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_host::{AsyncHost, AsyncHostError, AsyncHostResult}; + +const ASYNC_ERRNO_SUCCESS: i32 = 0; + +pub(super) struct AsyncContext { + pub(super) host: AsyncHost, + imports: v8::Global, +} + +impl AsyncContext { + pub(super) fn new<'s>( + scope: &mut v8::HandleScope<'s>, + imports: v8::Local<'s, v8::Object>, + host: AsyncHost, + ) -> Self { + Self { + host, + imports: v8::Global::new(scope, imports), + } + } +} + +pub(super) fn callback_context<'s>(args: &v8::FunctionCallbackArguments<'s>) -> &'s AsyncContext { + let data = args.data(); + assert!(data.is_external()); + let data: v8::Local = data.into(); + let ptr = v8::Local::::try_from(data).unwrap().value(); + unsafe { &*(ptr as *const AsyncContext) } +} + +pub(super) struct ImportArgs<'a, 'scope, 'args> { + scope: &'a mut v8::HandleScope<'scope>, + args: &'a v8::FunctionCallbackArguments<'args>, +} + +impl<'a, 'scope, 'args> ImportArgs<'a, 'scope, 'args> { + pub(super) fn new( + scope: &'a mut v8::HandleScope<'scope>, + args: &'a v8::FunctionCallbackArguments<'args>, + ) -> Self { + Self { scope, args } + } + + pub(super) fn i32(&mut self, index: i32) -> AsyncHostResult { + self.args + .get(index) + .int32_value(self.scope) + .ok_or(AsyncHostError::Inval) + } + + pub(super) fn i64(&mut self, index: i32) -> AsyncHostResult { + let value = self.args.get(index); + if value.is_big_int() { + let bigint = + v8::Local::::try_from(value).map_err(|_| AsyncHostError::Inval)?; + let (result, lossless) = bigint.i64_value(); + if lossless { + return Ok(result); + } + } + value.integer_value(self.scope).ok_or(AsyncHostError::Inval) + } +} + +fn memory_object<'s>( + scope: &mut v8::HandleScope<'s>, + context: &AsyncContext, +) -> AsyncHostResult> { + let imports = v8::Local::new(scope, &context.imports); + let key = v8::String::new(scope, "memory").ok_or(AsyncHostError::Fault)?; + let memory = imports + .get(scope, key.into()) + .ok_or(AsyncHostError::Fault)?; + v8::Local::::try_from(memory).map_err(|_| AsyncHostError::Fault) +} + +pub(super) fn with_memory_mut( + scope: &mut v8::HandleScope, + context: &AsyncContext, + f: impl FnOnce(&mut [u8]) -> AsyncHostResult, +) -> AsyncHostResult { + let memory_object = memory_object(scope, context)?; + let buffer = memory_object.buffer(); + let len = buffer.byte_length(); + + let Some(ptr) = buffer.data() else { + if len == 0 { + let mut empty = []; + return f(&mut empty); + } + return Err(AsyncHostError::Fault); + }; + + let memory = unsafe { std::slice::from_raw_parts_mut(ptr.as_ptr() as *mut u8, len) }; + f(memory) +} + +pub(super) fn finish_errno( + context: &AsyncContext, + ret: &mut v8::ReturnValue, + result: AsyncHostResult<()>, +) { + let errno = match result { + Ok(()) => ASYNC_ERRNO_SUCCESS, + Err(error) => context.host.record_error(error), + }; + ret.set_int32(errno); +} + +pub(super) fn finish_bool(ret: &mut v8::ReturnValue, value: bool) { + ret.set_int32(if value { 1 } else { 0 }); +} + +pub(super) fn throw_import_error( + scope: &mut v8::HandleScope, + import_name: &str, + error: AsyncHostError, +) { + let message = format!("moonbit_v0.{import_name} failed: {error:?}"); + let message = v8::String::new(scope, &message).unwrap_or_else(|| v8::String::empty(scope)); + let exception = v8::Exception::error(scope, message); + scope.throw_exception(exception); +} diff --git a/crates/moonrun/src/async_api/env_util.rs b/crates/moonrun/src/async_api/env_util.rs new file mode 100644 index 000000000..c4c289c0d --- /dev/null +++ b/crates/moonrun/src/async_api/env_util.rs @@ -0,0 +1,27 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_sys::internal::env_util::stub; + +pub(super) fn getpid( + scope: &mut v8::HandleScope, + _args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + ret.set(v8::Integer::new_from_unsigned(scope, stub::get_pid()).into()); +} diff --git a/crates/moonrun/src/async_api/event_loop.rs b/crates/moonrun/src/async_api/event_loop.rs new file mode 100644 index 000000000..3df6f66e8 --- /dev/null +++ b/crates/moonrun/src/async_api/event_loop.rs @@ -0,0 +1,45 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_sys::internal::event_loop::thread_pool; + +use super::context::{ImportArgs, callback_context, finish_bool}; + +pub(super) fn get_platform( + _scope: &mut v8::HandleScope, + _args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + ret.set_int32(thread_pool::get_platform()); +} + +pub(super) fn errno_is_cancelled( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let mut args = ImportArgs::new(scope, &args); + match args.i32(0) { + Ok(errno) => finish_bool(&mut ret, thread_pool::errno_is_cancelled(errno)), + Err(error) => { + context.host.record_error(error); + finish_bool(&mut ret, false); + } + } +} diff --git a/crates/moonrun/src/async_api/fd_util.rs b/crates/moonrun/src/async_api/fd_util.rs new file mode 100644 index 000000000..147089807 --- /dev/null +++ b/crates/moonrun/src/async_api/fd_util.rs @@ -0,0 +1,198 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_host::{AsyncHostError, AsyncHostResult, GuestMemory}; + +use super::context::{ + AsyncContext, ImportArgs, callback_context, throw_import_error, with_memory_mut, +}; + +const FILE_TIME_RECORD_LEN: i32 = 48; +const ATIME_SEC_OFFSET: i32 = 0; +const ATIME_NSEC_OFFSET: i32 = 8; +const MTIME_SEC_OFFSET: i32 = 16; +const MTIME_NSEC_OFFSET: i32 = 24; +const CTIME_SEC_OFFSET: i32 = 32; +const CTIME_NSEC_OFFSET: i32 = 40; + +pub(super) fn sizeof_file_time( + _scope: &mut v8::HandleScope, + _args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + ret.set_int32(FILE_TIME_RECORD_LEN); +} + +pub(super) fn get_atime_sec( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + file_time_i64( + scope, + &args, + ATIME_SEC_OFFSET, + "fd_util/get_atime_sec", + &mut ret, + ); +} + +pub(super) fn get_atime_nsec( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + file_time_i32( + scope, + &args, + ATIME_NSEC_OFFSET, + "fd_util/get_atime_nsec", + &mut ret, + ); +} + +pub(super) fn get_mtime_sec( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + file_time_i64( + scope, + &args, + MTIME_SEC_OFFSET, + "fd_util/get_mtime_sec", + &mut ret, + ); +} + +pub(super) fn get_mtime_nsec( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + file_time_i32( + scope, + &args, + MTIME_NSEC_OFFSET, + "fd_util/get_mtime_nsec", + &mut ret, + ); +} + +pub(super) fn get_ctime_sec( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + file_time_i64( + scope, + &args, + CTIME_SEC_OFFSET, + "fd_util/get_ctime_sec", + &mut ret, + ); +} + +pub(super) fn get_ctime_nsec( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + file_time_i32( + scope, + &args, + CTIME_NSEC_OFFSET, + "fd_util/get_ctime_nsec", + &mut ret, + ); +} + +pub(super) fn pipe( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + let dst = args.i32(0)?; + let len = args.i32(1)?; + if len < 2 { + return Err(AsyncHostError::Fault); + } + let fds = context.host.pipe()?; + with_memory_mut(scope, context, |memory| { + memory.write_i32_le(dst, fds[0])?; + memory.write_i32_le(dst.checked_add(4).ok_or(AsyncHostError::Fault)?, fds[1]) + }) + })(); + match result { + Ok(()) => ret.set_int32(0), + Err(error) => { + context.host.record_error(error); + ret.set_int32(-1); + } + } +} + +fn file_time_i64( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + field_offset: i32, + name: &str, + ret: &mut v8::ReturnValue, +) { + let context = callback_context(args); + match read_field(scope, args, context, field_offset, 8) + .map(|bytes| i64::from_le_bytes(bytes.as_slice().try_into().unwrap())) + { + Ok(value) => ret.set(v8::BigInt::new_from_i64(scope, value).into()), + Err(error) => throw_import_error(scope, name, error), + } +} + +fn file_time_i32( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + field_offset: i32, + name: &str, + ret: &mut v8::ReturnValue, +) { + let context = callback_context(args); + match read_field(scope, args, context, field_offset, 4) + .map(|bytes| i32::from_le_bytes(bytes.as_slice().try_into().unwrap())) + { + Ok(value) => ret.set_int32(value), + Err(error) => throw_import_error(scope, name, error), + } +} + +fn read_field( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, + field_offset: i32, + len: i32, +) -> AsyncHostResult> { + let mut args = ImportArgs::new(scope, args); + let ptr = args.i32(0)?; + let offset = ptr.checked_add(field_offset).ok_or(AsyncHostError::Fault)?; + with_memory_mut(scope, context, |memory| { + Ok(memory.read_exact(offset, len)?.to_vec()) + }) +} diff --git a/crates/moonrun/src/async_api/fs.rs b/crates/moonrun/src/async_api/fs.rs new file mode 100644 index 000000000..c482f6932 --- /dev/null +++ b/crates/moonrun/src/async_api/fs.rs @@ -0,0 +1,323 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_host::{AsyncHostError, AsyncHostResult, GuestMemory}; +use crate::async_sys::fs::dir; +use crate::async_sys::fs::stub; + +use super::context::{ + AsyncContext, ImportArgs, callback_context, finish_bool, throw_import_error, with_memory_mut, +}; + +pub(super) fn get_tmp_path_len( + _scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + match get_tmp_path_len_impl() { + Ok(len) => ret.set_int32(len), + Err(error) => { + context.host.record_error(error); + ret.set_int32(-1); + } + } +} + +pub(super) fn get_tmp_path( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + match get_tmp_path_impl(scope, &args, context) { + Ok(()) => ret.set_int32(0), + Err(error) => { + context.host.record_error(error); + ret.set_int32(-1); + } + } +} + +pub(super) fn close_fd( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + context.host.close_fd(args.i32(0)?) + })(); + match result { + Ok(()) => ret.set_int32(0), + Err(error) => { + context.host.record_error(error); + ret.set_int32(-1); + } + } +} + +pub(super) fn dir_buffer_min_size( + _scope: &mut v8::HandleScope, + _args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + ret.set_int32(dir::buffer_min_size()); +} + +pub(super) fn dir_entry_length( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + match with_dir_header(scope, &args, context, dir::entry_length) { + Ok(value) => ret.set_int32(value), + Err(error) => throw_import_error(scope, "fs/dir_entry_length", error), + } +} + +pub(super) fn dir_entry_name_len( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + match with_dir_header(scope, &args, context, dir::entry_name_len) { + Ok(value) => ret.set_int32(value), + Err(error) => throw_import_error(scope, "dir_entry_name_len", error), + } +} + +pub(super) fn dir_entry_name( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + let buf = args.i32(0)?; + let offset = args.i32(1)?; + with_memory_mut(scope, context, |memory| { + let header = dir_entry_header(memory, buf, offset)?; + dir::entry_name_len(header, 0)?; + dir::entry_name_ptr(buf, offset) + }) + })(); + match result { + Ok(value) => ret.set_int32(value), + Err(error) => throw_import_error(scope, "dir_entry_name", error), + } +} + +pub(super) fn dir_entry_is_dir( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + match with_dir_header(scope, &args, context, dir::entry_is_dir) { + Ok(value) => ret.set_int32(value), + Err(error) => throw_import_error(scope, "fs/dir_entry_is_dir", error), + } +} + +pub(super) fn dir_entry_is_hidden( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + match with_dir_header(scope, &args, context, dir::entry_is_hidden) { + Ok(value) => finish_bool(&mut ret, value), + Err(error) => throw_import_error(scope, "fs/dir_entry_is_hidden", error), + } +} + +pub(super) fn dir_entry_file_id( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + match with_dir_header(scope, &args, context, dir::entry_file_id) { + Ok(value) => ret.set(v8::BigInt::new_from_u64(scope, value).into()), + Err(error) => throw_import_error(scope, "dir_entry_file_id", error), + } +} + +fn get_tmp_path_len_impl() -> AsyncHostResult { + let len = tmp_path_utf16_units()?.len(); + i32::try_from(len).map_err(|_| AsyncHostError::Fault) +} + +fn get_tmp_path_impl( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, +) -> AsyncHostResult<()> { + let mut args = ImportArgs::new(scope, args); + let ptr = args.i32(0)?; + let len = args.i32(1)?; + let units = tmp_path_utf16_units()?; + let len = usize::try_from(len).map_err(|_| AsyncHostError::Fault)?; + if len != units.len() { + return Err(AsyncHostError::Inval); + } + let mut bytes = Vec::with_capacity(units.len() * 2); + for unit in units { + bytes.extend_from_slice(&unit.to_le_bytes()); + } + with_memory_mut(scope, context, |memory| memory.write_exact(ptr, &bytes)) +} + +fn with_dir_header( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, + f: impl FnOnce(&[u8], i32) -> AsyncHostResult, +) -> AsyncHostResult { + let mut args = ImportArgs::new(scope, args); + let buf = args.i32(0)?; + let offset = args.i32(1)?; + with_memory_mut(scope, context, |memory| { + let header = dir_entry_header(memory, buf, offset)?; + f(header, 0) + }) +} + +fn dir_entry_header( + memory: &(impl GuestMemory + ?Sized), + buf: i32, + offset: i32, +) -> AsyncHostResult<&[u8]> { + let header_ptr = buf.checked_add(offset).ok_or(AsyncHostError::Fault)?; + memory.read_exact(header_ptr, dir::HEADER_LEN as i32) +} + +fn tmp_path_utf16_units() -> AsyncHostResult> { + os_string_to_utf16_units(stub::get_tmp_path()?) +} + +#[cfg(unix)] +fn os_string_to_utf16_units(path: std::ffi::OsString) -> AsyncHostResult> { + use std::os::unix::ffi::OsStringExt; + + let path = String::from_utf8(path.into_vec()).map_err(|_| AsyncHostError::Inval)?; + Ok(path.encode_utf16().collect()) +} + +#[cfg(windows)] +fn os_string_to_utf16_units(path: std::ffi::OsString) -> AsyncHostResult> { + use std::os::windows::ffi::OsStrExt; + + Ok(path.as_os_str().encode_wide().collect()) +} + +pub(super) fn errno_is_lock_violation( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let mut args = ImportArgs::new(scope, &args); + match args.i32(0) { + Ok(errno) => finish_bool(&mut ret, stub::errno_is_lock_violation(errno)), + Err(error) => { + context.host.record_error(error); + finish_bool(&mut ret, false); + } + } +} + +pub(super) fn try_lock_file( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + let fd = args.i32(0)?; + let exclusive = args.i32(1)? != 0; + context.host.try_lock_file(fd, exclusive) + })(); + match result { + Ok(()) => ret.set_int32(0), + Err(error) => { + context.host.record_error(error); + ret.set_int32(-1); + } + } +} + +pub(super) fn unlock_file( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + context.host.unlock_file(args.i32(0)?) + })(); + match result { + Ok(()) => ret.set_int32(0), + Err(error) => { + context.host.record_error(error); + ret.set_int32(-1); + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[cfg(unix)] + #[test] + fn tmp_path_encodes_unix_path_as_utf16_units() { + let path = std::ffi::OsString::from("/tmp/\u{6587}"); + + let units = os_string_to_utf16_units(path).unwrap(); + + assert_eq!(units, "/tmp/\u{6587}".encode_utf16().collect::>()); + } + + #[cfg(unix)] + #[test] + fn tmp_path_rejects_non_utf8_unix_os_string() { + use std::os::unix::ffi::OsStringExt; + + let path = std::ffi::OsString::from_vec(b"/tmp/\xff".to_vec()); + + assert_eq!(os_string_to_utf16_units(path), Err(AsyncHostError::Inval)); + } + + #[cfg(windows)] + #[test] + fn tmp_path_preserves_windows_wide_units() { + let path = std::ffi::OsString::from("A\u{10000}"); + + let units = os_string_to_utf16_units(path).unwrap(); + + assert_eq!(units, vec![0x0041, 0xd800, 0xdc00]); + } +} diff --git a/crates/moonrun/src/async_api/memory.rs b/crates/moonrun/src/async_api/memory.rs new file mode 100644 index 000000000..c33d626da --- /dev/null +++ b/crates/moonrun/src/async_api/memory.rs @@ -0,0 +1,71 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_host::AsyncHostResult; + +use super::context::{AsyncContext, ImportArgs, callback_context, finish_errno, with_memory_mut}; + +pub(super) fn copy_from_guest( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + match copy_from_guest_impl(scope, &args, context) { + Ok(len) => ret.set_int32(len), + Err(error) => { + context.host.record_error(error); + ret.set_int32(-1); + } + } +} + +pub(super) fn zero_guest( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + finish_errno(context, &mut ret, zero_guest_impl(scope, &args, context)); +} + +fn copy_from_guest_impl( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, +) -> AsyncHostResult { + let mut args = ImportArgs::new(scope, args); + let ptr = args.i32(0)?; + let len = args.i32(1)?; + with_memory_mut(scope, context, |memory| { + context.host.copy_from_guest_len(memory, ptr, len) + }) +} + +fn zero_guest_impl( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, +) -> AsyncHostResult<()> { + let mut args = ImportArgs::new(scope, args); + let ptr = args.i32(0)?; + let len = args.i32(1)?; + with_memory_mut(scope, context, |memory| { + context.host.zero_guest(memory, ptr, len) + }) +} diff --git a/crates/moonrun/src/async_api/os_error.rs b/crates/moonrun/src/async_api/os_error.rs new file mode 100644 index 000000000..20bc483a7 --- /dev/null +++ b/crates/moonrun/src/async_api/os_error.rs @@ -0,0 +1,76 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_host::AsyncHostResult; +use crate::async_sys::os_error::stub; + +use super::context::{ImportArgs, callback_context, finish_bool}; + +pub(super) fn get_errno( + _scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + ret.set_int32(stub::get_errno(&context.host)); +} + +fn is_errno_predicate( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + predicate: impl FnOnce(i32) -> bool, +) -> AsyncHostResult { + let mut args = ImportArgs::new(scope, args); + let errno = args.i32(0)?; + Ok(predicate(errno)) +} + +macro_rules! errno_predicate { + ($callback:ident, $function:ident) => { + pub(super) fn $callback( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, + ) { + let context = callback_context(&args); + match is_errno_predicate(scope, &args, stub::$function) { + Ok(value) => finish_bool(&mut ret, value), + Err(error) => { + context.host.record_error(error); + finish_bool(&mut ret, false); + } + } + } + }; +} + +errno_predicate!(is_nonblocking_io_error, is_nonblocking_io_error); +errno_predicate!(is_eintr, is_eintr); +errno_predicate!(is_enoent, is_enoent); +errno_predicate!(is_eexist, is_eexist); +errno_predicate!(is_eacces, is_eacces); +errno_predicate!(is_econnrefused, is_econnrefused); +errno_predicate!(is_error_notify_enum_dir, is_error_notify_enum_dir); + +pub(super) fn get_enotdir( + _scope: &mut v8::HandleScope, + _args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + ret.set_int32(stub::get_enotdir()); +} diff --git a/crates/moonrun/src/async_api/process.rs b/crates/moonrun/src/async_api/process.rs new file mode 100644 index 000000000..2d6e60e9c --- /dev/null +++ b/crates/moonrun/src/async_api/process.rs @@ -0,0 +1,125 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_host::{AsyncHostError, AsyncHostResult, GuestMemory}; + +use super::context::{ + AsyncContext, ImportArgs, callback_context, throw_import_error, with_memory_mut, +}; + +pub(super) fn spawn_process( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + match spawn_process_impl(scope, &args, context) { + Ok(handle) => ret.set_int32(handle), + Err(error) => { + context.host.record_error(error); + ret.set_int32(-1); + } + } +} + +pub(super) fn make_wait_for_process_job( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + context.host.make_wait_for_process_job(args.i32(0)?) + })(); + match result { + Ok(handle) => ret.set_int32(handle), + Err(error) => throw_import_error(scope, "thread_pool/make_wait_for_process_job", error), + } +} + +fn spawn_process_impl( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, +) -> AsyncHostResult { + let mut args = ImportArgs::new(scope, args); + let argv_ptr = args.i32(0)?; + let argv_len = args.i32(1)?; + let argc = args.i32(2)?; + let stdin = args.i32(3)?; + let stdout = args.i32(4)?; + let stderr = args.i32(5)?; + + let (command, argv) = with_memory_mut(scope, context, |memory| { + let mut argv = read_packed_argv(memory, argv_ptr, argv_len, argc)?; + if argv.is_empty() { + return Err(AsyncHostError::Inval); + } + let command = argv.remove(0); + Ok((command, argv)) + })?; + + context + .host + .spawn_process(command, argv, stdin, stdout, stderr) +} + +fn read_packed_argv( + memory: &(impl GuestMemory + ?Sized), + ptr: i32, + len: i32, + argc: i32, +) -> AsyncHostResult> { + let bytes = memory.read_exact(ptr, len)?; + let argc = usize::try_from(argc).map_err(|_| AsyncHostError::Fault)?; + let mut argv = Vec::with_capacity(argc); + let mut offset = 0usize; + for _ in 0..argc { + let end = offset.checked_add(4).ok_or(AsyncHostError::Fault)?; + let len_bytes = bytes.get(offset..end).ok_or(AsyncHostError::Fault)?; + let arg_len = + u32::from_le_bytes(len_bytes.try_into().map_err(|_| AsyncHostError::Fault)?) as usize; + offset = end; + let end = offset.checked_add(arg_len).ok_or(AsyncHostError::Fault)?; + let arg_bytes = bytes.get(offset..end).ok_or(AsyncHostError::Fault)?; + let arg = std::str::from_utf8(arg_bytes).map_err(|_| AsyncHostError::Inval)?; + argv.push(arg.to_owned()); + offset = end; + } + if offset != bytes.len() { + return Err(AsyncHostError::Inval); + } + Ok(argv) +} + +#[cfg(test)] +mod tests { + use super::read_packed_argv; + + #[test] + fn read_packed_argv_decodes_len_prefixed_utf8_args() { + let mut bytes = Vec::new(); + for arg in ["moon", "build", "test_programs/lock_file"] { + bytes.extend_from_slice(&(arg.len() as u32).to_le_bytes()); + bytes.extend_from_slice(arg.as_bytes()); + } + let argv = read_packed_argv(bytes.as_slice(), 0, bytes.len() as i32, 3).unwrap(); + assert_eq!(argv, ["moon", "build", "test_programs/lock_file"]); + } +} diff --git a/crates/moonrun/src/async_api/registry.rs b/crates/moonrun/src/async_api/registry.rs new file mode 100644 index 000000000..754134aa1 --- /dev/null +++ b/crates/moonrun/src/async_api/registry.rs @@ -0,0 +1,898 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::v8_builder::ObjectExt; + +use super::{ + c_buffer, context::AsyncContext, env_util, event_loop, fd_util, fs, memory, os_error, process, + runtime, thread_pool, time, unsupported, +}; + +pub(crate) const MOONBIT_V0_MODULE: &str = "moonbit_v0"; +#[cfg(test)] +const NATIVE_ASYNC_PREFIX: &str = "moonbitlang_async_"; + +#[cfg(test)] +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum AsyncImportKind { + NativeMapped, + UnsupportedMvp, + WasmSupport, +} + +#[cfg(test)] +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum SourceRoot { + MoonbitAsync, + Moonrun, +} + +#[cfg(test)] +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +struct SourceLocation { + root: SourceRoot, + path: &'static str, +} + +#[cfg(test)] +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +struct AsyncImport { + kind: AsyncImportKind, + wasm_symbol: &'static str, + native_symbol: Option<&'static str>, + sources: &'static [SourceLocation], +} + +#[cfg(test)] +macro_rules! import_kind { + (native) => { + AsyncImportKind::NativeMapped + }; + (unsupported) => { + AsyncImportKind::UnsupportedMvp + }; + (support) => { + AsyncImportKind::WasmSupport + }; +} + +#[cfg(test)] +macro_rules! source_root { + (moonbit_async) => { + SourceRoot::MoonbitAsync + }; + (moonrun) => { + SourceRoot::Moonrun + }; +} + +macro_rules! declare_async_imports { + ($( + $kind:ident $callback:path => $wasm_symbol:literal, + native = $native_symbol:expr, + sources = [$($source_root:ident:$source_path:literal),+ $(,)?]; + )*) => { + #[cfg(test)] + const ASYNC_IMPORTS: &[AsyncImport] = &[ + $( + AsyncImport { + kind: import_kind!($kind), + wasm_symbol: $wasm_symbol, + native_symbol: $native_symbol, + sources: &[ + $( + SourceLocation { + root: source_root!($source_root), + path: $source_path, + }, + )+ + ], + }, + )* + ]; + + pub(super) fn register_imports<'s>( + obj: v8::Local<'s, v8::Object>, + scope: &mut v8::HandleScope<'s>, + context_ptr: *const AsyncContext, + ) { + $( + register_func_impl(obj, scope, $wasm_symbol, $callback, context_ptr); + )* + } + }; +} + +fn register_func_impl<'s>( + obj: v8::Local<'s, v8::Object>, + scope: &mut v8::HandleScope<'s>, + name: &str, + callback: impl v8::MapFnTo, + context_ptr: *const AsyncContext, +) { + let data = v8::External::new(scope, context_ptr as *mut std::ffi::c_void); + let function = v8::Function::builder(callback) + .data(data.into()) + .build(scope) + .unwrap(); + obj.set_value(scope, name, function.into()); +} + +// This block is the complete `moonbit_v0` ABI surface registered by moonrun. +// +// Entry shape: +// kind callback maps to "namespace/wasm_symbol", +// native = Some("moonbitlang_async_native_symbol") | None, +// sources = [moonbit_async:"path/in/async", moonrun:"path/in/moonrun"]; +// +// Kind legend: +// - native: maps a native async C-stub symbol to the same leaf name under a +// namespaced `moonbit_v0` field; tests require a `#[ported(...)]` +// implementation provenance. +// - support: wasm-only support import or host-control glue. It may list a +// native C symbol for provenance, but tests do not require a direct +// `#[ported(...)]` implementation. +// - unsupported: declared so wasm modules link, but currently returns the +// uniform unsupported stub. +declare_async_imports! { + // Runtime platform and worker control. + support runtime::exit => "runtime/exit", + native = None, + sources = [moonrun:"crates/moonrun/src/async_api/runtime.rs"]; + + support runtime::wait_for_event => "runtime/wait_for_event", + native = None, + sources = [moonrun:"crates/moonrun/src/async_host/event.rs"]; + + native event_loop::get_platform => "runtime/get_platform", + native = Some("moonbitlang_async_get_platform"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native event_loop::errno_is_cancelled => "thread_pool/errno_is_cancelled", + native = Some("moonbitlang_async_errno_is_cancelled"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::job_get_ret => "thread_pool/job_get_ret", + native = Some("moonbitlang_async_job_get_ret"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::job_get_err => "thread_pool/job_get_err", + native = Some("moonbitlang_async_job_get_err"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + support thread_pool::free_job => "thread_pool/free_job", + native = None, + sources = [moonrun:"crates/moonrun/src/async_api/thread_pool.rs"]; + + support thread_pool::run_job => "thread_pool/run_job", + native = None, + sources = [moonrun:"crates/moonrun/src/async_api/thread_pool.rs"]; + + native thread_pool::spawn_worker => "thread_pool/spawn_worker", + native = Some("moonbitlang_async_spawn_worker"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::free_worker => "thread_pool/free_worker", + native = Some("moonbitlang_async_free_worker"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::wake_worker => "thread_pool/wake_worker", + native = Some("moonbitlang_async_wake_worker"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::worker_enter_idle => "thread_pool/worker_enter_idle", + native = Some("moonbitlang_async_worker_enter_idle"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::cancel_worker => "thread_pool/cancel_worker", + native = Some("moonbitlang_async_cancel_worker"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + support thread_pool::fetch_completion => "thread_pool/fetch_completion", + native = Some("moonbitlang_async_fetch_completion"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::make_sleep_job => "thread_pool/make_sleep_job", + native = Some("moonbitlang_async_make_sleep_job"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + // Process entrypoints. `spawn_process` is wasm support glue for packed argv; + // wait still follows the native thread_pool.c job shape. + support process::spawn_process => "process/spawn_process", + native = None, + sources = [moonrun:"crates/moonrun/src/async_api/process.rs"]; + + native process::make_wait_for_process_job => "thread_pool/make_wait_for_process_job", + native = Some("moonbitlang_async_make_wait_for_process_job"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + // Time and guest-memory helpers. + native time::get_ms_since_epoch => "time/get_ms_since_epoch", + native = Some("moonbitlang_async_get_ms_since_epoch"), + sources = [moonbit_async:"src/internal/time/time.c"]; + + support time::sleep_ms => "time/sleep_ms", + native = None, + sources = [moonrun:"crates/moonrun/src/async_api/time.rs"]; + + support memory::copy_from_guest => "memory/copy_from_guest", + native = None, + sources = [moonrun:"crates/moonrun/src/async_host/mod.rs"]; + + support memory::zero_guest => "memory/zero_guest", + native = None, + sources = [moonrun:"crates/moonrun/src/async_host/mod.rs"]; + + // os_error/stub.c predicates and errno accessors. + native os_error::get_errno => "os_error/get_errno", + native = Some("moonbitlang_async_get_errno"), + sources = [moonbit_async:"src/os_error/stub.c"]; + + native os_error::is_nonblocking_io_error => "os_error/is_nonblocking_io_error", + native = Some("moonbitlang_async_is_nonblocking_io_error"), + sources = [moonbit_async:"src/os_error/stub.c"]; + + native os_error::is_eintr => "os_error/is_EINTR", + native = Some("moonbitlang_async_is_EINTR"), + sources = [moonbit_async:"src/os_error/stub.c"]; + + native os_error::is_enoent => "os_error/is_ENOENT", + native = Some("moonbitlang_async_is_ENOENT"), + sources = [moonbit_async:"src/os_error/stub.c"]; + + native os_error::is_eexist => "os_error/is_EEXIST", + native = Some("moonbitlang_async_is_EEXIST"), + sources = [moonbit_async:"src/os_error/stub.c"]; + + native os_error::is_eacces => "os_error/is_EACCES", + native = Some("moonbitlang_async_is_EACCES"), + sources = [moonbit_async:"src/os_error/stub.c"]; + + native os_error::is_econnrefused => "os_error/is_ECONNREFUSED", + native = Some("moonbitlang_async_is_ECONNREFUSED"), + sources = [moonbit_async:"src/os_error/stub.c"]; + + native os_error::is_error_notify_enum_dir => "os_error/is_ERROR_NOTIFY_ENUM_DIR", + native = Some("moonbitlang_async_is_ERROR_NOTIFY_ENUM_DIR"), + sources = [moonbit_async:"src/os_error/stub.c"]; + + native os_error::get_enotdir => "os_error/get_ENOTDIR", + native = Some("moonbitlang_async_get_ENOTDIR"), + sources = [moonbit_async:"src/os_error/stub.c"]; + + // internal/fd_util/stub.c. Raw-fd mutation helpers are present but + // unsupported because wasm async uses host handles first. + unsupported unsupported::i32 => "fd_util/get_invalid_handle", + native = Some("moonbitlang_async_get_invalid_handle"), + sources = [moonbit_async:"src/internal/fd_util/stub.c"]; + + native fs::close_fd => "fd_util/close_fd", + native = Some("moonbitlang_async_close_fd"), + sources = [moonbit_async:"src/internal/fd_util/stub.c"]; + + unsupported unsupported::i32 => "fd_util/fd_is_nonblocking", + native = Some("moonbitlang_async_fd_is_nonblocking"), + sources = [moonbit_async:"src/internal/fd_util/stub.c"]; + + unsupported unsupported::i32 => "fd_util/set_blocking", + native = Some("moonbitlang_async_set_blocking"), + sources = [moonbit_async:"src/internal/fd_util/stub.c"]; + + unsupported unsupported::i32 => "fd_util/set_nonblocking", + native = Some("moonbitlang_async_set_nonblocking"), + sources = [moonbit_async:"src/internal/fd_util/stub.c"]; + + unsupported unsupported::i32 => "fd_util/set_cloexec", + native = Some("moonbitlang_async_set_cloexec"), + sources = [moonbit_async:"src/internal/fd_util/stub.c"]; + + unsupported unsupported::i32 => "fd_util/create_named_pipe_server", + native = Some("moonbitlang_async_create_named_pipe_server"), + sources = [moonbit_async:"src/internal/fd_util/stub.c"]; + + unsupported unsupported::i32 => "fd_util/create_named_pipe_client", + native = Some("moonbitlang_async_create_named_pipe_client"), + sources = [moonbit_async:"src/internal/fd_util/stub.c"]; + + native fd_util::pipe => "fd_util/pipe", + native = Some("moonbitlang_async_pipe"), + sources = [moonbit_async:"src/internal/fd_util/stub.c"]; + + native fd_util::sizeof_file_time => "fd_util/sizeof_file_time", + native = Some("moonbitlang_async_sizeof_file_time"), + sources = [moonbit_async:"src/internal/fd_util/stub.c"]; + + native fd_util::get_atime_sec => "fd_util/get_atime_sec", + native = Some("moonbitlang_async_get_atime_sec"), + sources = [moonbit_async:"src/internal/fd_util/stub.c"]; + + native fd_util::get_atime_nsec => "fd_util/get_atime_nsec", + native = Some("moonbitlang_async_get_atime_nsec"), + sources = [moonbit_async:"src/internal/fd_util/stub.c"]; + + native fd_util::get_mtime_sec => "fd_util/get_mtime_sec", + native = Some("moonbitlang_async_get_mtime_sec"), + sources = [moonbit_async:"src/internal/fd_util/stub.c"]; + + native fd_util::get_mtime_nsec => "fd_util/get_mtime_nsec", + native = Some("moonbitlang_async_get_mtime_nsec"), + sources = [moonbit_async:"src/internal/fd_util/stub.c"]; + + native fd_util::get_ctime_sec => "fd_util/get_ctime_sec", + native = Some("moonbitlang_async_get_ctime_sec"), + sources = [moonbit_async:"src/internal/fd_util/stub.c"]; + + native fd_util::get_ctime_nsec => "fd_util/get_ctime_nsec", + native = Some("moonbitlang_async_get_ctime_nsec"), + sources = [moonbit_async:"src/internal/fd_util/stub.c"]; + + // Small internal utility stubs. + native env_util::getpid => "env_util/getpid", + native = Some("moonbitlang_async_getpid"), + sources = [moonbit_async:"src/internal/env_util/stub.c"]; + + native c_buffer::blit_to_c => "c_buffer/blit_to_c", + native = Some("moonbitlang_async_blit_to_c"), + sources = [moonbit_async:"src/internal/c_buffer/stub.c"]; + + native c_buffer::blit_from_c => "c_buffer/blit_from_c", + native = Some("moonbitlang_async_blit_from_c"), + sources = [moonbit_async:"src/internal/c_buffer/stub.c"]; + + native c_buffer::c_buffer_get => "c_buffer/c_buffer_get", + native = Some("moonbitlang_async_c_buffer_get"), + sources = [moonbit_async:"src/internal/c_buffer/stub.c"]; + + native c_buffer::strlen => "c_buffer/strlen", + native = Some("moonbitlang_async_strlen"), + sources = [moonbit_async:"src/internal/c_buffer/stub.c"]; + + native c_buffer::null_pointer => "c_buffer/null_pointer", + native = Some("moonbitlang_async_null_pointer"), + sources = [moonbit_async:"src/internal/c_buffer/stub.c"]; + + native c_buffer::pointer_is_null => "c_buffer/pointer_is_null", + native = Some("moonbitlang_async_pointer_is_null"), + sources = [moonbit_async:"src/internal/c_buffer/stub.c"]; + + unsupported unsupported::i32 => "os_string/c_buffer_as_string", + native = Some("moonbitlang_async_c_buffer_as_string"), + sources = [moonbit_async:"src/internal/os_string/stub.c"]; + + // fs/stub.c and fs/dir.c. + native fs::errno_is_lock_violation => "fs/errno_is_lock_violation", + native = Some("moonbitlang_async_errno_is_lock_violation"), + sources = [moonbit_async:"src/fs/stub.c"]; + + unsupported unsupported::i32 => "fs/dir_is_null", + native = Some("moonbitlang_async_dir_is_null"), + sources = [moonbit_async:"src/fs/stub.c"]; + + native fs::try_lock_file => "fs/try_lock_file", + native = Some("moonbitlang_async_try_lock_file"), + sources = [moonbit_async:"src/fs/stub.c"]; + + native fs::unlock_file => "fs/unlock_file", + native = Some("moonbitlang_async_unlock_file"), + sources = [moonbit_async:"src/fs/stub.c"]; + + // Returns the UTF-16 code-unit length that the guest must allocate for + // `fs/get_tmp_path`. + support fs::get_tmp_path_len => "fs/get_tmp_path_len", + native = None, + sources = [ + moonbit_async:"src/fs/stub.c", + moonrun:"crates/moonrun/src/async_api/fs.rs" + ]; + + // Writes the native temporary directory as UTF-16 code units into a + // guest-allocated MoonBit String. + native fs::get_tmp_path => "fs/get_tmp_path", + native = Some("moonbitlang_async_get_tmp_path"), + sources = [ + moonbit_async:"src/fs/stub.c", + moonrun:"crates/moonrun/src/async_sys/fs/stub.rs" + ]; + + native fs::dir_buffer_min_size => "fs/dir_buffer_min_size", + native = Some("moonbitlang_async_dir_buffer_min_size"), + sources = [moonbit_async:"src/fs/dir.c"]; + + native fs::dir_entry_length => "fs/dir_entry_length", + native = Some("moonbitlang_async_dir_entry_length"), + sources = [moonbit_async:"src/fs/dir.c"]; + + native fs::dir_entry_name_len => "fs/dir_entry_get_name_len", + native = Some("moonbitlang_async_dir_entry_get_name_len"), + sources = [moonbit_async:"src/fs/dir.c"]; + + native fs::dir_entry_name => "fs/dir_entry_get_name", + native = Some("moonbitlang_async_dir_entry_get_name"), + sources = [moonbit_async:"src/fs/dir.c"]; + + native fs::dir_entry_is_dir => "fs/dir_entry_is_dir", + native = Some("moonbitlang_async_dir_entry_is_dir"), + sources = [moonbit_async:"src/fs/dir.c"]; + + native fs::dir_entry_is_hidden => "fs/dir_entry_is_hidden", + native = Some("moonbitlang_async_dir_entry_is_hidden"), + sources = [moonbit_async:"src/fs/dir.c"]; + + native fs::dir_entry_file_id => "fs/dir_entry_get_file_id", + native = Some("moonbitlang_async_dir_entry_get_file_id"), + sources = [moonbit_async:"src/fs/dir.c"]; + + // Poller entrypoints are registered for link compatibility. The current + // wasm event loop slice uses worker completions and timers, not native + // epoll/kqueue/IOCP polling. + unsupported unsupported::i32 => "poll/poll_create", + native = Some("moonbitlang_async_poll_create"), + sources = [ + moonbit_async:"src/internal/event_loop/epoll.c", + moonbit_async:"src/internal/event_loop/kqueue.c", + moonbit_async:"src/internal/event_loop/iocp.c", + ]; + + unsupported unsupported::i32 => "poll/poll_destroy", + native = Some("moonbitlang_async_poll_destroy"), + sources = [ + moonbit_async:"src/internal/event_loop/epoll.c", + moonbit_async:"src/internal/event_loop/kqueue.c", + moonbit_async:"src/internal/event_loop/iocp.c", + ]; + + unsupported unsupported::i32 => "poll/poll_register", + native = Some("moonbitlang_async_poll_register"), + sources = [ + moonbit_async:"src/internal/event_loop/epoll.c", + moonbit_async:"src/internal/event_loop/kqueue.c", + moonbit_async:"src/internal/event_loop/iocp.c", + ]; + + unsupported unsupported::i32 => "poll/support_wait_pid_via_poll", + native = Some("moonbitlang_async_support_wait_pid_via_poll"), + sources = [ + moonbit_async:"src/internal/event_loop/epoll.c", + moonbit_async:"src/internal/event_loop/kqueue.c", + ]; + + unsupported unsupported::i32 => "poll/poll_register_pid", + native = Some("moonbitlang_async_poll_register_pid"), + sources = [ + moonbit_async:"src/internal/event_loop/epoll.c", + moonbit_async:"src/internal/event_loop/kqueue.c", + ]; + + unsupported unsupported::i32 => "poll/poll_remove", + native = Some("moonbitlang_async_poll_remove"), + sources = [ + moonbit_async:"src/internal/event_loop/epoll.c", + moonbit_async:"src/internal/event_loop/kqueue.c", + ]; + + unsupported unsupported::i32 => "poll/poll_remove_pid", + native = Some("moonbitlang_async_poll_remove_pid"), + sources = [ + moonbit_async:"src/internal/event_loop/epoll.c", + moonbit_async:"src/internal/event_loop/kqueue.c", + ]; + + unsupported unsupported::i32 => "poll/poll_wait", + native = Some("moonbitlang_async_poll_wait"), + sources = [ + moonbit_async:"src/internal/event_loop/epoll.c", + moonbit_async:"src/internal/event_loop/kqueue.c", + moonbit_async:"src/internal/event_loop/iocp.c", + ]; + + unsupported unsupported::i32 => "poll/event_list_get", + native = Some("moonbitlang_async_event_list_get"), + sources = [ + moonbit_async:"src/internal/event_loop/epoll.c", + moonbit_async:"src/internal/event_loop/kqueue.c", + moonbit_async:"src/internal/event_loop/iocp.c", + ]; + + unsupported unsupported::i32 => "poll/event_get_fd", + native = Some("moonbitlang_async_event_get_fd"), + sources = [ + moonbit_async:"src/internal/event_loop/epoll.c", + moonbit_async:"src/internal/event_loop/kqueue.c", + moonbit_async:"src/internal/event_loop/iocp.c", + ]; + + unsupported unsupported::i32 => "poll/event_get_events", + native = Some("moonbitlang_async_event_get_events"), + sources = [ + moonbit_async:"src/internal/event_loop/epoll.c", + moonbit_async:"src/internal/event_loop/kqueue.c", + ]; + + unsupported unsupported::i32 => "poll/event_get_io_result", + native = Some("moonbitlang_async_event_get_io_result"), + sources = [moonbit_async:"src/internal/event_loop/iocp.c"]; + + unsupported unsupported::i32 => "poll/event_get_bytes_transferred", + native = Some("moonbitlang_async_event_get_bytes_transferred"), + sources = [moonbit_async:"src/internal/event_loop/iocp.c"]; + + // Direct IO and Windows IO-result APIs are outside the current MVP. + unsupported unsupported::i32 => "io_windows/init_WSA", + native = Some("moonbitlang_async_init_WSA"), + sources = [moonbit_async:"src/internal/event_loop/io_windows.c"]; + + unsupported unsupported::i32 => "io_windows/cleanup_WSA", + native = Some("moonbitlang_async_cleanup_WSA"), + sources = [moonbit_async:"src/internal/event_loop/io_windows.c"]; + + unsupported unsupported::i32 => "io_windows/make_file_io_result", + native = Some("moonbitlang_async_make_file_io_result"), + sources = [moonbit_async:"src/internal/event_loop/io_windows.c"]; + + unsupported unsupported::i32 => "io_windows/make_socket_io_result", + native = Some("moonbitlang_async_make_socket_io_result"), + sources = [moonbit_async:"src/internal/event_loop/io_windows.c"]; + + unsupported unsupported::i32 => "io_windows/make_socket_with_addr_io_result", + native = Some("moonbitlang_async_make_socket_with_addr_io_result"), + sources = [moonbit_async:"src/internal/event_loop/io_windows.c"]; + + unsupported unsupported::i32 => "io_windows/make_connect_io_result", + native = Some("moonbitlang_async_make_connect_io_result"), + sources = [moonbit_async:"src/internal/event_loop/io_windows.c"]; + + unsupported unsupported::i32 => "io_windows/make_accept_io_result", + native = Some("moonbitlang_async_make_accept_io_result"), + sources = [moonbit_async:"src/internal/event_loop/io_windows.c"]; + + unsupported unsupported::i32 => "io_windows/make_read_dir_changes_io_result", + native = Some("moonbitlang_async_make_read_dir_changes_io_result"), + sources = [moonbit_async:"src/internal/event_loop/io_windows.c"]; + + unsupported unsupported::i32 => "io_windows/free_io_result", + native = Some("moonbitlang_async_free_io_result"), + sources = [moonbit_async:"src/internal/event_loop/io_windows.c"]; + + unsupported unsupported::i32 => "io_windows/io_result_get_job_id", + native = Some("moonbitlang_async_io_result_get_job_id"), + sources = [moonbit_async:"src/internal/event_loop/io_windows.c"]; + + unsupported unsupported::i32 => "io_windows/io_result_get_status", + native = Some("moonbitlang_async_io_result_get_status"), + sources = [moonbit_async:"src/internal/event_loop/io_windows.c"]; + + unsupported unsupported::i32 => "io_windows/cancel_io_result", + native = Some("moonbitlang_async_cancel_io_result"), + sources = [moonbit_async:"src/internal/event_loop/io_windows.c"]; + + unsupported unsupported::i32 => "io_windows/errno_is_read_EOF", + native = Some("moonbitlang_async_errno_is_read_EOF"), + sources = [moonbit_async:"src/internal/event_loop/io_windows.c"]; + + unsupported unsupported::i32 => "io/read", + native = Some("moonbitlang_async_read"), + sources = [ + moonbit_async:"src/internal/event_loop/io_unix.c", + moonbit_async:"src/internal/event_loop/io_windows.c", + ]; + + unsupported unsupported::i32 => "io/write", + native = Some("moonbitlang_async_write"), + sources = [ + moonbit_async:"src/internal/event_loop/io_unix.c", + moonbit_async:"src/internal/event_loop/io_windows.c", + ]; + + unsupported unsupported::i32 => "io/connect", + native = Some("moonbitlang_async_connect"), + sources = [ + moonbit_async:"src/internal/event_loop/io_unix.c", + moonbit_async:"src/internal/event_loop/io_windows.c", + ]; + + unsupported unsupported::i32 => "io_windows/setup_connected_socket", + native = Some("moonbitlang_async_setup_connected_socket"), + sources = [moonbit_async:"src/internal/event_loop/io_windows.c"]; + + unsupported unsupported::i32 => "io/accept", + native = Some("moonbitlang_async_accept"), + sources = [ + moonbit_async:"src/internal/event_loop/io_unix.c", + moonbit_async:"src/internal/event_loop/io_windows.c", + ]; + + unsupported unsupported::i32 => "io_windows/setup_accepted_socket", + native = Some("moonbitlang_async_setup_accepted_socket"), + sources = [moonbit_async:"src/internal/event_loop/io_windows.c"]; + + unsupported unsupported::i32 => "io_windows/get_std_handle", + native = Some("moonbitlang_async_get_std_handle"), + sources = [moonbit_async:"src/internal/event_loop/io_windows.c"]; + + unsupported unsupported::i32 => "io_windows/read_dir_changes", + native = Some("moonbitlang_async_read_dir_changes"), + sources = [moonbit_async:"src/internal/event_loop/io_windows.c"]; + + unsupported unsupported::i32 => "thread_pool/init_thread_pool", + native = Some("moonbitlang_async_init_thread_pool"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + // thread_pool.c FS jobs. Path-taking jobs use the Guest String Path ABI: + // MoonBit String pointer plus UTF-16 code-unit length. + native thread_pool::make_open_job => "thread_pool/make_open_job", + native = Some("moonbitlang_async_make_open_job"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::open_job_get_fd => "thread_pool/open_job_get_fd", + native = Some("moonbitlang_async_open_job_get_fd"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::open_job_get_kind => "thread_pool/open_job_get_kind", + native = Some("moonbitlang_async_open_job_get_kind"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::open_job_get_dev_id => "thread_pool/open_job_get_dev_id", + native = Some("moonbitlang_async_open_job_get_dev_id"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::open_job_get_file_id => "thread_pool/open_job_get_file_id", + native = Some("moonbitlang_async_open_job_get_file_id"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::make_read_job => "thread_pool/make_read_job", + native = Some("moonbitlang_async_make_read_job"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::make_write_job => "thread_pool/make_write_job", + native = Some("moonbitlang_async_make_write_job"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::make_file_kind_by_path_job => "thread_pool/make_file_kind_by_path_job", + native = Some("moonbitlang_async_make_file_kind_by_path_job"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::make_file_size_job => "thread_pool/make_file_size_job", + native = Some("moonbitlang_async_make_file_size_job"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::get_file_size_result => "thread_pool/get_file_size_result", + native = Some("moonbitlang_async_get_file_size_result"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::make_file_time_job => "thread_pool/make_file_time_job", + native = Some("moonbitlang_async_make_file_time_job"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::make_file_time_by_path_job => "thread_pool/make_file_time_by_path_job", + native = Some("moonbitlang_async_make_file_time_by_path_job"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::make_access_job => "thread_pool/make_access_job", + native = Some("moonbitlang_async_make_access_job"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::make_chmod_job => "thread_pool/make_chmod_job", + native = Some("moonbitlang_async_make_chmod_job"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::make_fsync_job => "thread_pool/make_fsync_job", + native = Some("moonbitlang_async_make_fsync_job"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::make_flock_job => "thread_pool/make_flock_job", + native = Some("moonbitlang_async_make_flock_job"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::make_remove_job => "thread_pool/make_remove_job", + native = Some("moonbitlang_async_make_remove_job"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::make_rename_job => "thread_pool/make_rename_job", + native = Some("moonbitlang_async_make_rename_job"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::make_symlink_job => "thread_pool/make_symlink_job", + native = Some("moonbitlang_async_make_symlink_job"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::make_mkdir_job => "thread_pool/make_mkdir_job", + native = Some("moonbitlang_async_make_mkdir_job"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::make_rmdir_job => "thread_pool/make_rmdir_job", + native = Some("moonbitlang_async_make_rmdir_job"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + native thread_pool::make_readdir_job => "thread_pool/make_readdir_job", + native = Some("moonbitlang_async_make_readdir_job"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + // Sockets, spawn jobs, and TLS remain deferred; they are registered as + // unsupported imports so wasm modules fail at the operation boundary rather + // than at instantiation. + unsupported unsupported::i32 => "socket/make_tcp_socket", + native = Some("moonbitlang_async_make_tcp_socket"), + sources = [moonbit_async:"src/socket/socket.c"]; + + unsupported unsupported::i32 => "socket/make_udp_socket", + native = Some("moonbitlang_async_make_udp_socket"), + sources = [moonbit_async:"src/socket/socket.c"]; + + unsupported unsupported::i32 => "thread_pool/make_spawn_job", + native = Some("moonbitlang_async_make_spawn_job"), + sources = [moonbit_async:"src/internal/event_loop/thread_pool.c"]; + + unsupported unsupported::i32 => "tls/schannel_new", + native = Some("moonbitlang_async_schannel_new"), + sources = [moonbit_async:"src/tls/schannel.c"]; + + unsupported unsupported::i32 => "tls/tls_client_ctx", + native = Some("moonbitlang_async_tls_client_ctx"), + sources = [moonbit_async:"src/tls/openssl.c"]; +} + +#[cfg(test)] +mod tests { + use std::{collections::BTreeSet, fs, path::Path}; + + use super::*; + + fn repo_root() -> &'static Path { + Path::new(env!("CARGO_MANIFEST_DIR")) + .parent() + .and_then(Path::parent) + .expect("moonrun crate must live under crates/moonrun") + } + + fn source_path(source: SourceLocation) -> std::path::PathBuf { + match source.root { + SourceRoot::MoonbitAsync => repo_root() + .join("third_party/moonbitlang_async") + .join(source.path), + SourceRoot::Moonrun => repo_root().join(source.path), + } + } + + #[test] + fn wasm_import_names_are_unique() { + let mut seen = BTreeSet::new(); + for import in ASYNC_IMPORTS { + assert!( + seen.insert(import.wasm_symbol), + "duplicate async import {}", + import.wasm_symbol + ); + } + } + + #[test] + fn runtime_exit_is_part_of_moonbit_v0() { + assert!( + ASYNC_IMPORTS.iter().any(|import| { + import.kind == AsyncImportKind::WasmSupport + && import.wasm_symbol == "runtime/exit" + && import.native_symbol.is_none() + }), + "async wasm integration must not depend on older runtime namespaces for exit" + ); + } + + #[test] + fn wasm_import_names_are_namespaced_and_keep_native_leaf_names() { + for import in ASYNC_IMPORTS { + let Some((namespace, leaf)) = import.wasm_symbol.split_once('/') else { + panic!("async import {} must be namespaced", import.wasm_symbol); + }; + assert!( + !namespace.is_empty(), + "empty namespace for {}", + import.wasm_symbol + ); + assert!( + !leaf.is_empty(), + "empty leaf name for {}", + import.wasm_symbol + ); + assert!( + !leaf.contains('/'), + "async import {} must use exactly one namespace separator", + import.wasm_symbol + ); + + let Some(native_symbol) = import.native_symbol else { + assert_eq!(import.kind, AsyncImportKind::WasmSupport); + continue; + }; + let suffix = native_symbol + .strip_prefix(NATIVE_ASYNC_PREFIX) + .expect("native async mapping must use the async C namespace"); + assert_eq!(leaf, suffix); + assert!(!leaf.starts_with("async_")); + } + } + + #[test] + fn declared_sources_exist_and_contain_native_symbols() { + for import in ASYNC_IMPORTS { + assert!( + !import.sources.is_empty(), + "async import {} must declare source files", + import.wasm_symbol + ); + for source in import.sources { + let source_path = source_path(*source); + let contents = fs::read_to_string(&source_path) + .unwrap_or_else(|error| panic!("failed to read {:?}: {error}", source_path)); + if let Some(native_symbol) = import.native_symbol { + assert!( + contents.contains(native_symbol), + "{:?} does not contain native symbol {} for wasm import {}", + source_path, + native_symbol, + import.wasm_symbol + ); + } + } + } + } + + #[test] + fn native_mapped_imports_have_ported_implementations() { + let ported_symbols = crate::async_sys::ported_symbols(); + + for import in ASYNC_IMPORTS { + if import.kind != AsyncImportKind::NativeMapped { + continue; + } + + let native_symbol = import + .native_symbol + .expect("native-mapped import must declare a native symbol"); + assert!( + import.sources.iter().any(|source| { + source.root == SourceRoot::MoonbitAsync + && ported_symbols.iter().any(|ported| { + ported.native_symbol == native_symbol && ported.source == source.path + }) + }), + "async import {} / {} has no Rust port origin", + import.wasm_symbol, + native_symbol + ); + } + } + + #[test] + fn ported_implementations_are_registered_imports() { + for ported in crate::async_sys::ported_symbols() { + assert!( + ASYNC_IMPORTS.iter().any(|import| { + import.native_symbol == Some(ported.native_symbol) + && import.sources.iter().any(|source| { + source.root == SourceRoot::MoonbitAsync && source.path == ported.source + }) + }), + "ported symbol {}::{} from {} / {} is not registered", + ported.rust_module, + ported.rust_symbol, + ported.source, + ported.native_symbol + ); + } + } +} diff --git a/crates/moonrun/src/async_api/runtime.rs b/crates/moonrun/src/async_api/runtime.rs new file mode 100644 index 000000000..29fb86383 --- /dev/null +++ b/crates/moonrun/src/async_api/runtime.rs @@ -0,0 +1,47 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use super::context::{ImportArgs, callback_context, finish_errno, throw_import_error}; + +pub(super) fn exit( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + _ret: v8::ReturnValue, +) { + let code = { + let mut args = ImportArgs::new(scope, &args); + args.i32(0) + }; + match code { + Ok(code) => std::process::exit(code), + Err(error) => throw_import_error(scope, "runtime/exit", error), + } +} + +pub(super) fn wait_for_event( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + context.host.wait_for_event(args.i32(0)?) + })(); + finish_errno(context, &mut ret, result); +} diff --git a/crates/moonrun/src/async_api/socket.rs b/crates/moonrun/src/async_api/socket.rs new file mode 100644 index 000000000..74d735b7b --- /dev/null +++ b/crates/moonrun/src/async_api/socket.rs @@ -0,0 +1,19 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +// Socket support is outside the first async wasm boundary slice. diff --git a/crates/moonrun/src/async_api/thread_pool.rs b/crates/moonrun/src/async_api/thread_pool.rs new file mode 100644 index 000000000..e6a23f831 --- /dev/null +++ b/crates/moonrun/src/async_api/thread_pool.rs @@ -0,0 +1,912 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use std::ffi::OsString; + +use crate::async_host::{AsyncHostError, AsyncHostResult, checked_range}; +use crate::async_sys::internal::event_loop::thread_pool; + +use super::context::{ + AsyncContext, ImportArgs, callback_context, throw_import_error, with_memory_mut, +}; + +pub(super) fn free_job( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + context.host.free_job(args.i32(0)?) + })(); + if let Err(error) = result { + throw_import_error(scope, "thread_pool/free_job", error); + return; + } + ret.set_undefined(); +} + +pub(super) fn job_get_ret( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + context.host.job_get_ret(args.i32(0)?) + })(); + match result { + Ok(value) => ret.set_int32(value as i32), + Err(error) => throw_import_error(scope, "thread_pool/job_get_ret", error), + } +} + +pub(super) fn job_get_err( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + context.host.job_get_err(args.i32(0)?) + })(); + match result { + Ok(value) => ret.set_int32(value), + Err(error) => throw_import_error(scope, "thread_pool/job_get_err", error), + } +} + +pub(super) fn run_job( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = run_job_impl(scope, &args, context); + if let Err(error) = result { + throw_import_error(scope, "thread_pool/run_job", error); + return; + } + ret.set_undefined(); +} + +pub(super) fn spawn_worker( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + context.host.spawn_worker(args.i32(0)?, args.i32(1)?) + })(); + match result { + Ok(handle) => ret.set_int32(handle), + Err(error) => throw_import_error(scope, "thread_pool/spawn_worker", error), + } +} + +pub(super) fn free_worker( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + context.host.free_worker(args.i32(0)?) + })(); + if let Err(error) = result { + throw_import_error(scope, "thread_pool/free_worker", error); + return; + } + ret.set_undefined(); +} + +pub(super) fn wake_worker( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + context + .host + .wake_worker(args.i32(0)?, args.i32(1)?, args.i32(2)?) + })(); + if let Err(error) = result { + throw_import_error(scope, "thread_pool/wake_worker", error); + return; + } + ret.set_undefined(); +} + +pub(super) fn worker_enter_idle( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + context.host.worker_enter_idle(args.i32(0)?) + })(); + if let Err(error) = result { + throw_import_error(scope, "thread_pool/worker_enter_idle", error); + return; + } + ret.set_undefined(); +} + +pub(super) fn cancel_worker( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + context.host.cancel_worker(args.i32(0)?) + })(); + match result { + Ok(status) => ret.set_int32(status), + Err(error) => throw_import_error(scope, "thread_pool/cancel_worker", error), + } +} + +pub(super) fn fetch_completion( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = fetch_completion_impl(scope, &args, context); + match result { + Ok(bytes) => ret.set_int32(bytes), + Err(error) => throw_import_error(scope, "thread_pool/fetch_completion", error), + } +} + +pub(super) fn make_sleep_job( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + context + .host + .insert_job(thread_pool::make_sleep_job(args.i32(0)?)) + })(); + match result { + Ok(handle) => ret.set_int32(handle), + Err(error) => throw_import_error(scope, "thread_pool/make_sleep_job", error), + } +} + +pub(super) fn make_open_job( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = make_open_job_impl(scope, &args, context); + match result { + Ok(handle) => ret.set_int32(handle), + Err(error) => throw_import_error(scope, "thread_pool/make_open_job", error), + } +} + +pub(super) fn make_read_job( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + let fd = args.i32(0)?; + let ptr = args.i32(1)?; + let offset = args.i32(2)?; + let len = args.i32(3)?; + let position = args.i64(4)?; + context + .host + .insert_job(thread_pool::make_read_job(fd, ptr, offset, len, position)) + })(); + match result { + Ok(handle) => ret.set_int32(handle), + Err(error) => throw_import_error(scope, "thread_pool/make_read_job", error), + } +} + +pub(super) fn make_write_job( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = make_write_job_impl(scope, &args, context); + match result { + Ok(handle) => ret.set_int32(handle), + Err(error) => throw_import_error(scope, "thread_pool/make_write_job", error), + } +} + +pub(super) fn make_file_kind_by_path_job( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = make_file_kind_by_path_job_impl(scope, &args, context); + match result { + Ok(handle) => ret.set_int32(handle), + Err(error) => throw_import_error(scope, "thread_pool/make_file_kind_by_path_job", error), + } +} + +pub(super) fn make_file_size_job( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + context + .host + .insert_job(thread_pool::make_file_size_job(args.i32(0)?)) + })(); + match result { + Ok(handle) => ret.set_int32(handle), + Err(error) => throw_import_error(scope, "thread_pool/make_file_size_job", error), + } +} + +pub(super) fn get_file_size_result( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + context.host.get_file_size_result(args.i32(0)?) + })(); + match result { + Ok(value) => ret.set(v8::BigInt::new_from_i64(scope, value).into()), + Err(error) => throw_import_error(scope, "thread_pool/get_file_size_result", error), + } +} + +pub(super) fn make_file_time_job( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + let fd = args.i32(0)?; + let out = args.i32(1)?; + let out_len = args.i32(2)?; + context + .host + .insert_job(thread_pool::make_file_time_job(fd, out, out_len)) + })(); + match result { + Ok(handle) => ret.set_int32(handle), + Err(error) => throw_import_error(scope, "thread_pool/make_file_time_job", error), + } +} + +pub(super) fn make_file_time_by_path_job( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = make_file_time_by_path_job_impl(scope, &args, context); + match result { + Ok(handle) => ret.set_int32(handle), + Err(error) => throw_import_error(scope, "thread_pool/make_file_time_by_path_job", error), + } +} + +pub(super) fn make_access_job( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = make_access_job_impl(scope, &args, context); + match result { + Ok(handle) => ret.set_int32(handle), + Err(error) => throw_import_error(scope, "thread_pool/make_access_job", error), + } +} + +pub(super) fn make_chmod_job( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = make_chmod_job_impl(scope, &args, context); + match result { + Ok(handle) => ret.set_int32(handle), + Err(error) => throw_import_error(scope, "thread_pool/make_chmod_job", error), + } +} + +pub(super) fn make_fsync_job( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + let fd = args.i32(0)?; + let only_data = args.i32(1)? != 0; + context + .host + .insert_job(thread_pool::make_fsync_job(fd, only_data)) + })(); + match result { + Ok(handle) => ret.set_int32(handle), + Err(error) => throw_import_error(scope, "thread_pool/make_fsync_job", error), + } +} + +pub(super) fn make_flock_job( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + let fd = args.i32(0)?; + let exclusive = args.i32(1)? != 0; + context + .host + .insert_job(thread_pool::make_flock_job(fd, exclusive)) + })(); + match result { + Ok(handle) => ret.set_int32(handle), + Err(error) => throw_import_error(scope, "thread_pool/make_flock_job", error), + } +} + +pub(super) fn make_remove_job( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = make_remove_job_impl(scope, &args, context); + match result { + Ok(handle) => ret.set_int32(handle), + Err(error) => throw_import_error(scope, "thread_pool/make_remove_job", error), + } +} + +pub(super) fn make_rename_job( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = make_rename_job_impl(scope, &args, context); + match result { + Ok(handle) => ret.set_int32(handle), + Err(error) => throw_import_error(scope, "thread_pool/make_rename_job", error), + } +} + +pub(super) fn make_symlink_job( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = make_symlink_job_impl(scope, &args, context); + match result { + Ok(handle) => ret.set_int32(handle), + Err(error) => throw_import_error(scope, "thread_pool/make_symlink_job", error), + } +} + +pub(super) fn make_mkdir_job( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = make_mkdir_job_impl(scope, &args, context); + match result { + Ok(handle) => ret.set_int32(handle), + Err(error) => throw_import_error(scope, "thread_pool/make_mkdir_job", error), + } +} + +pub(super) fn make_rmdir_job( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = make_rmdir_job_impl(scope, &args, context); + match result { + Ok(handle) => ret.set_int32(handle), + Err(error) => throw_import_error(scope, "thread_pool/make_rmdir_job", error), + } +} + +pub(super) fn make_readdir_job( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + let dir = args.i32(0)?; + let buf = args.i32(1)?; + let len = args.i32(2)?; + let restart = args.i32(3)? != 0; + context + .host + .insert_job(thread_pool::make_readdir_job(dir, buf, len, restart)) + })(); + match result { + Ok(handle) => ret.set_int32(handle), + Err(error) => throw_import_error(scope, "thread_pool/make_readdir_job", error), + } +} + +pub(super) fn open_job_get_fd( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + match open_job_i32(scope, &args, context, |handle| { + context.host.open_job_get_fd(handle) + }) { + Ok(value) => ret.set_int32(value), + Err(error) => throw_import_error(scope, "thread_pool/open_job_get_fd", error), + } +} + +pub(super) fn open_job_get_kind( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + match open_job_i32(scope, &args, context, |handle| { + context.host.open_job_get_kind(handle) + }) { + Ok(value) => ret.set_int32(value), + Err(error) => throw_import_error(scope, "thread_pool/open_job_get_kind", error), + } +} + +pub(super) fn open_job_get_dev_id( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + match open_job_u64(scope, &args, context, |handle| { + context.host.open_job_get_dev_id(handle) + }) { + Ok(value) => ret.set(v8::BigInt::new_from_u64(scope, value).into()), + Err(error) => throw_import_error(scope, "thread_pool/open_job_get_dev_id", error), + } +} + +pub(super) fn open_job_get_file_id( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + match open_job_u64(scope, &args, context, |handle| { + context.host.open_job_get_file_id(handle) + }) { + Ok(value) => ret.set(v8::BigInt::new_from_u64(scope, value).into()), + Err(error) => throw_import_error(scope, "thread_pool/open_job_get_file_id", error), + } +} + +fn run_job_impl( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, +) -> AsyncHostResult<()> { + let mut args = ImportArgs::new(scope, args); + let job = args.i32(0)?; + with_memory_mut(scope, context, |memory| context.host.run_job(memory, job)) +} + +fn fetch_completion_impl( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, +) -> AsyncHostResult { + let mut args = ImportArgs::new(scope, args); + let dst = args.i32(0)?; + let max_jobs = args.i32(1)?; + with_memory_mut(scope, context, |memory| { + context.host.fetch_completion(memory, dst, max_jobs) + }) +} + +fn make_open_job_impl( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, +) -> AsyncHostResult { + let mut args = ImportArgs::new(scope, args); + let path_ptr = args.i32(0)?; + let path_len = args.i32(1)?; + let access = args.i32(2)?; + let create_mode = args.i32(3)?; + let append = args.i32(4)? != 0; + let sync = args.i32(5)?; + let mode = args.i32(6)?; + + let filename = read_guest_path(scope, context, path_ptr, path_len)?; + + context.host.insert_job(thread_pool::make_open_job( + filename, + access, + create_mode, + append, + sync, + mode, + )) +} + +fn make_write_job_impl( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, +) -> AsyncHostResult { + let mut args = ImportArgs::new(scope, args); + let fd = args.i32(0)?; + let ptr = args.i32(1)?; + let offset = args.i32(2)?; + let len = args.i32(3)?; + let position = args.i64(4)?; + let offset_ptr = ptr.checked_add(offset).ok_or(AsyncHostError::Fault)?; + let data = with_memory_mut(scope, context, |memory| { + Ok(checked_range(memory, offset_ptr, len)?.to_vec()) + })?; + + context + .host + .insert_job(thread_pool::make_write_job(fd, data, position)) +} + +fn make_file_kind_by_path_job_impl( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, +) -> AsyncHostResult { + let mut args = ImportArgs::new(scope, args); + let parent = args.i32(0)?; + let path_ptr = args.i32(1)?; + let path_len = args.i32(2)?; + let follow_symlink = args.i32(3)? != 0; + let path = read_guest_path(scope, context, path_ptr, path_len)?; + + context + .host + .insert_job(thread_pool::make_file_kind_by_path_job( + parent, + path, + follow_symlink, + )) +} + +fn make_file_time_by_path_job_impl( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, +) -> AsyncHostResult { + let mut args = ImportArgs::new(scope, args); + let path_ptr = args.i32(0)?; + let path_len = args.i32(1)?; + let out = args.i32(2)?; + let out_len = args.i32(3)?; + let follow_symlink = args.i32(4)? != 0; + let path = read_guest_path(scope, context, path_ptr, path_len)?; + + context + .host + .insert_job(thread_pool::make_file_time_by_path_job( + path, + out, + out_len, + follow_symlink, + )) +} + +fn make_access_job_impl( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, +) -> AsyncHostResult { + let mut args = ImportArgs::new(scope, args); + let path_ptr = args.i32(0)?; + let path_len = args.i32(1)?; + let access = args.i32(2)?; + let path = read_guest_path(scope, context, path_ptr, path_len)?; + + context + .host + .insert_job(thread_pool::make_access_job(path, access)) +} + +fn make_chmod_job_impl( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, +) -> AsyncHostResult { + let mut args = ImportArgs::new(scope, args); + let path_ptr = args.i32(0)?; + let path_len = args.i32(1)?; + let mode = args.i32(2)?; + let path = read_guest_path(scope, context, path_ptr, path_len)?; + + context + .host + .insert_job(thread_pool::make_chmod_job(path, mode)) +} + +fn make_remove_job_impl( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, +) -> AsyncHostResult { + let mut args = ImportArgs::new(scope, args); + let path_ptr = args.i32(0)?; + let path_len = args.i32(1)?; + let path = read_guest_path(scope, context, path_ptr, path_len)?; + + context.host.insert_job(thread_pool::make_remove_job(path)) +} + +fn make_rename_job_impl( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, +) -> AsyncHostResult { + let mut args = ImportArgs::new(scope, args); + let old_path_ptr = args.i32(0)?; + let old_path_len = args.i32(1)?; + let new_path_ptr = args.i32(2)?; + let new_path_len = args.i32(3)?; + let replace = args.i32(4)? != 0; + let old_path = read_guest_path(scope, context, old_path_ptr, old_path_len)?; + let new_path = read_guest_path(scope, context, new_path_ptr, new_path_len)?; + + context + .host + .insert_job(thread_pool::make_rename_job(old_path, new_path, replace)) +} + +fn make_symlink_job_impl( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, +) -> AsyncHostResult { + let mut args = ImportArgs::new(scope, args); + let target_ptr = args.i32(0)?; + let target_len = args.i32(1)?; + let path_ptr = args.i32(2)?; + let path_len = args.i32(3)?; + let force_symlink = args.i32(4)? != 0; + let target = read_guest_path(scope, context, target_ptr, target_len)?; + let path = read_guest_path(scope, context, path_ptr, path_len)?; + + context + .host + .insert_job(thread_pool::make_symlink_job(target, path, force_symlink)) +} + +fn make_mkdir_job_impl( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, +) -> AsyncHostResult { + let mut args = ImportArgs::new(scope, args); + let path_ptr = args.i32(0)?; + let path_len = args.i32(1)?; + let mode = args.i32(2)?; + let path = read_guest_path(scope, context, path_ptr, path_len)?; + + context + .host + .insert_job(thread_pool::make_mkdir_job(path, mode)) +} + +fn make_rmdir_job_impl( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + context: &AsyncContext, +) -> AsyncHostResult { + let mut args = ImportArgs::new(scope, args); + let path_ptr = args.i32(0)?; + let path_len = args.i32(1)?; + let path = read_guest_path(scope, context, path_ptr, path_len)?; + + context.host.insert_job(thread_pool::make_rmdir_job(path)) +} + +fn read_guest_path( + scope: &mut v8::HandleScope, + context: &AsyncContext, + ptr: i32, + len: i32, +) -> AsyncHostResult { + // Async path imports pass MoonBit String data, so `len` is UTF-16 code + // units. Do not treat this as UTF-8 bytes or a native C string. + let byte_len = len.checked_mul(2).ok_or(AsyncHostError::Fault)?; + with_memory_mut(scope, context, |memory| { + let bytes = checked_range(memory, ptr, byte_len)?; + decode_guest_path(bytes) + }) +} + +fn decode_guest_path(bytes: &[u8]) -> AsyncHostResult { + if !bytes.len().is_multiple_of(2) { + return Err(AsyncHostError::Inval); + } + let units = utf16_units_from_guest_bytes(bytes); + os_string_from_utf16_path(&units) +} + +fn utf16_units_from_guest_bytes(bytes: &[u8]) -> Vec { + bytes + .chunks_exact(2) + .map(|unit| u16::from_le_bytes([unit[0], unit[1]])) + .collect() +} + +#[cfg(unix)] +fn os_string_from_utf16_path(units: &[u16]) -> AsyncHostResult { + use std::os::unix::ffi::OsStringExt; + + let path = String::from_utf16(units).map_err(|_| AsyncHostError::Inval)?; + Ok(OsString::from_vec(path.into_bytes())) +} + +#[cfg(windows)] +fn os_string_from_utf16_path(units: &[u16]) -> AsyncHostResult { + use std::os::windows::ffi::OsStringExt; + + Ok(OsString::from_wide(units)) +} + +fn open_job_i32( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + _context: &AsyncContext, + f: impl FnOnce(i32) -> AsyncHostResult, +) -> AsyncHostResult { + let mut args = ImportArgs::new(scope, args); + f(args.i32(0)?) +} + +fn open_job_u64( + scope: &mut v8::HandleScope, + args: &v8::FunctionCallbackArguments, + _context: &AsyncContext, + f: impl FnOnce(i32) -> AsyncHostResult, +) -> AsyncHostResult { + let mut args = ImportArgs::new(scope, args); + f(args.i32(0)?) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[cfg(unix)] + #[test] + fn guest_path_decodes_utf16_to_unix_bytes() { + use std::os::unix::ffi::OsStrExt; + + let bytes = guest_string_bytes("async-fs-smoke-\u{6587}.txt"); + let path = decode_guest_path(&bytes).unwrap(); + + assert_eq!( + path.as_os_str().as_bytes(), + "async-fs-smoke-\u{6587}.txt".as_bytes() + ); + } + + #[cfg(windows)] + #[test] + fn guest_path_decodes_utf16_on_windows() { + use std::os::windows::ffi::OsStrExt; + + let bytes = guest_string_bytes("async-fs-smoke-\u{6587}.txt"); + let path = decode_guest_path(&bytes).unwrap(); + + assert_eq!( + path.as_os_str().encode_wide().collect::>(), + "async-fs-smoke-\u{6587}.txt" + .encode_utf16() + .collect::>() + ); + } + + #[test] + fn guest_path_decodes_utf16_code_units_not_utf8_bytes() { + let bytes = guest_string_bytes("a\u{1f600}.txt"); + + assert_eq!( + utf16_units_from_guest_bytes(&bytes), + vec![0x0061, 0xd83d, 0xde00, 0x002e, 0x0074, 0x0078, 0x0074] + ); + } + + #[test] + fn guest_path_rejects_odd_utf16_byte_count() { + assert!(matches!( + decode_guest_path(&[0x61]), + Err(AsyncHostError::Inval) + )); + } + + #[cfg(windows)] + #[test] + fn guest_path_length_is_utf16_code_units_on_windows() { + let bytes = guest_string_bytes("a.txt"); + let path = decode_guest_path(&bytes).unwrap(); + + assert_eq!(path, OsString::from("a.txt")); + } + + #[cfg(unix)] + #[test] + fn guest_path_rejects_invalid_utf16_on_unix() { + assert!(matches!( + decode_guest_path(&[0x00, 0xd8]), + Err(AsyncHostError::Inval) + )); + } + + fn guest_string_bytes(path: &str) -> Vec { + path.encode_utf16() + .flat_map(u16::to_le_bytes) + .collect::>() + } +} diff --git a/crates/moonrun/src/async_api/time.rs b/crates/moonrun/src/async_api/time.rs new file mode 100644 index 000000000..4435b9294 --- /dev/null +++ b/crates/moonrun/src/async_api/time.rs @@ -0,0 +1,81 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +#[cfg(unix)] +use crate::async_host::AsyncHostError; +use crate::async_host::AsyncHostResult; +use crate::async_sys::internal::time::clock; + +use super::context::{ImportArgs, callback_context, finish_errno}; + +pub(super) fn get_ms_since_epoch( + scope: &mut v8::HandleScope, + _args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let value = v8::BigInt::new_from_i64(scope, clock::get_ms_since_epoch()); + ret.set(value.into()); +} + +pub(super) fn sleep_ms( + scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + let result = (|| { + let mut args = ImportArgs::new(scope, &args); + sleep_ms_impl(args.i32(0)?) + })(); + finish_errno(context, &mut ret, result); +} + +fn sleep_ms_impl(duration_ms: i32) -> AsyncHostResult<()> { + if duration_ms <= 0 { + return Ok(()); + } + sleep_ms_sys(duration_ms) +} + +#[cfg(unix)] +fn sleep_ms_sys(duration_ms: i32) -> AsyncHostResult<()> { + if unsafe { libc::poll(std::ptr::null_mut(), 0, duration_ms) } < 0 { + return Err(AsyncHostError::Native(errno())); + } + Ok(()) +} + +#[cfg(unix)] +fn errno() -> i32 { + #[cfg(target_os = "linux")] + { + unsafe { *libc::__errno_location() } + } + #[cfg(target_os = "macos")] + { + unsafe { *libc::__error() } + } +} + +#[cfg(windows)] +fn sleep_ms_sys(duration_ms: i32) -> AsyncHostResult<()> { + unsafe { + windows_sys::Win32::System::Threading::Sleep(duration_ms as u32); + } + Ok(()) +} diff --git a/crates/moonrun/src/async_api/tls.rs b/crates/moonrun/src/async_api/tls.rs new file mode 100644 index 000000000..021594256 --- /dev/null +++ b/crates/moonrun/src/async_api/tls.rs @@ -0,0 +1,19 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +// TLS support is outside the first async wasm boundary slice. diff --git a/crates/moonrun/src/async_api/unsupported.rs b/crates/moonrun/src/async_api/unsupported.rs new file mode 100644 index 000000000..d6079b19a --- /dev/null +++ b/crates/moonrun/src/async_api/unsupported.rs @@ -0,0 +1,28 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use super::context::callback_context; + +pub(super) fn i32( + _scope: &mut v8::HandleScope, + args: v8::FunctionCallbackArguments, + mut ret: v8::ReturnValue, +) { + let context = callback_context(&args); + ret.set_int32(context.host.unsupported_return()); +} diff --git a/crates/moonrun/src/async_host/event.rs b/crates/moonrun/src/async_host/event.rs new file mode 100644 index 000000000..3f7bd5803 --- /dev/null +++ b/crates/moonrun/src/async_host/event.rs @@ -0,0 +1,221 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_host::{AsyncHostError, AsyncHostResult}; + +#[cfg(unix)] +mod sys { + use super::{AsyncHostError, AsyncHostResult}; + + pub(crate) struct HostEvent { + read_fd: i32, + write_fd: i32, + } + + impl HostEvent { + pub(crate) fn new() -> AsyncHostResult { + let mut fds = [0; 2]; + if unsafe { libc::pipe(fds.as_mut_ptr()) } != 0 { + return Err(last_error()); + } + + if let Err(error) = set_nonblocking(fds[0]).and_then(|()| set_nonblocking(fds[1])) { + unsafe { + libc::close(fds[0]); + libc::close(fds[1]); + } + return Err(error); + } + + Ok(Self { + read_fd: fds[0], + write_fd: fds[1], + }) + } + + pub(crate) fn notify(&self) -> AsyncHostResult<()> { + let byte = [1_u8]; + loop { + let n = unsafe { libc::write(self.write_fd, byte.as_ptr().cast(), byte.len()) }; + if n == 1 { + return Ok(()); + } + let error = errno(); + if error == libc::EINTR { + continue; + } + if would_block(error) { + return Ok(()); + } + return Err(AsyncHostError::Native(error)); + } + } + + pub(crate) fn clear(&self) -> AsyncHostResult<()> { + let mut buf = [0_u8; 64]; + loop { + let n = unsafe { libc::read(self.read_fd, buf.as_mut_ptr().cast(), buf.len()) }; + if n > 0 { + continue; + } + if n == 0 { + return Ok(()); + } + let error = errno(); + if error == libc::EINTR { + continue; + } + if would_block(error) { + return Ok(()); + } + return Err(AsyncHostError::Native(error)); + } + } + + pub(crate) fn wait(&self, timeout_ms: i32) -> AsyncHostResult<()> { + let mut pollfd = libc::pollfd { + fd: self.read_fd, + events: libc::POLLIN, + revents: 0, + }; + loop { + let n = unsafe { libc::poll(&mut pollfd, 1, timeout_ms) }; + if n >= 0 { + return Ok(()); + } + let error = errno(); + if error == libc::EINTR { + continue; + } + return Err(AsyncHostError::Native(error)); + } + } + } + + impl Drop for HostEvent { + fn drop(&mut self) { + unsafe { + libc::close(self.read_fd); + libc::close(self.write_fd); + } + } + } + + fn set_nonblocking(fd: i32) -> AsyncHostResult<()> { + let flags = unsafe { libc::fcntl(fd, libc::F_GETFL) }; + if flags < 0 { + return Err(last_error()); + } + if unsafe { libc::fcntl(fd, libc::F_SETFL, flags | libc::O_NONBLOCK) } < 0 { + return Err(last_error()); + } + Ok(()) + } + + fn last_error() -> AsyncHostError { + AsyncHostError::Native(errno()) + } + + fn errno() -> i32 { + #[cfg(target_os = "linux")] + { + unsafe { *libc::__errno_location() } + } + #[cfg(target_os = "macos")] + { + unsafe { *libc::__error() } + } + } + + fn would_block(error: i32) -> bool { + error == libc::EAGAIN || error == libc::EWOULDBLOCK + } +} + +#[cfg(windows)] +mod sys { + use std::ptr::null; + + use super::{AsyncHostError, AsyncHostResult}; + use windows_sys::Win32::Foundation::{ + CloseHandle, GetLastError, HANDLE, WAIT_FAILED, WAIT_OBJECT_0, WAIT_TIMEOUT, + }; + use windows_sys::Win32::System::Threading::{ + CreateEventW, INFINITE, ResetEvent, SetEvent, WaitForSingleObject, + }; + + pub(crate) struct HostEvent { + handle: HANDLE, + } + + // Win32 event handles are process-wide kernel objects and are safe to wait + // and signal from multiple host worker threads. + unsafe impl Send for HostEvent {} + unsafe impl Sync for HostEvent {} + + impl HostEvent { + pub(crate) fn new() -> AsyncHostResult { + let handle = unsafe { CreateEventW(null(), 1, 0, null()) }; + if handle.is_null() { + return Err(last_error()); + } + Ok(Self { handle }) + } + + pub(crate) fn notify(&self) -> AsyncHostResult<()> { + if unsafe { SetEvent(self.handle) } == 0 { + return Err(last_error()); + } + Ok(()) + } + + pub(crate) fn clear(&self) -> AsyncHostResult<()> { + if unsafe { ResetEvent(self.handle) } == 0 { + return Err(last_error()); + } + Ok(()) + } + + pub(crate) fn wait(&self, timeout_ms: i32) -> AsyncHostResult<()> { + let timeout = if timeout_ms < 0 { + INFINITE + } else { + timeout_ms as u32 + }; + match unsafe { WaitForSingleObject(self.handle, timeout) } { + WAIT_OBJECT_0 | WAIT_TIMEOUT => Ok(()), + WAIT_FAILED => Err(last_error()), + _ => Err(last_error()), + } + } + } + + impl Drop for HostEvent { + fn drop(&mut self) { + unsafe { + CloseHandle(self.handle); + } + } + } + + fn last_error() -> AsyncHostError { + AsyncHostError::Native(unsafe { GetLastError() } as i32) + } +} + +pub(crate) use sys::HostEvent; diff --git a/crates/moonrun/src/async_host/mod.rs b/crates/moonrun/src/async_host/mod.rs new file mode 100644 index 000000000..d48bfe831 --- /dev/null +++ b/crates/moonrun/src/async_host/mod.rs @@ -0,0 +1,809 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use std::collections::VecDeque; +use std::fs::File; +use std::sync::{Arc, Mutex}; + +mod event; + +use crate::async_sys::internal::event_loop::thread_pool::{ + self, HostFile, HostFileTable, HostProcess, HostProcessTable, HostWorkerHandle, HostWorkerJob, + Job, +}; + +#[cfg(not(any(target_os = "linux", target_os = "macos", windows)))] +compile_error!("moonrun async wasm host currently supports only Linux, macOS, and Windows hosts"); + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum AsyncHostError { + Fault, + Inval, + Badf, + NotSupported, + Native(i32), +} + +pub(crate) type AsyncHostResult = Result; + +#[cfg(unix)] +mod native_errno { + pub(crate) const BADF: i32 = libc::EBADF; + pub(crate) const FAULT: i32 = libc::EFAULT; + pub(crate) const INVAL: i32 = libc::EINVAL; + pub(crate) const NOT_SUPPORTED: i32 = libc::ENOSYS; +} + +#[cfg(windows)] +mod native_errno { + use windows_sys::Win32::Foundation::{ + ERROR_CALL_NOT_IMPLEMENTED, ERROR_INVALID_ADDRESS, ERROR_INVALID_HANDLE, + ERROR_INVALID_PARAMETER, + }; + + pub(crate) const BADF: i32 = ERROR_INVALID_HANDLE as i32; + pub(crate) const FAULT: i32 = ERROR_INVALID_ADDRESS as i32; + pub(crate) const INVAL: i32 = ERROR_INVALID_PARAMETER as i32; + pub(crate) const NOT_SUPPORTED: i32 = ERROR_CALL_NOT_IMPLEMENTED as i32; +} + +impl AsyncHostError { + pub(crate) fn errno(self) -> i32 { + match self { + Self::Fault => native_errno::FAULT, + Self::Inval => native_errno::INVAL, + Self::Badf => native_errno::BADF, + Self::NotSupported => native_errno::NOT_SUPPORTED, + Self::Native(errno) => errno, + } + } +} + +pub(crate) trait GuestMemory { + fn bytes(&self) -> &[u8]; + + fn bytes_mut(&mut self) -> &mut [u8]; + + fn read_exact(&self, offset: i32, len: i32) -> AsyncHostResult<&[u8]> { + let (offset, end) = guest_bounds(offset, len)?; + self.bytes().get(offset..end).ok_or(AsyncHostError::Fault) + } + + fn read_exact_mut(&mut self, offset: i32, len: i32) -> AsyncHostResult<&mut [u8]> { + let (offset, end) = guest_bounds(offset, len)?; + self.bytes_mut() + .get_mut(offset..end) + .ok_or(AsyncHostError::Fault) + } + + fn write_exact(&mut self, offset: i32, data: &[u8]) -> AsyncHostResult<()> { + let len = i32::try_from(data.len()).map_err(|_| AsyncHostError::Fault)?; + let dst = self.read_exact_mut(offset, len)?; + dst.copy_from_slice(data); + Ok(()) + } + + fn write_with_capacity( + &mut self, + offset: i32, + capacity: i32, + data: &[u8], + ) -> AsyncHostResult<()> { + let data_len = i32::try_from(data.len()).map_err(|_| AsyncHostError::Fault)?; + if data_len > capacity { + return Err(AsyncHostError::Fault); + } + let dst = self.read_exact_mut(offset, capacity)?; + dst[..data.len()].copy_from_slice(data); + Ok(()) + } + + fn fill_exact(&mut self, offset: i32, len: i32, value: u8) -> AsyncHostResult<()> { + let dst = self.read_exact_mut(offset, len)?; + dst.fill(value); + Ok(()) + } + + fn write_i32_le(&mut self, offset: i32, value: i32) -> AsyncHostResult<()> { + self.write_exact(offset, &value.to_le_bytes()) + } +} + +fn guest_bounds(offset: i32, len: i32) -> AsyncHostResult<(usize, usize)> { + let offset = usize::try_from(offset).map_err(|_| AsyncHostError::Fault)?; + let len = usize::try_from(len).map_err(|_| AsyncHostError::Fault)?; + let end = offset.checked_add(len).ok_or(AsyncHostError::Fault)?; + Ok((offset, end)) +} + +impl GuestMemory for [u8] { + fn bytes(&self) -> &[u8] { + self + } + + fn bytes_mut(&mut self) -> &mut [u8] { + self + } +} + +impl GuestMemory for [u8; N] { + fn bytes(&self) -> &[u8] { + self.as_slice() + } + + fn bytes_mut(&mut self) -> &mut [u8] { + self.as_mut_slice() + } +} + +#[derive(Debug)] +struct HandleTable { + slots: Vec>, +} + +#[derive(Debug)] +struct HandleSlot { + generation: u16, + value: Option, + reserved: bool, +} + +// Guest handles are signed i32 values, so the high generation bit must stay +// clear for every handle returned to MoonBit. +const MAX_HANDLE_GENERATION: u16 = 0x7fff; + +impl Default for HandleTable { + fn default() -> Self { + Self { slots: Vec::new() } + } +} + +impl HandleTable { + fn insert(&mut self, value: T) -> AsyncHostResult { + if let Some((index, slot)) = self + .slots + .iter_mut() + .enumerate() + .find(|(_, slot)| slot.value.is_none() && !slot.reserved) + { + slot.value = Some(value); + return encode_handle(index, slot.generation); + } + + let index = self.slots.len(); + self.slots.push(HandleSlot { + generation: 1, + value: Some(value), + reserved: false, + }); + encode_handle(index, 1) + } + + fn get(&self, handle: i32) -> AsyncHostResult<&T> { + let (index, generation) = decode_handle(handle)?; + let slot = self.slots.get(index).ok_or(AsyncHostError::Badf)?; + if slot.generation != generation { + return Err(AsyncHostError::Badf); + } + slot.value.as_ref().ok_or(AsyncHostError::Badf) + } + + fn get_mut(&mut self, handle: i32) -> AsyncHostResult<&mut T> { + let (index, generation) = decode_handle(handle)?; + let slot = self.slots.get_mut(index).ok_or(AsyncHostError::Badf)?; + if slot.generation != generation { + return Err(AsyncHostError::Badf); + } + slot.value.as_mut().ok_or(AsyncHostError::Badf) + } + + fn remove(&mut self, handle: i32) -> AsyncHostResult { + let (index, generation) = decode_handle(handle)?; + let slot = self.slots.get_mut(index).ok_or(AsyncHostError::Badf)?; + if slot.generation != generation { + return Err(AsyncHostError::Badf); + } + let value = slot.value.take().ok_or(AsyncHostError::Badf)?; + slot.reserved = false; + slot.generation = next_generation(slot.generation); + Ok(value) + } + + fn discard(&mut self, handle: i32) -> AsyncHostResult<()> { + let (index, generation) = decode_handle(handle)?; + let slot = self.slots.get_mut(index).ok_or(AsyncHostError::Badf)?; + if slot.generation != generation { + return Err(AsyncHostError::Badf); + } + // Worker jobs are removed from the table while the worker owns them. + // Cancellation may still free the guest handle immediately; in that + // case the worker drops the job when it later finishes. + if slot.value.is_none() && !slot.reserved { + return Err(AsyncHostError::Badf); + } + slot.value = None; + slot.reserved = false; + slot.generation = next_generation(slot.generation); + Ok(()) + } + + fn take(&mut self, handle: i32) -> AsyncHostResult { + let (index, generation) = decode_handle(handle)?; + let slot = self.slots.get_mut(index).ok_or(AsyncHostError::Badf)?; + if slot.generation != generation { + return Err(AsyncHostError::Badf); + } + let value = slot.value.take().ok_or(AsyncHostError::Badf)?; + slot.reserved = true; + Ok(value) + } + + fn put(&mut self, handle: i32, value: T) -> AsyncHostResult<()> { + let (index, generation) = decode_handle(handle)?; + let slot = self.slots.get_mut(index).ok_or(AsyncHostError::Badf)?; + if slot.generation != generation || slot.value.is_some() || !slot.reserved { + return Err(AsyncHostError::Badf); + } + slot.value = Some(value); + slot.reserved = false; + Ok(()) + } +} + +impl HostFileTable for HandleTable { + fn insert_file(&mut self, file: File) -> AsyncHostResult { + self.insert(HostFile::new(file)) + } + + fn with_file_mut( + &mut self, + handle: i32, + f: impl FnOnce(&mut File) -> AsyncHostResult, + ) -> AsyncHostResult { + f(self.get_mut(handle)?.file_mut()) + } + + fn with_host_file_mut( + &mut self, + handle: i32, + f: impl FnOnce(&mut HostFile) -> AsyncHostResult, + ) -> AsyncHostResult { + f(self.get_mut(handle)?) + } +} + +impl HostProcessTable for HandleTable { + fn insert_process(&mut self, process: HostProcess) -> AsyncHostResult { + self.insert(process) + } + + fn take_process(&mut self, handle: i32) -> AsyncHostResult { + self.remove(handle) + } +} + +fn encode_handle(index: usize, generation: u16) -> AsyncHostResult { + if index >= 0x1_0000 { + return Err(AsyncHostError::Fault); + } + if generation == 0 || generation > MAX_HANDLE_GENERATION { + return Err(AsyncHostError::Fault); + } + Ok(((i32::from(generation)) << 16) | i32::try_from(index).unwrap()) +} + +fn decode_handle(handle: i32) -> AsyncHostResult<(usize, u16)> { + if handle <= 0 { + return Err(AsyncHostError::Badf); + } + let index = (handle as u32 & 0xffff) as usize; + let generation = ((handle as u32 >> 16) & 0xffff) as u16; + if generation == 0 { + return Err(AsyncHostError::Badf); + } + Ok((index, generation)) +} + +fn next_generation(generation: u16) -> u16 { + match generation.checked_add(1) { + Some(next) if next <= MAX_HANDLE_GENERATION => next, + _ => 1, + } +} + +#[derive(Default)] +struct AsyncHostState { + errno: i32, + jobs: HandleTable, + files: HandleTable, + processes: HandleTable, + workers: HandleTable, + completions: VecDeque, +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +struct HostCompletion { + job_id: i32, + job_handle: i32, +} + +pub(crate) struct AsyncHost { + state: Arc>, + event: Arc, +} + +impl Default for AsyncHost { + fn default() -> Self { + Self { + state: Arc::new(Mutex::new(AsyncHostState::default())), + event: Arc::new(event::HostEvent::new().expect("failed to create async host event")), + } + } +} + +impl AsyncHost { + pub(crate) fn get_errno(&self) -> i32 { + self.state.lock().unwrap().errno + } + + pub(crate) fn set_errno(&self, errno: i32) { + self.state.lock().unwrap().errno = errno; + } + + pub(crate) fn record_error(&self, error: AsyncHostError) -> i32 { + let errno = error.errno(); + self.set_errno(errno); + errno + } + + pub(crate) fn unsupported_return(&self) -> i32 { + self.record_error(AsyncHostError::NotSupported); + -1 + } + + pub(crate) fn copy_from_guest_len( + &self, + memory: &(impl GuestMemory + ?Sized), + offset: i32, + len: i32, + ) -> AsyncHostResult { + let len = memory.read_exact(offset, len)?.len(); + i32::try_from(len).map_err(|_| AsyncHostError::Fault) + } + + pub(crate) fn zero_guest( + &self, + memory: &mut (impl GuestMemory + ?Sized), + offset: i32, + len: i32, + ) -> AsyncHostResult<()> { + memory.fill_exact(offset, len, 0) + } + + pub(crate) fn insert_job(&self, job: Job) -> AsyncHostResult { + self.state.lock().unwrap().jobs.insert(job) + } + + pub(crate) fn free_job(&self, handle: i32) -> AsyncHostResult<()> { + self.state.lock().unwrap().jobs.discard(handle)?; + Ok(()) + } + + pub(crate) fn job_get_ret(&self, handle: i32) -> AsyncHostResult { + Ok( + crate::async_sys::internal::event_loop::thread_pool::job_get_ret( + self.state.lock().unwrap().jobs.get(handle)?, + ), + ) + } + + pub(crate) fn job_get_err(&self, handle: i32) -> AsyncHostResult { + Ok( + crate::async_sys::internal::event_loop::thread_pool::job_get_err( + self.state.lock().unwrap().jobs.get(handle)?, + ), + ) + } + + pub(crate) fn open_job_get_fd(&self, handle: i32) -> AsyncHostResult { + let state = self.state.lock().unwrap(); + let result = thread_pool::open_job_result(state.jobs.get(handle)?)?; + Ok(thread_pool::open_job_get_fd(result)) + } + + pub(crate) fn open_job_get_kind(&self, handle: i32) -> AsyncHostResult { + let state = self.state.lock().unwrap(); + let result = thread_pool::open_job_result(state.jobs.get(handle)?)?; + Ok(thread_pool::open_job_get_kind(result)) + } + + pub(crate) fn open_job_get_dev_id(&self, handle: i32) -> AsyncHostResult { + let state = self.state.lock().unwrap(); + let result = thread_pool::open_job_result(state.jobs.get(handle)?)?; + Ok(thread_pool::open_job_get_dev_id(result)) + } + + pub(crate) fn open_job_get_file_id(&self, handle: i32) -> AsyncHostResult { + let state = self.state.lock().unwrap(); + let result = thread_pool::open_job_result(state.jobs.get(handle)?)?; + Ok(thread_pool::open_job_get_file_id(result)) + } + + pub(crate) fn get_file_size_result(&self, handle: i32) -> AsyncHostResult { + crate::async_sys::internal::event_loop::thread_pool::get_file_size_result( + self.state.lock().unwrap().jobs.get(handle)?, + ) + } + + pub(crate) fn close_fd(&self, handle: i32) -> AsyncHostResult<()> { + self.state.lock().unwrap().files.remove(handle)?; + Ok(()) + } + + pub(crate) fn pipe(&self) -> AsyncHostResult<[i32; 2]> { + crate::async_sys::internal::fd_util::stub::pipe_host_files( + &mut self.state.lock().unwrap().files, + ) + } + + pub(crate) fn try_lock_file(&self, handle: i32, exclusive: bool) -> AsyncHostResult<()> { + let mut state = self.state.lock().unwrap(); + crate::async_sys::fs::stub::try_lock_host_file(state.files.get_mut(handle)?, exclusive) + } + + pub(crate) fn unlock_file(&self, handle: i32) -> AsyncHostResult<()> { + let mut state = self.state.lock().unwrap(); + crate::async_sys::fs::stub::unlock_host_file(state.files.get_mut(handle)?) + } + + pub(crate) fn spawn_process( + &self, + command: String, + args: Vec, + stdin: i32, + stdout: i32, + stderr: i32, + ) -> AsyncHostResult { + let mut state = self.state.lock().unwrap(); + let AsyncHostState { + files, processes, .. + } = &mut *state; + thread_pool::spawn_process(files, processes, command, args, stdin, stdout, stderr) + } + + pub(crate) fn make_wait_for_process_job(&self, process: i32) -> AsyncHostResult { + let mut state = self.state.lock().unwrap(); + let job = + thread_pool::make_wait_for_process_job_from_handle(&mut state.processes, process)?; + state.jobs.insert(job) + } + + pub(crate) fn run_job( + &self, + memory: &mut (impl GuestMemory + ?Sized), + handle: i32, + ) -> AsyncHostResult<()> { + let mut job = self.state.lock().unwrap().jobs.take(handle)?; + let mut files = SharedFileTable { + state: Arc::clone(&self.state), + }; + thread_pool::run_host_job(&mut job, &mut files); + thread_pool::complete_guest_job(&mut job, memory)?; + self.state.lock().unwrap().jobs.put(handle, job)?; + Ok(()) + } + + pub(crate) fn spawn_worker(&self, job_id: i32, job_handle: i32) -> AsyncHostResult { + let worker = self.spawn_worker_thread(HostWorkerJob { job_id, job_handle }); + self.state.lock().unwrap().workers.insert(worker) + } + + pub(crate) fn wake_worker( + &self, + worker_handle: i32, + job_id: i32, + job_handle: i32, + ) -> AsyncHostResult<()> { + let state = self.state.lock().unwrap(); + let worker = state.workers.get(worker_handle)?; + thread_pool::wake_worker(worker, HostWorkerJob { job_id, job_handle }); + Ok(()) + } + + pub(crate) fn worker_enter_idle(&self, worker_handle: i32) -> AsyncHostResult<()> { + let state = self.state.lock().unwrap(); + let worker = state.workers.get(worker_handle)?; + thread_pool::worker_enter_idle(worker); + Ok(()) + } + + pub(crate) fn free_worker(&self, worker_handle: i32) -> AsyncHostResult<()> { + let worker = self.state.lock().unwrap().workers.remove(worker_handle)?; + thread_pool::free_worker(worker); + Ok(()) + } + + pub(crate) fn cancel_worker(&self, worker_handle: i32) -> AsyncHostResult { + let state = self.state.lock().unwrap(); + let worker = state.workers.get(worker_handle)?; + Ok(thread_pool::cancel_worker(worker)) + } + + pub(crate) fn fetch_completion( + &self, + memory: &mut (impl GuestMemory + ?Sized), + dst: i32, + max_jobs: i32, + ) -> AsyncHostResult { + let max_jobs = usize::try_from(max_jobs).map_err(|_| AsyncHostError::Fault)?; + let mut state = self.state.lock().unwrap(); + let n = max_jobs.min(state.completions.len()); + if n == 0 { + return Ok(0); + } + let bytes = n + .checked_mul(std::mem::size_of::()) + .ok_or(AsyncHostError::Fault)?; + let bytes_i32 = i32::try_from(bytes).map_err(|_| AsyncHostError::Fault)?; + memory.read_exact(dst, bytes_i32)?; + + let completions = state + .completions + .iter() + .take(n) + .copied() + .collect::>(); + for completion in &completions { + if let Ok(job) = state.jobs.get_mut(completion.job_handle) { + thread_pool::complete_guest_job(job, memory)?; + } + } + for (index, completion) in completions.into_iter().enumerate() { + let removed = state.completions.pop_front().ok_or(AsyncHostError::Inval)?; + debug_assert_eq!(removed, completion); + let offset = dst + .checked_add(i32::try_from(index * 4).map_err(|_| AsyncHostError::Fault)?) + .ok_or(AsyncHostError::Fault)?; + memory.write_i32_le(offset, completion.job_id)?; + } + Ok(bytes_i32) + } + + pub(crate) fn wait_for_event(&self, timeout_ms: i32) -> AsyncHostResult<()> { + if timeout_ms == 0 { + return Ok(()); + } + if !self.state.lock().unwrap().completions.is_empty() { + return Ok(()); + } + self.event.clear()?; + if !self.state.lock().unwrap().completions.is_empty() { + return Ok(()); + } + self.event.wait(timeout_ms)?; + self.event.clear() + } + + fn spawn_worker_thread(&self, init_job: HostWorkerJob) -> HostWorkerHandle { + let state = Arc::clone(&self.state); + let run_state = Arc::clone(&state); + let event = Arc::clone(&self.event); + thread_pool::spawn_worker( + init_job, + move |worker_job| { + let Ok(mut job) = run_state.lock().unwrap().jobs.take(worker_job.job_handle) else { + return; + }; + + let mut files = SharedFileTable { + state: Arc::clone(&run_state), + }; + thread_pool::run_host_job(&mut job, &mut files); + + let mut state = run_state.lock().unwrap(); + let _ = state.jobs.put(worker_job.job_handle, job); + }, + move |worker_job| { + // Even if cancellation discarded the job handle, the event loop + // still needs the completion to move the worker out of running. + state.lock().unwrap().completions.push_back(HostCompletion { + job_id: worker_job.job_id, + job_handle: worker_job.job_handle, + }); + event.notify().expect("failed to notify async host event"); + }, + ) + } +} + +struct SharedFileTable { + state: Arc>, +} + +impl HostFileTable for SharedFileTable { + fn insert_file(&mut self, file: File) -> AsyncHostResult { + self.state.lock().unwrap().files.insert(HostFile::new(file)) + } + + fn with_file_mut( + &mut self, + handle: i32, + f: impl FnOnce(&mut File) -> AsyncHostResult, + ) -> AsyncHostResult { + let mut file = { + let mut state = self.state.lock().unwrap(); + state + .files + .get_mut(handle)? + .file_mut() + .try_clone() + .map_err(native_io_error)? + }; + f(&mut file) + } + + fn with_host_file_mut( + &mut self, + handle: i32, + f: impl FnOnce(&mut HostFile) -> AsyncHostResult, + ) -> AsyncHostResult { + let mut state = self.state.lock().unwrap(); + f(state.files.get_mut(handle)?) + } +} + +fn native_io_error(error: std::io::Error) -> AsyncHostError { + AsyncHostError::Native( + error + .raw_os_error() + .unwrap_or_else(|| AsyncHostError::Inval.errno()), + ) +} + +pub(crate) fn checked_range(memory: &[u8], offset: i32, len: i32) -> AsyncHostResult<&[u8]> { + memory.read_exact(offset, len) +} + +pub(crate) fn checked_mut_range( + memory: &mut [u8], + offset: i32, + len: i32, +) -> AsyncHostResult<&mut [u8]> { + memory.read_exact_mut(offset, len) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn checked_range_accepts_in_bounds_access() { + let memory = [1, 2, 3, 4]; + + assert_eq!(checked_range(&memory, 1, 2).unwrap(), &[2, 3]); + assert!(checked_range(&memory, 4, 0).unwrap().is_empty()); + } + + #[test] + fn checked_range_rejects_out_of_bounds_access() { + let memory = [0; 4]; + + for (offset, len) in [(-1, 1), (0, -1), (3, 2), (i32::MAX, 1), (2, i32::MAX)] { + assert_eq!( + checked_range(&memory, offset, len), + Err(AsyncHostError::Fault) + ); + } + } + + #[test] + fn checked_mut_range_accepts_in_bounds_access() { + let mut memory = [1, 2, 3, 4]; + + checked_mut_range(&mut memory, 1, 2).unwrap().fill(9); + + assert_eq!(memory, [1, 9, 9, 4]); + } + + #[test] + fn checked_mut_range_rejects_out_of_bounds_access() { + let mut memory = [0; 4]; + + for (offset, len) in [(-1, 1), (0, -1), (3, 2), (i32::MAX, 1), (2, i32::MAX)] { + assert_eq!( + checked_mut_range(&mut memory, offset, len), + Err(AsyncHostError::Fault) + ); + } + } + + #[test] + fn guest_memory_writes_fixed_little_endian_words() { + let mut memory = [0; 16]; + + memory.write_i32_le(2, 0x1020_3040).unwrap(); + + assert_eq!(&memory[2..6], &[0x40, 0x30, 0x20, 0x10]); + assert_eq!(memory.write_i32_le(14, 1), Err(AsyncHostError::Fault)); + } + + #[test] + fn fetch_completion_copies_job_output_before_publishing_job_id() { + let host = AsyncHost::default(); + let job = thread_pool::make_read_job(0, 8, 0, 3, -1); + let job_handle = host.insert_job(job).unwrap(); + { + let mut state = host.state.lock().unwrap(); + let job = state.jobs.get_mut(job_handle).unwrap(); + let thread_pool::JobPayload::Read { result, .. } = job.payload_mut() else { + panic!("expected read job"); + }; + *result = Some(b"abc".to_vec()); + state.completions.push_back(HostCompletion { + job_id: 42, + job_handle, + }); + } + + let mut memory = vec![0; 16]; + let bytes = host.fetch_completion(memory.as_mut_slice(), 0, 1).unwrap(); + + assert_eq!(bytes, 4); + assert_eq!(i32::from_le_bytes(memory[0..4].try_into().unwrap()), 42); + assert_eq!(&memory[8..11], b"abc"); + assert!(host.state.lock().unwrap().completions.is_empty()); + } + + #[test] + fn process_spawn_and_wait_uses_host_process_table() { + let host = AsyncHost::default(); + let process = host + .spawn_process("true".to_string(), Vec::new(), -1, -1, -1) + .unwrap(); + let job = host.make_wait_for_process_job(process).unwrap(); + let mut memory = []; + + host.run_job(&mut memory, job).unwrap(); + + assert_eq!(host.job_get_ret(job).unwrap(), 0); + } + + #[test] + fn table_reuse_keeps_handles_positive_after_generation_wrap() { + let mut table = HandleTable::default(); + let mut handle = table.insert(1).unwrap(); + + for _ in 0..=MAX_HANDLE_GENERATION { + assert!(handle > 0); + assert_eq!(table.remove(handle), Ok(1)); + handle = table.insert(1).unwrap(); + } + + assert!(handle > 0); + let (_, generation) = decode_handle(handle).unwrap(); + assert!((1..=MAX_HANDLE_GENERATION).contains(&generation)); + } + + #[test] + fn unsupported_records_native_errno() { + let host = AsyncHost::default(); + + assert_eq!(host.unsupported_return(), -1); + assert_eq!(host.get_errno(), AsyncHostError::NotSupported.errno()); + } +} diff --git a/crates/moonrun/src/async_sys/fs/dir.rs b/crates/moonrun/src/async_sys/fs/dir.rs new file mode 100644 index 000000000..556bf409e --- /dev/null +++ b/crates/moonrun/src/async_sys/fs/dir.rs @@ -0,0 +1,169 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_host::{AsyncHostError, AsyncHostResult}; +use crate::async_sys::ported_fns; + +pub(crate) const HEADER_LEN: usize = 24; +pub(crate) const BUFFER_MIN_SIZE: usize = 1024; + +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) struct EntryRecord { + pub(crate) name: Vec, + pub(crate) is_dir: i32, + pub(crate) is_hidden: bool, + pub(crate) file_id: u64, +} + +impl EntryRecord { + pub(crate) fn encoded_len(&self) -> AsyncHostResult { + aligned_record_len(self.name.len()) + } + + pub(crate) fn encode_into(&self, dst: &mut Vec) -> AsyncHostResult<()> { + let len = self.encoded_len()?; + let len_u32 = u32::try_from(len).map_err(|_| AsyncHostError::Fault)?; + let name_len = u32::try_from(self.name.len()).map_err(|_| AsyncHostError::Fault)?; + + dst.extend_from_slice(&len_u32.to_le_bytes()); + dst.extend_from_slice(&name_len.to_le_bytes()); + dst.extend_from_slice(&self.is_dir.to_le_bytes()); + dst.extend_from_slice(&(self.is_hidden as u32).to_le_bytes()); + dst.extend_from_slice(&self.file_id.to_le_bytes()); + dst.extend_from_slice(&self.name); + dst.resize(dst.len() + len - HEADER_LEN - self.name.len(), 0); + Ok(()) + } +} + +fn aligned_record_len(name_len: usize) -> AsyncHostResult { + let len = HEADER_LEN + .checked_add(name_len) + .ok_or(AsyncHostError::Fault)?; + Ok((len + 7) & !7) +} + +fn read_u32(buf: &[u8], offset: i32, field_offset: usize) -> AsyncHostResult { + let offset = usize::try_from(offset).map_err(|_| AsyncHostError::Fault)?; + let start = offset + .checked_add(field_offset) + .ok_or(AsyncHostError::Fault)?; + let end = start.checked_add(4).ok_or(AsyncHostError::Fault)?; + let bytes = buf.get(start..end).ok_or(AsyncHostError::Fault)?; + Ok(u32::from_le_bytes(bytes.try_into().unwrap())) +} + +fn read_i32(buf: &[u8], offset: i32, field_offset: usize) -> AsyncHostResult { + Ok(read_u32(buf, offset, field_offset)? as i32) +} + +fn read_u64(buf: &[u8], offset: i32, field_offset: usize) -> AsyncHostResult { + let offset = usize::try_from(offset).map_err(|_| AsyncHostError::Fault)?; + let start = offset + .checked_add(field_offset) + .ok_or(AsyncHostError::Fault)?; + let end = start.checked_add(8).ok_or(AsyncHostError::Fault)?; + let bytes = buf.get(start..end).ok_or(AsyncHostError::Fault)?; + Ok(u64::from_le_bytes(bytes.try_into().unwrap())) +} + +ported_fns! { + #[ported( + source = "src/fs/dir.c", + original = "moonbitlang_async_dir_buffer_min_size" + )] + pub(crate) fn buffer_min_size() -> i32 { + BUFFER_MIN_SIZE as i32 + } + + #[ported( + source = "src/fs/dir.c", + original = "moonbitlang_async_dir_entry_length" + )] + pub(crate) fn entry_length(buf: &[u8], offset: i32) -> AsyncHostResult { + i32::try_from(read_u32(buf, offset, 0)?).map_err(|_| AsyncHostError::Fault) + } + + #[ported( + source = "src/fs/dir.c", + original = "moonbitlang_async_dir_entry_get_name_len" + )] + pub(crate) fn entry_name_len(buf: &[u8], offset: i32) -> AsyncHostResult { + i32::try_from(read_u32(buf, offset, 4)?).map_err(|_| AsyncHostError::Fault) + } + + #[ported( + source = "src/fs/dir.c", + original = "moonbitlang_async_dir_entry_get_name" + )] + pub(crate) fn entry_name_ptr(buf_ptr: i32, offset: i32) -> AsyncHostResult { + buf_ptr + .checked_add(offset) + .and_then(|ptr| ptr.checked_add(HEADER_LEN as i32)) + .ok_or(AsyncHostError::Fault) + } + + #[ported( + source = "src/fs/dir.c", + original = "moonbitlang_async_dir_entry_is_dir" + )] + pub(crate) fn entry_is_dir(buf: &[u8], offset: i32) -> AsyncHostResult { + read_i32(buf, offset, 8) + } + + #[ported( + source = "src/fs/dir.c", + original = "moonbitlang_async_dir_entry_is_hidden" + )] + pub(crate) fn entry_is_hidden(buf: &[u8], offset: i32) -> AsyncHostResult { + Ok(read_u32(buf, offset, 12)? != 0) + } + + #[ported( + source = "src/fs/dir.c", + original = "moonbitlang_async_dir_entry_get_file_id" + )] + pub(crate) fn entry_file_id(buf: &[u8], offset: i32) -> AsyncHostResult { + read_u64(buf, offset, 16) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn records_use_fixed_little_endian_layout() { + let record = EntryRecord { + name: b"abc".to_vec(), + is_dir: 1, + is_hidden: false, + file_id: 0x0102_0304_0506_0708, + }; + let mut buf = Vec::new(); + + record.encode_into(&mut buf).unwrap(); + + assert_eq!(entry_length(&buf, 0), Ok(32)); + assert_eq!(entry_name_len(&buf, 0), Ok(3)); + assert_eq!(entry_is_dir(&buf, 0), Ok(1)); + assert_eq!(entry_is_hidden(&buf, 0), Ok(false)); + assert_eq!(entry_file_id(&buf, 0), Ok(0x0102_0304_0506_0708)); + assert_eq!(&buf[HEADER_LEN..HEADER_LEN + 3], b"abc"); + } +} diff --git a/crates/moonrun/src/async_sys/fs/mod.rs b/crates/moonrun/src/async_sys/fs/mod.rs new file mode 100644 index 000000000..25522b068 --- /dev/null +++ b/crates/moonrun/src/async_sys/fs/mod.rs @@ -0,0 +1,20 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +pub(crate) mod dir; +pub(crate) mod stub; diff --git a/crates/moonrun/src/async_sys/fs/stub.rs b/crates/moonrun/src/async_sys/fs/stub.rs new file mode 100644 index 000000000..935673d85 --- /dev/null +++ b/crates/moonrun/src/async_sys/fs/stub.rs @@ -0,0 +1,346 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use std::ffi::OsString; +use std::fs::File; + +use crate::async_host::{AsyncHostError, AsyncHostResult}; +use crate::async_sys::internal::event_loop::thread_pool::HostFile; +use crate::async_sys::ported_fns; + +#[cfg(unix)] +#[allow(dead_code)] +pub(crate) type RawFileHandle = std::os::fd::RawFd; + +#[cfg(windows)] +#[allow(dead_code)] +pub(crate) type RawFileHandle = windows_sys::Win32::Foundation::HANDLE; + +ported_fns! { + #[ported( + source = "src/fs/stub.c", + original = "moonbitlang_async_dir_is_null" + )] + #[cfg(unix)] + #[allow(dead_code)] + pub(crate) fn dir_is_null(dir: *mut libc::DIR) -> bool { + dir.is_null() + } + + #[ported( + source = "src/fs/stub.c", + original = "moonbitlang_async_errno_is_lock_violation" + )] + pub(crate) fn errno_is_lock_violation(errno: i32) -> bool { + #[cfg(unix)] + { + errno == libc::EWOULDBLOCK + } + #[cfg(windows)] + { + use windows_sys::Win32::Foundation::ERROR_LOCK_VIOLATION; + errno == ERROR_LOCK_VIOLATION as i32 + } + } + + #[ported( + source = "src/fs/stub.c", + original = "moonbitlang_async_get_tmp_path" + )] + pub(crate) fn get_tmp_path() -> AsyncHostResult { + tmp_path_from_native_stub() + } + + #[ported( + source = "src/fs/stub.c", + original = "moonbitlang_async_try_lock_file" + )] + #[allow(dead_code)] + pub(crate) fn try_lock_file(handle: RawFileHandle, exclusive: bool) -> AsyncHostResult<()> { + try_lock_file_from_native_stub(handle, exclusive) + } + + #[ported( + source = "src/fs/stub.c", + original = "moonbitlang_async_unlock_file" + )] + #[allow(dead_code)] + pub(crate) fn unlock_file(handle: RawFileHandle) -> AsyncHostResult<()> { + unlock_file_from_native_stub(handle) + } +} + +#[cfg(unix)] +#[allow(dead_code)] +fn try_lock_file_from_native_stub(fd: RawFileHandle, exclusive: bool) -> AsyncHostResult<()> { + let operation = libc::LOCK_NB + | if exclusive { + libc::LOCK_EX + } else { + libc::LOCK_SH + }; + if unsafe { libc::flock(fd, operation) } == 0 { + Ok(()) + } else { + Err(last_native_error()) + } +} + +#[cfg(unix)] +#[allow(dead_code)] +fn unlock_file_from_native_stub(fd: RawFileHandle) -> AsyncHostResult<()> { + if unsafe { libc::flock(fd, libc::LOCK_UN) } == 0 { + Ok(()) + } else { + Err(last_native_error()) + } +} + +#[cfg(unix)] +pub(crate) fn try_lock_std_file(file: &File, exclusive: bool) -> AsyncHostResult<()> { + use std::os::fd::AsRawFd; + + try_lock_file(file.as_raw_fd(), exclusive) +} + +#[cfg(unix)] +pub(crate) fn unlock_std_file(file: &File) -> AsyncHostResult<()> { + use std::os::fd::AsRawFd; + + unlock_file(file.as_raw_fd()) +} + +#[cfg(windows)] +#[allow(dead_code)] +fn try_lock_file_from_native_stub(handle: RawFileHandle, exclusive: bool) -> AsyncHostResult<()> { + use std::mem::zeroed; + use windows_sys::Win32::Storage::FileSystem::{ + LOCKFILE_EXCLUSIVE_LOCK, LOCKFILE_FAIL_IMMEDIATELY, LockFileEx, + }; + use windows_sys::Win32::System::IO::OVERLAPPED; + + let mut overlapped: OVERLAPPED = unsafe { zeroed() }; + // Keep parity with async's native stub: lock a one-byte sentinel range at + // the end of the 64-bit file-position space. + overlapped.Anonymous.Anonymous.Offset = 0xfffffffe; + overlapped.Anonymous.Anonymous.OffsetHigh = 0xffffffff; + let flags = LOCKFILE_FAIL_IMMEDIATELY + | if exclusive { + LOCKFILE_EXCLUSIVE_LOCK + } else { + 0 + }; + + if unsafe { LockFileEx(handle, flags, 0, 1, 0, &mut overlapped) } != 0 { + Ok(()) + } else { + Err(last_native_error()) + } +} + +#[cfg(windows)] +#[allow(dead_code)] +fn unlock_file_from_native_stub(handle: RawFileHandle) -> AsyncHostResult<()> { + use windows_sys::Win32::Storage::FileSystem::UnlockFile; + + if unsafe { UnlockFile(handle, 0xfffffffe, 0xffffffff, 1, 0) } != 0 { + Ok(()) + } else { + Err(last_native_error()) + } +} + +#[cfg(windows)] +pub(crate) fn try_lock_std_file(file: &File, exclusive: bool) -> AsyncHostResult<()> { + use std::os::windows::io::AsRawHandle; + + try_lock_file(file.as_raw_handle(), exclusive) +} + +#[cfg(windows)] +pub(crate) fn unlock_std_file(file: &File) -> AsyncHostResult<()> { + use std::os::windows::io::AsRawHandle; + + unlock_file(file.as_raw_handle()) +} + +pub(crate) fn try_lock_host_file(file: &mut HostFile, exclusive: bool) -> AsyncHostResult<()> { + try_lock_std_file(file.file_mut(), exclusive) +} + +pub(crate) fn unlock_host_file(file: &mut HostFile) -> AsyncHostResult<()> { + #[cfg(windows)] + { + if let Some(lock_file) = file.take_lock_file() { + return unlock_std_file(&lock_file); + } + } + + unlock_std_file(file.file_mut()) +} + +#[allow(dead_code)] +fn last_native_error() -> AsyncHostError { + AsyncHostError::Native( + std::io::Error::last_os_error() + .raw_os_error() + .unwrap_or_else(|| AsyncHostError::Inval.errno()), + ) +} + +#[cfg(unix)] +fn tmp_path_from_native_stub() -> AsyncHostResult { + // POSIX reserves TMPDIR for temporary-file placement. The async tmpdir + // layer concatenates this base path with a generated name, so the host + // normalizes the Unix base to include the separator. + Ok(std::env::var_os("TMPDIR") + .and_then(separator_terminated_unix_path) + .unwrap_or_else(default_unix_tmp_path)) +} + +#[cfg(all(unix, target_os = "android"))] +fn default_unix_tmp_path() -> OsString { + OsString::from("/data/local/tmp/") +} + +#[cfg(all(unix, not(target_os = "android")))] +fn default_unix_tmp_path() -> OsString { + OsString::from("/tmp/") +} + +#[cfg(unix)] +fn separator_terminated_unix_path(path: OsString) -> Option { + use std::os::unix::ffi::{OsStrExt, OsStringExt}; + + let bytes = path.as_os_str().as_bytes(); + if bytes.is_empty() { + return None; + } + if bytes.ends_with(b"/") { + return Some(path); + } + + let mut bytes = path.into_vec(); + bytes.push(b'/'); + Some(OsString::from_vec(bytes)) +} + +#[cfg(windows)] +fn tmp_path_from_native_stub() -> AsyncHostResult { + use crate::async_host::AsyncHostError; + use std::os::windows::ffi::OsStringExt; + use windows_sys::Win32::Foundation::{GetLastError, MAX_PATH}; + use windows_sys::Win32::Storage::FileSystem::GetTempPath2W; + + let mut buffer = [0u16; MAX_PATH as usize + 1]; + let len = unsafe { GetTempPath2W(buffer.len() as u32, buffer.as_mut_ptr()) }; + if len == 0 { + return Err(AsyncHostError::Native(unsafe { GetLastError() as i32 })); + } + + let len = usize::try_from(len).map_err(|_| AsyncHostError::Fault)?; + Ok(OsString::from_wide(&buffer[..len])) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn tmp_path_is_non_empty_and_separator_terminated() { + let path = get_tmp_path().unwrap(); + + assert!(!path.as_os_str().is_empty()); + let path = path.to_string_lossy(); + assert!(path.ends_with('/') || path.ends_with('\\')); + } + + #[cfg(all(unix, not(target_os = "android")))] + #[test] + fn default_unix_tmp_path_matches_async_stub_fallback() { + use std::os::unix::ffi::OsStrExt; + + assert_eq!(default_unix_tmp_path().as_os_str().as_bytes(), b"/tmp/"); + } + + #[cfg(unix)] + #[test] + fn tmpdir_env_value_is_separator_terminated() { + use std::os::unix::ffi::OsStrExt; + + assert_eq!( + separator_terminated_unix_path(OsString::from("/var/tmp")) + .unwrap() + .as_os_str() + .as_bytes(), + b"/var/tmp/" + ); + } + + #[cfg(unix)] + #[test] + fn empty_tmpdir_env_value_is_ignored() { + assert_eq!(separator_terminated_unix_path(OsString::from("")), None); + } + + #[cfg(unix)] + #[test] + fn unix_lock_violation_matches_async_stub() { + assert!(errno_is_lock_violation(libc::EWOULDBLOCK)); + assert!(!errno_is_lock_violation(libc::EINVAL)); + } + + #[cfg(unix)] + #[test] + fn unix_dir_is_null_matches_async_stub() { + assert!(dir_is_null(std::ptr::null_mut())); + assert!(!dir_is_null( + std::ptr::NonNull::::dangling().as_ptr() + )); + } + + #[cfg(unix)] + #[test] + fn unix_try_lock_and_unlock_file_match_async_stub() { + use std::os::fd::AsRawFd; + + let path = + std::env::temp_dir().join(format!("moonrun-async-lock-test-{}", std::process::id())); + let file = std::fs::OpenOptions::new() + .create(true) + .truncate(true) + .read(true) + .write(true) + .open(&path) + .unwrap(); + + try_lock_file(file.as_raw_fd(), true).unwrap(); + unlock_file(file.as_raw_fd()).unwrap(); + std::fs::remove_file(path).unwrap(); + } + + #[cfg(unix)] + #[test] + fn unix_invalid_lock_file_records_native_errno() { + assert_eq!( + try_lock_file(-1, true), + Err(AsyncHostError::Native(libc::EBADF)) + ); + assert_eq!(unlock_file(-1), Err(AsyncHostError::Native(libc::EBADF))); + } +} diff --git a/crates/moonrun/src/async_sys/internal/c_buffer/mod.rs b/crates/moonrun/src/async_sys/internal/c_buffer/mod.rs new file mode 100644 index 000000000..fa380a27b --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/c_buffer/mod.rs @@ -0,0 +1,19 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +pub(crate) mod stub; diff --git a/crates/moonrun/src/async_sys/internal/c_buffer/stub.rs b/crates/moonrun/src/async_sys/internal/c_buffer/stub.rs new file mode 100644 index 000000000..152e87fe1 --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/c_buffer/stub.rs @@ -0,0 +1,132 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_host::{AsyncHostError, AsyncHostResult}; +use crate::async_sys::ported_fns; + +ported_fns! { + #[ported( + source = "src/internal/c_buffer/stub.c", + original = "moonbitlang_async_blit_to_c" + )] + #[allow(dead_code)] + pub(crate) fn blit_to_c(dst: &mut [u8], src: &[u8], offset: i32, len: i32) -> AsyncHostResult<()> { + let src = checked_range(src, offset, len)?; + let dst = dst.get_mut(..src.len()).ok_or(AsyncHostError::Fault)?; + dst.copy_from_slice(src); + Ok(()) + } + + #[ported( + source = "src/internal/c_buffer/stub.c", + original = "moonbitlang_async_blit_from_c" + )] + #[allow(dead_code)] + pub(crate) fn blit_from_c(src: &[u8], dst: &mut [u8], offset: i32, len: i32) -> AsyncHostResult<()> { + let len = usize::try_from(len).map_err(|_| AsyncHostError::Fault)?; + let src = src.get(..len).ok_or(AsyncHostError::Fault)?; + let offset = usize::try_from(offset).map_err(|_| AsyncHostError::Fault)?; + let end = offset.checked_add(len).ok_or(AsyncHostError::Fault)?; + let dst = dst.get_mut(offset..end).ok_or(AsyncHostError::Fault)?; + dst.copy_from_slice(src); + Ok(()) + } + + #[ported( + source = "src/internal/c_buffer/stub.c", + original = "moonbitlang_async_c_buffer_get" + )] + #[allow(dead_code)] + pub(crate) fn c_buffer_get(buf: &[u8], index: i32) -> AsyncHostResult { + let index = usize::try_from(index).map_err(|_| AsyncHostError::Fault)?; + buf.get(index).copied().ok_or(AsyncHostError::Fault) + } + + #[ported( + source = "src/internal/c_buffer/stub.c", + original = "moonbitlang_async_strlen" + )] + #[allow(dead_code)] + pub(crate) fn strlen(buf: &[u8]) -> AsyncHostResult { + let len = buf + .iter() + .position(|byte| *byte == 0) + .ok_or(AsyncHostError::Fault)?; + i32::try_from(len).map_err(|_| AsyncHostError::Fault) + } + + #[ported( + source = "src/internal/c_buffer/stub.c", + original = "moonbitlang_async_null_pointer" + )] + #[allow(dead_code)] + pub(crate) fn null_pointer() -> i32 { + 0 + } + + #[ported( + source = "src/internal/c_buffer/stub.c", + original = "moonbitlang_async_pointer_is_null" + )] + #[allow(dead_code)] + pub(crate) fn pointer_is_null(ptr: i32) -> bool { + ptr == 0 + } +} + +fn checked_range(src: &[u8], offset: i32, len: i32) -> AsyncHostResult<&[u8]> { + let offset = usize::try_from(offset).map_err(|_| AsyncHostError::Fault)?; + let len = usize::try_from(len).map_err(|_| AsyncHostError::Fault)?; + let end = offset.checked_add(len).ok_or(AsyncHostError::Fault)?; + src.get(offset..end).ok_or(AsyncHostError::Fault) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn blit_to_c_copies_from_source_offset() { + let mut dst = [0; 3]; + + blit_to_c(&mut dst, b"abcdef", 2, 3).unwrap(); + + assert_eq!(&dst, b"cde"); + } + + #[test] + fn blit_from_c_copies_to_destination_offset() { + let mut dst = *b"abcdef"; + + blit_from_c(b"XY", &mut dst, 2, 2).unwrap(); + + assert_eq!(&dst, b"abXYef"); + } + + #[test] + fn strlen_stops_at_first_nul() { + assert_eq!(strlen(b"abc\0def").unwrap(), 3); + } + + #[test] + fn null_pointer_helpers_match_c_stub() { + assert_eq!(null_pointer(), 0); + assert!(pointer_is_null(0)); + assert!(!pointer_is_null(1)); + } +} diff --git a/crates/moonrun/src/async_sys/internal/env_util/mod.rs b/crates/moonrun/src/async_sys/internal/env_util/mod.rs new file mode 100644 index 000000000..fa380a27b --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/env_util/mod.rs @@ -0,0 +1,19 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +pub(crate) mod stub; diff --git a/crates/moonrun/src/async_sys/internal/env_util/stub.rs b/crates/moonrun/src/async_sys/internal/env_util/stub.rs new file mode 100644 index 000000000..e8bd9f7f1 --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/env_util/stub.rs @@ -0,0 +1,40 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_sys::ported_fns; + +ported_fns! { + #[ported( + source = "src/internal/env_util/stub.c", + original = "moonbitlang_async_getpid" + )] + #[allow(dead_code)] + pub(crate) fn get_pid() -> u32 { + std::process::id() + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn get_pid_matches_current_process() { + assert_eq!(get_pid(), std::process::id()); + } +} diff --git a/crates/moonrun/src/async_sys/internal/event_loop/io_unix.rs b/crates/moonrun/src/async_sys/internal/event_loop/io_unix.rs new file mode 100644 index 000000000..b701f3c0d --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/event_loop/io_unix.rs @@ -0,0 +1,99 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_host::{AsyncHostError, AsyncHostResult}; +use crate::async_sys::internal::fd_util::stub::RawFd; +use crate::async_sys::ported_fns; + +ported_fns! { + #[ported( + source = "src/internal/event_loop/io_unix.c", + original = "moonbitlang_async_read" + )] + #[allow(dead_code)] + pub(crate) fn read(fd: RawFd, buf: &mut [u8], offset: i32, len: i32) -> AsyncHostResult { + let range = checked_mut_range(buf, offset, len)?; + let ret = unsafe { libc::read(fd, range.as_mut_ptr().cast(), range.len()) }; + if ret < 0 { + Err(last_native_error()) + } else { + i32::try_from(ret).map_err(|_| AsyncHostError::Fault) + } + } + + #[ported( + source = "src/internal/event_loop/io_unix.c", + original = "moonbitlang_async_write" + )] + #[allow(dead_code)] + pub(crate) fn write(fd: RawFd, buf: &[u8], offset: i32, len: i32) -> AsyncHostResult { + let range = checked_range(buf, offset, len)?; + let ret = unsafe { libc::write(fd, range.as_ptr().cast(), range.len()) }; + if ret < 0 { + Err(last_native_error()) + } else { + i32::try_from(ret).map_err(|_| AsyncHostError::Fault) + } + } +} + +fn checked_range(buf: &[u8], offset: i32, len: i32) -> AsyncHostResult<&[u8]> { + let offset = usize::try_from(offset).map_err(|_| AsyncHostError::Fault)?; + let len = usize::try_from(len).map_err(|_| AsyncHostError::Fault)?; + let end = offset.checked_add(len).ok_or(AsyncHostError::Fault)?; + buf.get(offset..end).ok_or(AsyncHostError::Fault) +} + +fn checked_mut_range(buf: &mut [u8], offset: i32, len: i32) -> AsyncHostResult<&mut [u8]> { + let offset = usize::try_from(offset).map_err(|_| AsyncHostError::Fault)?; + let len = usize::try_from(len).map_err(|_| AsyncHostError::Fault)?; + let end = offset.checked_add(len).ok_or(AsyncHostError::Fault)?; + buf.get_mut(offset..end).ok_or(AsyncHostError::Fault) +} + +fn last_native_error() -> AsyncHostError { + AsyncHostError::Native( + std::io::Error::last_os_error() + .raw_os_error() + .unwrap_or_else(|| AsyncHostError::Inval.errno()), + ) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn read_and_write_use_buffer_offsets() { + let mut fds = [0; 2]; + assert_eq!(unsafe { libc::pipe(fds.as_mut_ptr()) }, 0); + let read_fd = fds[0]; + let write_fd = fds[1]; + + assert_eq!(write(write_fd, b"abcdef", 2, 3).unwrap(), 3); + + let mut buf = *b"------"; + assert_eq!(read(read_fd, &mut buf, 1, 3).unwrap(), 3); + assert_eq!(&buf, b"-cde--"); + + unsafe { + libc::close(read_fd); + libc::close(write_fd); + } + } +} diff --git a/crates/moonrun/src/async_sys/internal/event_loop/io_windows.rs b/crates/moonrun/src/async_sys/internal/event_loop/io_windows.rs new file mode 100644 index 000000000..96fb70cdb --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/event_loop/io_windows.rs @@ -0,0 +1,757 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_host::{AsyncHostError, AsyncHostResult}; +use crate::async_sys::ported_fns; + +use windows_sys::Win32::Foundation::HANDLE; +use windows_sys::Win32::System::IO::OVERLAPPED; + +#[repr(i32)] +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +#[allow(dead_code)] +pub(crate) enum IoResultKind { + File = 0, + Socket, + SocketWithAddr, + Connect, + Accept, + ReadDirChanges, +} + +#[repr(C)] +#[allow(dead_code)] +pub(crate) struct IoResultHeader { + overlapped: OVERLAPPED, + kind: IoResultKind, + job_id: i32, +} + +#[allow(dead_code)] +pub(crate) struct FileIoResult { + header: IoResultHeader, + buf: Vec, + offset: usize, + len: usize, +} + +#[allow(dead_code)] +pub(crate) struct SocketIoResult { + header: IoResultHeader, + buf: Vec, + offset: usize, + len: usize, + flags: u32, +} + +#[allow(dead_code)] +pub(crate) struct SocketWithAddrIoResult { + header: IoResultHeader, + buf: Vec, + offset: usize, + len: usize, + flags: u32, + addr: Vec, + addr_len: i32, +} + +#[allow(dead_code)] +pub(crate) struct ConnectIoResult { + header: IoResultHeader, + addr: Vec, +} + +#[allow(dead_code)] +pub(crate) struct AcceptIoResult { + header: IoResultHeader, + bytes_received: u32, + accept_buffer: [u8; ACCEPT_BUFFER_LEN], +} + +#[allow(dead_code)] +pub(crate) struct ReadDirChangesIoResult { + header: IoResultHeader, + buf: Vec, +} + +#[allow(dead_code)] +pub(crate) enum IoResult { + File(FileIoResult), + Socket(SocketIoResult), + SocketWithAddr(SocketWithAddrIoResult), + Connect(ConnectIoResult), + Accept(AcceptIoResult), + ReadDirChanges(ReadDirChangesIoResult), +} + +const ACCEPT_BUFFER_LEN: usize = + std::mem::size_of::() * 2; + +impl IoResultHeader { + fn new(job_id: i32, kind: IoResultKind) -> Self { + Self { + overlapped: unsafe { std::mem::zeroed() }, + kind, + job_id, + } + } +} + +impl IoResult { + fn header(&self) -> &IoResultHeader { + match self { + Self::File(result) => &result.header, + Self::Socket(result) => &result.header, + Self::SocketWithAddr(result) => &result.header, + Self::Connect(result) => &result.header, + Self::Accept(result) => &result.header, + Self::ReadDirChanges(result) => &result.header, + } + } + + fn overlapped_mut(&mut self) -> *mut OVERLAPPED { + match self { + Self::File(result) => &mut result.header.overlapped, + Self::Socket(result) => &mut result.header.overlapped, + Self::SocketWithAddr(result) => &mut result.header.overlapped, + Self::Connect(result) => &mut result.header.overlapped, + Self::Accept(result) => &mut result.header.overlapped, + Self::ReadDirChanges(result) => &mut result.header.overlapped, + } + } +} + +ported_fns! { + #[ported( + source = "src/internal/event_loop/io_windows.c", + original = "moonbitlang_async_init_WSA" + )] + #[allow(dead_code)] + pub(crate) fn init_wsa() -> i32 { + use windows_sys::Win32::Networking::WinSock::{WSADATA, WSAStartup}; + + let mut data = unsafe { std::mem::zeroed::() }; + unsafe { WSAStartup(0x0202, &mut data) } + } + + #[ported( + source = "src/internal/event_loop/io_windows.c", + original = "moonbitlang_async_cleanup_WSA" + )] + #[allow(dead_code)] + pub(crate) fn cleanup_wsa() -> i32 { + unsafe { windows_sys::Win32::Networking::WinSock::WSACleanup() } + } + + #[ported( + source = "src/internal/event_loop/io_windows.c", + original = "moonbitlang_async_make_file_io_result" + )] + #[allow(dead_code)] + pub(crate) fn make_file_io_result( + job_id: i32, + buf: Vec, + offset: i32, + len: i32, + position: i64, + ) -> AsyncHostResult { + let offset = usize::try_from(offset).map_err(|_| AsyncHostError::Fault)?; + let len = usize::try_from(len).map_err(|_| AsyncHostError::Fault)?; + checked_range(&buf, offset, len)?; + let mut header = IoResultHeader::new(job_id, IoResultKind::File); + header.overlapped.Anonymous.Anonymous.Offset = position as u32; + header.overlapped.Anonymous.Anonymous.OffsetHigh = (position >> 32) as u32; + Ok(IoResult::File(FileIoResult { + header, + buf, + offset, + len, + })) + } + + #[ported( + source = "src/internal/event_loop/io_windows.c", + original = "moonbitlang_async_make_socket_io_result" + )] + #[allow(dead_code)] + pub(crate) fn make_socket_io_result( + job_id: i32, + buf: Vec, + offset: i32, + len: i32, + flags: i32, + ) -> AsyncHostResult { + let offset = usize::try_from(offset).map_err(|_| AsyncHostError::Fault)?; + let len = usize::try_from(len).map_err(|_| AsyncHostError::Fault)?; + checked_range(&buf, offset, len)?; + Ok(IoResult::Socket(SocketIoResult { + header: IoResultHeader::new(job_id, IoResultKind::Socket), + buf, + offset, + len, + flags: flags as u32, + })) + } + + #[ported( + source = "src/internal/event_loop/io_windows.c", + original = "moonbitlang_async_make_socket_with_addr_io_result" + )] + #[allow(dead_code)] + pub(crate) fn make_socket_with_addr_io_result( + job_id: i32, + buf: Vec, + offset: i32, + len: i32, + flags: i32, + addr: Vec, + ) -> AsyncHostResult { + let offset = usize::try_from(offset).map_err(|_| AsyncHostError::Fault)?; + let len = usize::try_from(len).map_err(|_| AsyncHostError::Fault)?; + checked_range(&buf, offset, len)?; + let addr_len = sockaddr_len(&addr)?; + Ok(IoResult::SocketWithAddr(SocketWithAddrIoResult { + header: IoResultHeader::new(job_id, IoResultKind::SocketWithAddr), + buf, + offset, + len, + flags: flags as u32, + addr, + addr_len, + })) + } + + #[ported( + source = "src/internal/event_loop/io_windows.c", + original = "moonbitlang_async_make_connect_io_result" + )] + #[allow(dead_code)] + pub(crate) fn make_connect_io_result(job_id: i32, addr: Vec) -> AsyncHostResult { + sockaddr_len(&addr)?; + Ok(IoResult::Connect(ConnectIoResult { + header: IoResultHeader::new(job_id, IoResultKind::Connect), + addr, + })) + } + + #[ported( + source = "src/internal/event_loop/io_windows.c", + original = "moonbitlang_async_make_accept_io_result" + )] + #[allow(dead_code)] + pub(crate) fn make_accept_io_result(job_id: i32) -> IoResult { + IoResult::Accept(AcceptIoResult { + header: IoResultHeader::new(job_id, IoResultKind::Accept), + bytes_received: 0, + accept_buffer: [0; ACCEPT_BUFFER_LEN], + }) + } + + #[ported( + source = "src/internal/event_loop/io_windows.c", + original = "moonbitlang_async_make_read_dir_changes_io_result" + )] + #[allow(dead_code)] + pub(crate) fn make_read_dir_changes_io_result(job_id: i32, buf: Vec, len: i32) -> AsyncHostResult { + let len = usize::try_from(len).map_err(|_| AsyncHostError::Fault)?; + buf.get(..len).ok_or(AsyncHostError::Fault)?; + Ok(IoResult::ReadDirChanges(ReadDirChangesIoResult { + header: IoResultHeader::new(job_id, IoResultKind::ReadDirChanges), + buf, + })) + } + + #[ported( + source = "src/internal/event_loop/io_windows.c", + original = "moonbitlang_async_free_io_result" + )] + #[allow(dead_code)] + pub(crate) fn free_io_result(result: IoResult) { + drop(result); + } + + #[ported( + source = "src/internal/event_loop/io_windows.c", + original = "moonbitlang_async_io_result_get_job_id" + )] + #[allow(dead_code)] + pub(crate) fn io_result_get_job_id(result: &IoResult) -> i32 { + result.header().job_id + } + + #[ported( + source = "src/internal/event_loop/io_windows.c", + original = "moonbitlang_async_io_result_get_status" + )] + #[allow(dead_code)] + pub(crate) fn io_result_get_status(result: &mut IoResult, handle: HANDLE) -> AsyncHostResult { + use windows_sys::Win32::System::IO::GetOverlappedResult; + + let mut bytes_transferred = 0; + if unsafe { GetOverlappedResult(handle, result.overlapped_mut(), &mut bytes_transferred, 0) } != 0 { + i32::try_from(bytes_transferred).map_err(|_| AsyncHostError::Fault) + } else { + Err(last_native_error()) + } + } + + #[ported( + source = "src/internal/event_loop/io_windows.c", + original = "moonbitlang_async_cancel_io_result" + )] + #[allow(dead_code)] + pub(crate) fn cancel_io_result(result: &mut IoResult, handle: HANDLE) -> i32 { + use windows_sys::Win32::Foundation::{ERROR_IO_INCOMPLETE, ERROR_NOT_FOUND, GetLastError}; + use windows_sys::Win32::System::IO::{CancelIoEx, GetOverlappedResult}; + + if unsafe { CancelIoEx(handle, result.overlapped_mut()) } == 0 { + return if unsafe { GetLastError() } == ERROR_NOT_FOUND { 0 } else { -1 }; + } + + let mut bytes_transferred = 0; + if unsafe { GetOverlappedResult(handle, result.overlapped_mut(), &mut bytes_transferred, 0) } != 0 { + return 0; + } + if unsafe { GetLastError() } == ERROR_IO_INCOMPLETE { 1 } else { 0 } + } + + #[ported( + source = "src/internal/event_loop/io_windows.c", + original = "moonbitlang_async_errno_is_read_EOF" + )] + #[allow(dead_code)] + pub(crate) fn errno_is_read_eof(errno: i32) -> bool { + use windows_sys::Win32::Foundation::{ERROR_BROKEN_PIPE, ERROR_HANDLE_EOF}; + + errno == ERROR_HANDLE_EOF as i32 || errno == ERROR_BROKEN_PIPE as i32 + } + + #[ported( + source = "src/internal/event_loop/io_windows.c", + original = "moonbitlang_async_read" + )] + #[allow(dead_code)] + pub(crate) fn read(handle: HANDLE, result: &mut IoResult) -> AsyncHostResult { + use windows_sys::Win32::Foundation::ERROR_HANDLE_EOF; + use windows_sys::Win32::Networking::WinSock::{SOCKET, WSARecv, WSARecvFrom, WSABUF}; + use windows_sys::Win32::Storage::FileSystem::ReadFile; + + let mut n_read = 0; + let success = match result { + IoResult::File(result) => unsafe { + ReadFile( + handle, + result.buf.as_mut_ptr().add(result.offset), + result.len as u32, + &mut n_read, + &mut result.header.overlapped, + ) != 0 + }, + IoResult::Socket(result) => { + let buf = WSABUF { + len: result.len as u32, + buf: unsafe { result.buf.as_mut_ptr().add(result.offset).cast() }, + }; + unsafe { + WSARecv( + handle as SOCKET, + &buf, + 1, + &mut n_read, + &mut result.flags, + &mut result.header.overlapped, + None, + ) == 0 + } + } + IoResult::SocketWithAddr(result) => { + let buf = WSABUF { + len: result.len as u32, + buf: unsafe { result.buf.as_mut_ptr().add(result.offset).cast() }, + }; + unsafe { + WSARecvFrom( + handle as SOCKET, + &buf, + 1, + &mut n_read, + &mut result.flags, + result.addr.as_mut_ptr().cast(), + &mut result.addr_len, + &mut result.header.overlapped, + None, + ) == 0 + } + } + _ => return Err(AsyncHostError::Inval), + }; + + if success { + i32::try_from(n_read).map_err(|_| AsyncHostError::Fault) + } else if last_errno() == ERROR_HANDLE_EOF as i32 { + Ok(0) + } else { + Err(last_native_error()) + } + } + + #[ported( + source = "src/internal/event_loop/io_windows.c", + original = "moonbitlang_async_write" + )] + #[allow(dead_code)] + pub(crate) fn write(handle: HANDLE, result: &mut IoResult) -> AsyncHostResult { + use windows_sys::Win32::Networking::WinSock::{SOCKET, WSABUF, WSASend, WSASendTo}; + use windows_sys::Win32::Storage::FileSystem::WriteFile; + + let mut n_written = 0; + let success = match result { + IoResult::File(result) => unsafe { + WriteFile( + handle, + result.buf.as_ptr().add(result.offset), + result.len as u32, + &mut n_written, + &mut result.header.overlapped, + ) != 0 + }, + IoResult::Socket(result) => { + let buf = WSABUF { + len: result.len as u32, + buf: unsafe { result.buf.as_mut_ptr().add(result.offset).cast() }, + }; + unsafe { + WSASend( + handle as SOCKET, + &buf, + 1, + &mut n_written, + result.flags, + &mut result.header.overlapped, + None, + ) == 0 + } + } + IoResult::SocketWithAddr(result) => { + let buf = WSABUF { + len: result.len as u32, + buf: unsafe { result.buf.as_mut_ptr().add(result.offset).cast() }, + }; + unsafe { + WSASendTo( + handle as SOCKET, + &buf, + 1, + &mut n_written, + result.flags, + result.addr.as_ptr().cast(), + result.addr_len, + &mut result.header.overlapped, + None, + ) == 0 + } + } + _ => return Err(AsyncHostError::Inval), + }; + + if success { + i32::try_from(n_written).map_err(|_| AsyncHostError::Fault) + } else { + Err(last_native_error()) + } + } + + #[ported( + source = "src/internal/event_loop/io_windows.c", + original = "moonbitlang_async_connect" + )] + #[allow(dead_code)] + pub(crate) fn connect(handle: HANDLE, result: &mut IoResult) -> AsyncHostResult<()> { + use windows_sys::Win32::Networking::WinSock::{ + AF_INET, AF_INET6, SOCKADDR, SOCKADDR_IN, SOCKADDR_IN6, SOCKET, bind, + }; + + let IoResult::Connect(result) = result else { + return Err(AsyncHostError::Inval); + }; + let Some(connect_ex) = get_connect_ex(handle) else { + return Err(AsyncHostError::Native(last_errno())); + }; + let family = sockaddr_family(&result.addr)?; + let bind_result = if family == AF_INET { + let addr = SOCKADDR_IN { + sin_family: AF_INET, + sin_port: 0, + sin_addr: unsafe { std::mem::zeroed() }, + sin_zero: [0; 8], + }; + unsafe { + bind( + handle as SOCKET, + (&addr as *const SOCKADDR_IN).cast::(), + std::mem::size_of::() as i32, + ) + } + } else if family == AF_INET6 { + let addr = SOCKADDR_IN6 { + sin6_family: AF_INET6, + sin6_port: 0, + sin6_flowinfo: 0, + sin6_addr: unsafe { std::mem::zeroed() }, + Anonymous: unsafe { std::mem::zeroed() }, + }; + unsafe { + bind( + handle as SOCKET, + (&addr as *const SOCKADDR_IN6).cast::(), + std::mem::size_of::() as i32, + ) + } + } else { + return Err(AsyncHostError::Inval); + }; + if bind_result != 0 { + return Err(last_native_error()); + } + + let ok = unsafe { + connect_ex( + handle as SOCKET, + result.addr.as_ptr().cast(), + sockaddr_len(&result.addr)?, + std::ptr::null(), + 0, + std::ptr::null_mut(), + &mut result.header.overlapped, + ) + }; + if ok != 0 { Ok(()) } else { Err(last_native_error()) } + } + + #[ported( + source = "src/internal/event_loop/io_windows.c", + original = "moonbitlang_async_setup_connected_socket" + )] + #[allow(dead_code)] + pub(crate) fn setup_connected_socket(handle: HANDLE) -> AsyncHostResult<()> { + use windows_sys::Win32::Networking::WinSock::{ + SO_UPDATE_CONNECT_CONTEXT, SOCKET, SOL_SOCKET, setsockopt, + }; + + let yes = 1u32; + let ret = unsafe { + setsockopt( + handle as SOCKET, + SOL_SOCKET, + SO_UPDATE_CONNECT_CONTEXT, + (&yes as *const u32).cast(), + std::mem::size_of::() as i32, + ) + }; + if ret == 0 { Ok(()) } else { Err(last_native_error()) } + } + + #[ported( + source = "src/internal/event_loop/io_windows.c", + original = "moonbitlang_async_accept" + )] + #[allow(dead_code)] + pub(crate) fn accept(handle: HANDLE, conn_sock: HANDLE, result: &mut IoResult) -> AsyncHostResult<()> { + use windows_sys::Win32::Networking::WinSock::SOCKET; + + let IoResult::Accept(result) = result else { + return Err(AsyncHostError::Inval); + }; + let Some(accept_ex) = get_accept_ex(handle) else { + return Err(AsyncHostError::Native(last_errno())); + }; + let ok = unsafe { + accept_ex( + handle as SOCKET, + conn_sock as SOCKET, + result.accept_buffer.as_mut_ptr().cast(), + 0, + std::mem::size_of::() as u32, + std::mem::size_of::() as u32, + &mut result.bytes_received, + &mut result.header.overlapped, + ) + }; + if ok != 0 { Ok(()) } else { Err(last_native_error()) } + } + + #[ported( + source = "src/internal/event_loop/io_windows.c", + original = "moonbitlang_async_setup_accepted_socket" + )] + #[allow(dead_code)] + pub(crate) fn setup_accepted_socket(listen_sock: HANDLE, accept_sock: HANDLE) -> AsyncHostResult<()> { + use windows_sys::Win32::Networking::WinSock::{ + SO_UPDATE_ACCEPT_CONTEXT, SOCKET, SOL_SOCKET, setsockopt, + }; + + let ret = unsafe { + setsockopt( + accept_sock as SOCKET, + SOL_SOCKET, + SO_UPDATE_ACCEPT_CONTEXT, + (&listen_sock as *const HANDLE).cast(), + std::mem::size_of::() as i32, + ) + }; + if ret == 0 { Ok(()) } else { Err(last_native_error()) } + } + + #[ported( + source = "src/internal/event_loop/io_windows.c", + original = "moonbitlang_async_get_std_handle" + )] + #[allow(dead_code)] + pub(crate) fn get_std_handle(id: i32) -> HANDLE { + unsafe { windows_sys::Win32::System::Console::GetStdHandle(id as u32) } + } + + #[ported( + source = "src/internal/event_loop/io_windows.c", + original = "moonbitlang_async_read_dir_changes" + )] + #[allow(dead_code)] + pub(crate) fn read_dir_changes(dir: HANDLE, result: &mut IoResult) -> AsyncHostResult<()> { + use windows_sys::Win32::Foundation::{ERROR_IO_PENDING, SetLastError}; + use windows_sys::Win32::Storage::FileSystem::{ + FILE_NOTIFY_CHANGE_CREATION, FILE_NOTIFY_CHANGE_DIR_NAME, FILE_NOTIFY_CHANGE_FILE_NAME, + FILE_NOTIFY_CHANGE_LAST_WRITE, FILE_NOTIFY_CHANGE_SIZE, ReadDirectoryChangesW, + }; + + let IoResult::ReadDirChanges(result) = result else { + return Err(AsyncHostError::Inval); + }; + let mut bytes_returned = 0; + let ret = unsafe { + ReadDirectoryChangesW( + dir, + result.buf.as_mut_ptr().cast(), + result.buf.len() as u32, + 1, + FILE_NOTIFY_CHANGE_SIZE + | FILE_NOTIFY_CHANGE_LAST_WRITE + | FILE_NOTIFY_CHANGE_FILE_NAME + | FILE_NOTIFY_CHANGE_DIR_NAME + | FILE_NOTIFY_CHANGE_CREATION, + &mut bytes_returned, + &mut result.header.overlapped, + None, + ) + }; + if ret == 0 { + return Err(last_native_error()); + } + unsafe { SetLastError(ERROR_IO_PENDING) }; + Err(AsyncHostError::Native(ERROR_IO_PENDING as i32)) + } +} + +fn checked_range(buf: &[u8], offset: usize, len: usize) -> AsyncHostResult<&[u8]> { + let end = offset.checked_add(len).ok_or(AsyncHostError::Fault)?; + buf.get(offset..end).ok_or(AsyncHostError::Fault) +} + +fn sockaddr_family(addr: &[u8]) -> AsyncHostResult { + let bytes = addr.get(..2).ok_or(AsyncHostError::Fault)?; + Ok(u16::from_ne_bytes([bytes[0], bytes[1]])) +} + +fn sockaddr_len(addr: &[u8]) -> AsyncHostResult { + use windows_sys::Win32::Networking::WinSock::{AF_INET, AF_INET6, SOCKADDR_IN, SOCKADDR_IN6}; + + let family = sockaddr_family(addr)?; + let len = if family == AF_INET { + std::mem::size_of::() + } else if family == AF_INET6 { + std::mem::size_of::() + } else { + return Err(AsyncHostError::Inval); + }; + addr.get(..len).ok_or(AsyncHostError::Fault)?; + Ok(len as i32) +} + +fn get_connect_ex(handle: HANDLE) -> windows_sys::Win32::Networking::WinSock::LPFN_CONNECTEX { + use windows_sys::Win32::Networking::WinSock::{LPFN_CONNECTEX, WSAID_CONNECTEX}; + + let mut result: LPFN_CONNECTEX = None; + if get_wsa_extension( + handle, + &WSAID_CONNECTEX, + (&mut result as *mut LPFN_CONNECTEX).cast(), + ) { + result + } else { + None + } +} + +fn get_accept_ex(handle: HANDLE) -> windows_sys::Win32::Networking::WinSock::LPFN_ACCEPTEX { + use windows_sys::Win32::Networking::WinSock::{LPFN_ACCEPTEX, WSAID_ACCEPTEX}; + + let mut result: LPFN_ACCEPTEX = None; + if get_wsa_extension( + handle, + &WSAID_ACCEPTEX, + (&mut result as *mut LPFN_ACCEPTEX).cast(), + ) { + result + } else { + None + } +} + +fn get_wsa_extension( + handle: HANDLE, + guid: &windows_sys::core::GUID, + out: *mut std::ffi::c_void, +) -> bool { + use windows_sys::Win32::Networking::WinSock::{ + SIO_GET_EXTENSION_FUNCTION_POINTER, SOCKET, WSAIoctl, + }; + + let mut pointer_size = 0; + unsafe { + WSAIoctl( + handle as SOCKET, + SIO_GET_EXTENSION_FUNCTION_POINTER, + (guid as *const windows_sys::core::GUID).cast(), + std::mem::size_of::() as u32, + out, + std::mem::size_of::() as u32, + &mut pointer_size, + std::ptr::null_mut(), + None, + ) == 0 + } +} + +fn last_errno() -> i32 { + std::io::Error::last_os_error() + .raw_os_error() + .unwrap_or_else(|| AsyncHostError::Inval.errno()) +} + +fn last_native_error() -> AsyncHostError { + AsyncHostError::Native(last_errno()) +} diff --git a/crates/moonrun/src/async_sys/internal/event_loop/mod.rs b/crates/moonrun/src/async_sys/internal/event_loop/mod.rs new file mode 100644 index 000000000..b32712a01 --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/event_loop/mod.rs @@ -0,0 +1,24 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +#[cfg(any(target_os = "linux", target_os = "macos"))] +pub(crate) mod io_unix; +#[cfg(windows)] +pub(crate) mod io_windows; +pub(crate) mod poll; +pub(crate) mod thread_pool; diff --git a/crates/moonrun/src/async_sys/internal/event_loop/poll/epoll.rs b/crates/moonrun/src/async_sys/internal/event_loop/poll/epoll.rs new file mode 100644 index 000000000..ca8e0bd76 --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/event_loop/poll/epoll.rs @@ -0,0 +1,247 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_host::{AsyncHostError, AsyncHostResult}; +use crate::async_sys::internal::fd_util::stub::RawFd; +use crate::async_sys::ported_fns; + +use super::{ + EVENT_BUFFER_SIZE, PROCESS_EVENT, PollEvent, PollInstance, READ_EVENT, WRITE_EVENT, + last_native_error, +}; + +const PID_MASK: u64 = 1 << 63; + +ported_fns! { + #[ported( + source = "src/internal/event_loop/epoll.c", + original = "moonbitlang_async_poll_create" + )] + #[allow(dead_code)] + pub(crate) fn poll_create() -> AsyncHostResult { + let fd = unsafe { libc::epoll_create1(0) }; + if fd < 0 { + Err(last_native_error()) + } else { + Ok(PollInstance { + fd, + events: Vec::new(), + }) + } + } + + #[ported( + source = "src/internal/event_loop/epoll.c", + original = "moonbitlang_async_poll_destroy" + )] + #[allow(dead_code)] + pub(crate) fn poll_destroy(instance: PollInstance) { + drop(instance); + } + + #[ported( + source = "src/internal/event_loop/epoll.c", + original = "moonbitlang_async_poll_register" + )] + #[allow(dead_code)] + pub(crate) fn poll_register( + instance: &PollInstance, + fd: RawFd, + prev_events: i32, + new_events: i32, + oneshot: bool, + ) -> AsyncHostResult<()> { + let mut event = libc::epoll_event { + events: epoll_event_mask(prev_events | new_events, oneshot)?, + u64: fd as u64, + }; + let op = if prev_events == 0 { + libc::EPOLL_CTL_ADD + } else { + libc::EPOLL_CTL_MOD + }; + if unsafe { libc::epoll_ctl(instance.fd, op, fd, &mut event) } < 0 { + Err(last_native_error()) + } else { + Ok(()) + } + } + + #[ported( + source = "src/internal/event_loop/epoll.c", + original = "moonbitlang_async_support_wait_pid_via_poll" + )] + #[allow(dead_code)] + pub(crate) fn support_wait_pid_via_poll() -> bool { + let pidfd = unsafe { libc::syscall(libc::SYS_pidfd_open, std::process::id(), 0) }; + if pidfd >= 0 { + unsafe { libc::close(pidfd as RawFd) }; + true + } else { + false + } + } + + #[ported( + source = "src/internal/event_loop/epoll.c", + original = "moonbitlang_async_poll_register_pid" + )] + #[allow(dead_code)] + pub(crate) fn poll_register_pid(instance: &PollInstance, pid: i32) -> AsyncHostResult { + let pidfd = unsafe { libc::syscall(libc::SYS_pidfd_open, pid, 0) }; + if pidfd < 0 { + return Err(last_native_error()); + } + + let pidfd = pidfd as RawFd; + let mut event = libc::epoll_event { + events: libc::EPOLLIN as u32, + u64: PID_MASK | pidfd as u64, + }; + if unsafe { libc::epoll_ctl(instance.fd, libc::EPOLL_CTL_ADD, pidfd, &mut event) } < 0 { + let error = last_native_error(); + unsafe { libc::close(pidfd) }; + Err(error) + } else { + Ok(pidfd) + } + } + + #[ported( + source = "src/internal/event_loop/epoll.c", + original = "moonbitlang_async_poll_remove" + )] + #[allow(dead_code)] + pub(crate) fn poll_remove(instance: &PollInstance, fd: RawFd, _events: i32) -> AsyncHostResult<()> { + if unsafe { libc::epoll_ctl(instance.fd, libc::EPOLL_CTL_DEL, fd, std::ptr::null_mut()) } < 0 { + Err(last_native_error()) + } else { + Ok(()) + } + } + + #[ported( + source = "src/internal/event_loop/epoll.c", + original = "moonbitlang_async_poll_remove_pid" + )] + #[allow(dead_code)] + pub(crate) fn poll_remove_pid(instance: &PollInstance, pidfd: i32) -> AsyncHostResult<()> { + let ret = unsafe { libc::epoll_ctl(instance.fd, libc::EPOLL_CTL_DEL, pidfd, std::ptr::null_mut()) }; + let error = if ret < 0 { Some(last_native_error()) } else { None }; + unsafe { libc::close(pidfd) }; + match error { + Some(error) => Err(error), + None => Ok(()), + } + } + + #[ported( + source = "src/internal/event_loop/epoll.c", + original = "moonbitlang_async_poll_wait" + )] + #[allow(dead_code)] + pub(crate) fn poll_wait(instance: &mut PollInstance, timeout: i32) -> AsyncHostResult { + let mut events = vec![libc::epoll_event { events: 0, u64: 0 }; EVENT_BUFFER_SIZE]; + let count = unsafe { + libc::epoll_wait( + instance.fd, + events.as_mut_ptr(), + EVENT_BUFFER_SIZE as i32, + timeout, + ) + }; + if count < 0 { + return Err(last_native_error()); + } + instance.events = events + .into_iter() + .take(count as usize) + .map(|event| { + let is_pid = (event.u64 & PID_MASK) != 0; + PollEvent { + fd: (event.u64 & !PID_MASK) as RawFd, + events: if is_pid { + PROCESS_EVENT + } else { + epoll_result_events(event.events) + }, + } + }) + .collect(); + Ok(count) + } + + #[ported( + source = "src/internal/event_loop/epoll.c", + original = "moonbitlang_async_event_list_get" + )] + #[allow(dead_code)] + pub(crate) fn event_list_get(instance: &PollInstance, index: i32) -> AsyncHostResult<&PollEvent> { + let index = usize::try_from(index).map_err(|_| AsyncHostError::Fault)?; + instance.events.get(index).ok_or(AsyncHostError::Fault) + } + + #[ported( + source = "src/internal/event_loop/epoll.c", + original = "moonbitlang_async_event_get_fd" + )] + #[allow(dead_code)] + pub(crate) fn event_get_fd(event: &PollEvent) -> RawFd { + event.fd + } + + #[ported( + source = "src/internal/event_loop/epoll.c", + original = "moonbitlang_async_event_get_events" + )] + #[allow(dead_code)] + pub(crate) fn event_get_events(event: &PollEvent) -> i32 { + event.events + } +} + +fn epoll_event_mask(events: i32, oneshot: bool) -> AsyncHostResult { + let mut mask = match events { + 0 => 0, + READ_EVENT => libc::EPOLLIN, + WRITE_EVENT => libc::EPOLLOUT, + events if events == (READ_EVENT | WRITE_EVENT) => libc::EPOLLIN | libc::EPOLLOUT, + _ => return Err(AsyncHostError::Inval), + }; + if oneshot { + mask |= libc::EPOLLONESHOT; + } + mask |= libc::EPOLLET; + mask |= libc::EPOLLRDHUP; + Ok(mask as u32) +} + +fn epoll_result_events(events: u32) -> i32 { + if (events & (libc::EPOLLERR | libc::EPOLLHUP | libc::EPOLLRDHUP) as u32) != 0 { + return READ_EVENT | WRITE_EVENT; + } + + let mut result = 0; + if (events & libc::EPOLLIN as u32) != 0 { + result |= READ_EVENT; + } + if (events & libc::EPOLLOUT as u32) != 0 { + result |= WRITE_EVENT; + } + result +} diff --git a/crates/moonrun/src/async_sys/internal/event_loop/poll/iocp.rs b/crates/moonrun/src/async_sys/internal/event_loop/poll/iocp.rs new file mode 100644 index 000000000..48fdaddb2 --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/event_loop/poll/iocp.rs @@ -0,0 +1,161 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_host::{AsyncHostError, AsyncHostResult}; +use crate::async_sys::internal::fd_util::stub::RawFd; +use crate::async_sys::ported_fns; + +use super::{EVENT_BUFFER_SIZE, PollEvent, PollInstance, last_errno, last_native_error}; + +ported_fns! { + #[ported( + source = "src/internal/event_loop/iocp.c", + original = "moonbitlang_async_poll_create" + )] + #[allow(dead_code)] + pub(crate) fn poll_create() -> AsyncHostResult { + use windows_sys::Win32::Foundation::INVALID_HANDLE_VALUE; + use windows_sys::Win32::System::IO::CreateIoCompletionPort; + + let fd = unsafe { CreateIoCompletionPort(INVALID_HANDLE_VALUE, std::ptr::null_mut(), 0, 0) }; + if fd.is_null() { + Err(last_native_error()) + } else { + Ok(PollInstance { + fd, + events: Vec::new(), + }) + } + } + + #[ported( + source = "src/internal/event_loop/iocp.c", + original = "moonbitlang_async_poll_destroy" + )] + #[allow(dead_code)] + pub(crate) fn poll_destroy(instance: PollInstance) { + drop(instance); + } + + #[ported( + source = "src/internal/event_loop/iocp.c", + original = "moonbitlang_async_poll_register" + )] + #[allow(dead_code)] + pub(crate) fn poll_register(instance: &PollInstance, fd: RawFd) -> AsyncHostResult<()> { + use windows_sys::Win32::Storage::FileSystem::SetFileCompletionNotificationModes; + use windows_sys::Win32::System::IO::CreateIoCompletionPort; + use windows_sys::Win32::System::WindowsProgramming::FILE_SKIP_COMPLETION_PORT_ON_SUCCESS; + + if unsafe { SetFileCompletionNotificationModes(fd, FILE_SKIP_COMPLETION_PORT_ON_SUCCESS as u8) } == 0 { + return Err(last_native_error()); + } + let registered = + unsafe { CreateIoCompletionPort(fd, instance.fd, fd as usize, 0) }; + if registered.is_null() { + Err(last_native_error()) + } else { + Ok(()) + } + } + + #[ported( + source = "src/internal/event_loop/iocp.c", + original = "moonbitlang_async_poll_wait" + )] + #[allow(dead_code)] + pub(crate) fn poll_wait(instance: &mut PollInstance, timeout: i32) -> AsyncHostResult { + use windows_sys::Win32::Foundation::WAIT_TIMEOUT; + use windows_sys::Win32::System::IO::GetQueuedCompletionStatusEx; + use windows_sys::Win32::System::Threading::INFINITE; + + let mut entries = vec![empty_overlapped_entry(); EVENT_BUFFER_SIZE]; + let mut count = 0; + let ok = unsafe { + GetQueuedCompletionStatusEx( + instance.fd, + entries.as_mut_ptr(), + EVENT_BUFFER_SIZE as u32, + &mut count, + if timeout < 0 { INFINITE } else { timeout as u32 }, + 0, + ) + }; + if ok == 0 { + if last_errno() == WAIT_TIMEOUT as i32 { + instance.events.clear(); + return Ok(0); + } + return Err(last_native_error()); + } + instance.events = entries + .into_iter() + .take(count as usize) + .map(|entry| PollEvent { + fd: entry.lpCompletionKey as RawFd, + events: 0, + io_result: entry.lpOverlapped, + bytes_transferred: entry.dwNumberOfBytesTransferred as i32, + }) + .collect(); + i32::try_from(count).map_err(|_| AsyncHostError::Fault) + } + + #[ported( + source = "src/internal/event_loop/iocp.c", + original = "moonbitlang_async_event_list_get" + )] + #[allow(dead_code)] + pub(crate) fn event_list_get(instance: &PollInstance, index: i32) -> AsyncHostResult<&PollEvent> { + let index = usize::try_from(index).map_err(|_| AsyncHostError::Fault)?; + instance.events.get(index).ok_or(AsyncHostError::Fault) + } + + #[ported( + source = "src/internal/event_loop/iocp.c", + original = "moonbitlang_async_event_get_fd" + )] + #[allow(dead_code)] + pub(crate) fn event_get_fd(event: &PollEvent) -> RawFd { + event.fd + } + + #[ported( + source = "src/internal/event_loop/iocp.c", + original = "moonbitlang_async_event_get_io_result" + )] + #[allow(dead_code)] + pub(crate) fn event_get_io_result( + event: &PollEvent, + ) -> *mut windows_sys::Win32::System::IO::OVERLAPPED { + event.io_result + } + + #[ported( + source = "src/internal/event_loop/iocp.c", + original = "moonbitlang_async_event_get_bytes_transferred" + )] + #[allow(dead_code)] + pub(crate) fn event_get_bytes_transferred(event: &PollEvent) -> i32 { + event.bytes_transferred + } +} + +fn empty_overlapped_entry() -> windows_sys::Win32::System::IO::OVERLAPPED_ENTRY { + unsafe { std::mem::zeroed() } +} diff --git a/crates/moonrun/src/async_sys/internal/event_loop/poll/kqueue.rs b/crates/moonrun/src/async_sys/internal/event_loop/poll/kqueue.rs new file mode 100644 index 000000000..669a4cfce --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/event_loop/poll/kqueue.rs @@ -0,0 +1,276 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_host::{AsyncHostError, AsyncHostResult}; +use crate::async_sys::internal::fd_util::stub::RawFd; +use crate::async_sys::ported_fns; + +use super::{ + EVENT_BUFFER_SIZE, PROCESS_EVENT, PollEvent, PollInstance, READ_EVENT, WRITE_EVENT, last_errno, + last_native_error, +}; + +ported_fns! { + #[ported( + source = "src/internal/event_loop/kqueue.c", + original = "moonbitlang_async_poll_create" + )] + #[allow(dead_code)] + pub(crate) fn poll_create() -> AsyncHostResult { + let fd = unsafe { libc::kqueue() }; + if fd < 0 { + Err(last_native_error()) + } else { + Ok(PollInstance { + fd, + events: Vec::new(), + }) + } + } + + #[ported( + source = "src/internal/event_loop/kqueue.c", + original = "moonbitlang_async_poll_destroy" + )] + #[allow(dead_code)] + pub(crate) fn poll_destroy(instance: PollInstance) { + drop(instance); + } + + #[ported( + source = "src/internal/event_loop/kqueue.c", + original = "moonbitlang_async_poll_register" + )] + #[allow(dead_code)] + pub(crate) fn poll_register( + instance: &PollInstance, + fd: RawFd, + _prev_events: i32, + new_events: i32, + oneshot: bool, + ) -> AsyncHostResult<()> { + let filter = kqueue_event_filter(new_events)?; + let flags = libc::EV_ADD | libc::EV_CLEAR | if oneshot { libc::EV_DISPATCH } else { 0 }; + let event = new_kevent(fd as libc::uintptr_t, filter, flags, 0, 0); + if unsafe { + libc::kevent( + instance.fd, + &event, + 1, + std::ptr::null_mut(), + 0, + std::ptr::null(), + ) + } < 0 + { + Err(last_native_error()) + } else { + Ok(()) + } + } + + #[ported( + source = "src/internal/event_loop/kqueue.c", + original = "moonbitlang_async_support_wait_pid_via_poll" + )] + #[allow(dead_code)] + pub(crate) fn support_wait_pid_via_poll() -> bool { + true + } + + #[ported( + source = "src/internal/event_loop/kqueue.c", + original = "moonbitlang_async_poll_register_pid" + )] + #[allow(dead_code)] + pub(crate) fn poll_register_pid(instance: &PollInstance, pid: i32) -> AsyncHostResult { + let event = new_kevent( + pid as libc::uintptr_t, + libc::EVFILT_PROC, + libc::EV_ADD, + libc::NOTE_EXITSTATUS, + 0, + ); + let ret = unsafe { + libc::kevent( + instance.fd, + &event, + 1, + std::ptr::null_mut(), + 0, + std::ptr::null(), + ) + }; + if ret >= 0 { + Ok(pid) + } else if last_errno() == libc::ESRCH { + Ok(-2) + } else { + Err(last_native_error()) + } + } + + #[ported( + source = "src/internal/event_loop/kqueue.c", + original = "moonbitlang_async_poll_remove" + )] + #[allow(dead_code)] + pub(crate) fn poll_remove(instance: &PollInstance, fd: RawFd, events: i32) -> AsyncHostResult<()> { + let filter = kqueue_event_filter(events)?; + let event = new_kevent(fd as libc::uintptr_t, filter, libc::EV_DELETE, 0, 0); + if unsafe { + libc::kevent( + instance.fd, + &event, + 1, + std::ptr::null_mut(), + 0, + std::ptr::null(), + ) + } < 0 + { + Err(last_native_error()) + } else { + Ok(()) + } + } + + #[ported( + source = "src/internal/event_loop/kqueue.c", + original = "moonbitlang_async_poll_remove_pid" + )] + #[allow(dead_code)] + pub(crate) fn poll_remove_pid(_instance: &PollInstance, _pid: i32) -> AsyncHostResult<()> { + Ok(()) + } + + #[ported( + source = "src/internal/event_loop/kqueue.c", + original = "moonbitlang_async_poll_wait" + )] + #[allow(dead_code)] + pub(crate) fn poll_wait(instance: &mut PollInstance, timeout: i32) -> AsyncHostResult { + let timeout_spec = libc::timespec { + tv_sec: (timeout / 1000) as libc::time_t, + tv_nsec: ((timeout % 1000) * 1_000_000) as libc::c_long, + }; + let mut events = vec![empty_kevent(); EVENT_BUFFER_SIZE]; + let count = unsafe { + libc::kevent( + instance.fd, + std::ptr::null(), + 0, + events.as_mut_ptr(), + EVENT_BUFFER_SIZE as i32, + if timeout < 0 { + std::ptr::null() + } else { + &timeout_spec + }, + ) + }; + if count < 0 { + return Err(last_native_error()); + } + instance.events = events + .into_iter() + .take(count as usize) + .map(|event| PollEvent { + fd: event.ident as RawFd, + events: kqueue_result_events(&event), + }) + .collect(); + Ok(count) + } + + #[ported( + source = "src/internal/event_loop/kqueue.c", + original = "moonbitlang_async_event_list_get" + )] + #[allow(dead_code)] + pub(crate) fn event_list_get(instance: &PollInstance, index: i32) -> AsyncHostResult<&PollEvent> { + let index = usize::try_from(index).map_err(|_| AsyncHostError::Fault)?; + instance.events.get(index).ok_or(AsyncHostError::Fault) + } + + #[ported( + source = "src/internal/event_loop/kqueue.c", + original = "moonbitlang_async_event_get_fd" + )] + #[allow(dead_code)] + pub(crate) fn event_get_fd(event: &PollEvent) -> RawFd { + event.fd + } + + #[ported( + source = "src/internal/event_loop/kqueue.c", + original = "moonbitlang_async_event_get_events" + )] + #[allow(dead_code)] + pub(crate) fn event_get_events(event: &PollEvent) -> i32 { + event.events + } +} + +fn kqueue_event_filter(events: i32) -> AsyncHostResult { + match events { + READ_EVENT => Ok(libc::EVFILT_READ), + WRITE_EVENT => Ok(libc::EVFILT_WRITE), + events if events == (READ_EVENT | WRITE_EVENT) => { + Ok(libc::EVFILT_READ | libc::EVFILT_WRITE) + } + _ => Err(AsyncHostError::Inval), + } +} + +fn kqueue_result_events(event: &libc::kevent) -> i32 { + if event.filter == libc::EVFILT_READ { + return READ_EVENT; + } + if event.filter == libc::EVFILT_WRITE { + return WRITE_EVENT; + } + if event.filter == libc::EVFILT_PROC { + return PROCESS_EVENT; + } + if (event.flags & libc::EV_ERROR) != 0 { + return READ_EVENT | WRITE_EVENT; + } + 0 +} + +fn new_kevent( + ident: libc::uintptr_t, + filter: i16, + flags: u16, + fflags: u32, + data: libc::intptr_t, +) -> libc::kevent { + let mut event = empty_kevent(); + event.ident = ident; + event.filter = filter; + event.flags = flags; + event.fflags = fflags; + event.data = data; + event.udata = std::ptr::null_mut(); + event +} + +fn empty_kevent() -> libc::kevent { + unsafe { std::mem::zeroed() } +} diff --git a/crates/moonrun/src/async_sys/internal/event_loop/poll/mod.rs b/crates/moonrun/src/async_sys/internal/event_loop/poll/mod.rs new file mode 100644 index 000000000..08daadd45 --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/event_loop/poll/mod.rs @@ -0,0 +1,167 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_host::AsyncHostError; +use crate::async_sys::internal::fd_util::stub::RawFd; + +#[cfg(target_os = "linux")] +mod epoll; +#[cfg(windows)] +mod iocp; +#[cfg(target_os = "macos")] +mod kqueue; + +#[cfg(target_os = "linux")] +#[allow(unused_imports)] +pub(crate) use epoll::*; +#[cfg(windows)] +#[allow(unused_imports)] +pub(crate) use iocp::*; +#[cfg(target_os = "macos")] +#[allow(unused_imports)] +pub(crate) use kqueue::*; + +pub(super) const EVENT_BUFFER_SIZE: usize = 1024; +#[allow(dead_code)] +pub(super) const READ_EVENT: i32 = 1; +#[allow(dead_code)] +pub(super) const WRITE_EVENT: i32 = 2; +#[allow(dead_code)] +pub(super) const PROCESS_EVENT: i32 = 4; + +#[cfg(test)] +#[cfg(target_os = "linux")] +pub(crate) const PORTED_SYMBOLS: &[crate::async_sys::PortedSymbol] = epoll::PORTED_SYMBOLS; + +#[cfg(test)] +#[cfg(target_os = "macos")] +pub(crate) const PORTED_SYMBOLS: &[crate::async_sys::PortedSymbol] = kqueue::PORTED_SYMBOLS; + +#[cfg(test)] +#[cfg(windows)] +pub(crate) const PORTED_SYMBOLS: &[crate::async_sys::PortedSymbol] = iocp::PORTED_SYMBOLS; + +#[derive(Debug)] +#[allow(dead_code)] +pub(crate) struct PollInstance { + fd: RawFd, + events: Vec, +} + +impl PollInstance { + #[allow(dead_code)] + pub(crate) fn raw_fd(&self) -> RawFd { + self.fd + } +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +#[allow(dead_code)] +pub(crate) struct PollEvent { + fd: RawFd, + events: i32, + #[cfg(windows)] + io_result: *mut windows_sys::Win32::System::IO::OVERLAPPED, + #[cfg(windows)] + bytes_transferred: i32, +} + +impl Drop for PollInstance { + fn drop(&mut self) { + #[cfg(any(target_os = "linux", target_os = "macos"))] + unsafe { + libc::close(self.fd); + } + #[cfg(windows)] + unsafe { + windows_sys::Win32::Foundation::CloseHandle(self.fd); + } + } +} + +#[cfg(any(target_os = "linux", target_os = "macos"))] +pub(super) fn last_errno() -> i32 { + std::io::Error::last_os_error() + .raw_os_error() + .unwrap_or_else(|| AsyncHostError::Inval.errno()) +} + +#[cfg(any(target_os = "linux", target_os = "macos"))] +pub(super) fn last_native_error() -> AsyncHostError { + AsyncHostError::Native(last_errno()) +} + +#[cfg(windows)] +pub(super) fn last_errno() -> i32 { + std::io::Error::last_os_error() + .raw_os_error() + .unwrap_or_else(|| AsyncHostError::Inval.errno()) +} + +#[cfg(windows)] +pub(super) fn last_native_error() -> AsyncHostError { + AsyncHostError::Native(last_errno()) +} + +// Android would take the native C `__linux__` epoll path, but it is outside the +// V8-backed moonrun async MVP. Keep the cfg split explicit instead of treating +// every Unix-like target as supported. + +#[cfg(test)] +#[cfg(any(target_os = "linux", target_os = "macos"))] +mod tests { + use super::*; + + #[test] + fn event_list_get_rejects_missing_event() { + let poll = poll_create().unwrap(); + + assert_eq!( + event_list_get(&poll, 0).copied(), + Err(AsyncHostError::Fault) + ); + } + + #[test] + fn poll_wait_reports_pipe_readiness() { + let mut fds = [0; 2]; + assert_eq!(unsafe { libc::pipe(fds.as_mut_ptr()) }, 0); + let read_fd = fds[0]; + let write_fd = fds[1]; + + let mut poll = poll_create().unwrap(); + poll_register(&poll, read_fd, 0, READ_EVENT, false).unwrap(); + let byte = b"x"; + assert_eq!( + unsafe { libc::write(write_fd, byte.as_ptr().cast(), byte.len()) }, + 1 + ); + + let count = poll_wait(&mut poll, 100).unwrap(); + assert_eq!(count, 1); + let event = *event_list_get(&poll, 0).unwrap(); + assert_eq!(event_get_fd(&event), read_fd); + assert_eq!(event_get_events(&event) & READ_EVENT, READ_EVENT); + + poll_remove(&poll, read_fd, READ_EVENT).unwrap(); + unsafe { + libc::close(read_fd); + libc::close(write_fd); + } + } +} diff --git a/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/fs.rs b/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/fs.rs new file mode 100644 index 000000000..aa0f3332e --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/fs.rs @@ -0,0 +1,1857 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use std::ffi::OsString; +use std::fs::File; + +use crate::async_host::{AsyncHostError, AsyncHostResult}; +use crate::async_sys::fs::dir::EntryRecord; +use crate::async_sys::internal::fd_util; + +use super::{FileTimeResult, GuestBuffer, HostFile, HostFileTable, HostHandle, OpenJobResult}; + +#[allow(clippy::too_many_arguments)] +pub(super) fn run_open_job( + files: &mut impl HostFileTable, + result: &mut Option, + filename: OsString, + access: i32, + create_mode: i32, + append: bool, + sync: i32, + mode: i32, +) -> AsyncHostResult { + let OpenedFile { + file, + kind, + dev_id, + file_id, + } = open_native_file(filename, access, create_mode, append, sync, mode)?; + let fd = files.insert_file(file)?; + *result = Some(OpenJobResult { + fd, + kind, + dev_id, + file_id, + }); + Ok(0) +} + +pub(super) fn run_read_job( + files: &mut impl HostFileTable, + fd: HostHandle, + dst: GuestBuffer, + position: i64, + result: &mut Option>, +) -> AsyncHostResult { + files.with_file_mut(fd, |file| { + let mut buf = vec![0; usize::try_from(dst.len).map_err(|_| AsyncHostError::Fault)?]; + let n = read_from_native_file(file, &mut buf, position)?; + buf.truncate(n); + *result = Some(buf); + Ok(n as i64) + }) +} + +pub(super) fn run_write_job( + files: &mut impl HostFileTable, + fd: HostHandle, + data: &[u8], + position: i64, +) -> AsyncHostResult { + files.with_file_mut(fd, |file| { + let n = write_to_native_file(file, data, position)?; + Ok(n as i64) + }) +} + +pub(super) fn run_file_kind_by_path_job( + files: &mut impl HostFileTable, + parent: HostHandle, + path: OsString, + follow_symlink: bool, +) -> AsyncHostResult { + file_kind_by_path(files, parent, path, follow_symlink).map(i64::from) +} + +pub(super) fn run_file_size_job( + files: &mut impl HostFileTable, + fd: HostHandle, + result: &mut i64, +) -> AsyncHostResult { + files.with_file_mut(fd, |file| { + *result = file_size(file)?; + Ok(0) + }) +} + +pub(super) fn run_file_time_job( + files: &mut impl HostFileTable, + fd: HostHandle, + result: &mut Option, +) -> AsyncHostResult { + files.with_file_mut(fd, |file| { + *result = Some(FileTimeResult::new(file_time(file)?)); + Ok(0) + }) +} + +pub(super) fn run_file_time_by_path_job( + path: OsString, + follow_symlink: bool, + result: &mut Option, +) -> AsyncHostResult { + *result = Some(FileTimeResult::new(file_time_by_path( + path, + follow_symlink, + )?)); + Ok(0) +} + +pub(super) fn run_access_job(path: OsString, access: i32) -> AsyncHostResult { + access_native_path(path, access)?; + Ok(0) +} + +pub(super) fn run_chmod_job(path: OsString, mode: i32) -> AsyncHostResult { + chmod_native_path(path, mode)?; + Ok(0) +} + +pub(super) fn run_fsync_job( + files: &mut impl HostFileTable, + fd: HostHandle, + only_data: bool, +) -> AsyncHostResult { + files.with_file_mut(fd, |file| { + sync_native_file(file, only_data)?; + Ok(0) + }) +} + +pub(super) fn run_flock_job( + files: &mut impl HostFileTable, + fd: HostHandle, + exclusive: bool, +) -> AsyncHostResult { + #[cfg(windows)] + { + let lock_file = files.with_file_mut(fd, |file| { + let lock_file = file.try_clone().map_err(native_io_error)?; + lock_native_file(&lock_file, exclusive)?; + Ok(lock_file) + })?; + files.with_host_file_mut(fd, |file| { + file.set_lock_file(lock_file); + Ok(0) + }) + } + + #[cfg(not(windows))] + files.with_file_mut(fd, |file| { + lock_native_file(file, exclusive)?; + Ok(0) + }) +} + +pub(super) fn run_remove_job(path: OsString) -> AsyncHostResult { + remove_native_path(path)?; + Ok(0) +} + +pub(super) fn run_rename_job( + old_path: OsString, + new_path: OsString, + replace: bool, +) -> AsyncHostResult { + rename_native_path(old_path, new_path, replace)?; + Ok(0) +} + +pub(super) fn run_symlink_job( + target: OsString, + path: OsString, + force_symlink: bool, +) -> AsyncHostResult { + symlink_native_path(target, path, force_symlink)?; + Ok(0) +} + +pub(super) fn run_mkdir_job(path: OsString, mode: i32) -> AsyncHostResult { + mkdir_native_path(path, mode)?; + Ok(0) +} + +pub(super) fn run_rmdir_job(path: OsString) -> AsyncHostResult { + rmdir_native_path(path)?; + Ok(0) +} + +pub(super) fn run_readdir_job( + files: &mut impl HostFileTable, + dir: HostHandle, + dst: GuestBuffer, + restart: bool, + result: &mut Option>, +) -> AsyncHostResult { + files.with_host_file_mut(dir, |file| { + let len = usize::try_from(dst.len).map_err(|_| AsyncHostError::Fault)?; + let records = read_native_dir(file, len, restart)?; + let ret = i64::try_from(records.len()).map_err(|_| AsyncHostError::Fault)?; + *result = Some(records); + Ok(ret) + }) +} + +#[cfg(unix)] +#[allow(clippy::unnecessary_cast)] +fn open_native_file( + filename: OsString, + access: i32, + create_mode: i32, + append: bool, + sync: i32, + mode: i32, +) -> AsyncHostResult { + use std::ffi::CString; + use std::os::fd::FromRawFd; + use std::os::unix::ffi::OsStringExt; + + let access_flag = match access { + 0 | 3 => libc::O_RDONLY, + 1 => libc::O_WRONLY, + 2 => libc::O_RDWR, + _ => return Err(AsyncHostError::Inval), + }; + let create_flag = match create_mode { + 0 => 0, + 1 => libc::O_TRUNC, + 2 => libc::O_CREAT, + 3 => libc::O_CREAT | libc::O_TRUNC, + 4 => libc::O_CREAT | libc::O_EXCL, + _ => return Err(AsyncHostError::Inval), + }; + let sync_flag = match sync { + 0 => 0, + 1 => libc::O_DSYNC, + 2 => libc::O_SYNC, + _ => return Err(AsyncHostError::Inval), + }; + let append_flag = if append { libc::O_APPEND } else { 0 }; + let filename = CString::new(filename.into_vec()).map_err(|_| AsyncHostError::Inval)?; + let fd = unsafe { + libc::open( + filename.as_ptr(), + access_flag | sync_flag | create_flag | append_flag | libc::O_CLOEXEC, + mode as libc::c_uint, + ) + }; + if fd < 0 { + return Err(last_native_error()); + } + + let mut stat = std::mem::MaybeUninit::::uninit(); + if unsafe { libc::fstat(fd, stat.as_mut_ptr()) } < 0 { + let error = last_native_error(); + unsafe { + libc::close(fd); + } + return Err(error); + } + let stat = unsafe { stat.assume_init() }; + let file = unsafe { File::from_raw_fd(fd) }; + Ok(OpenedFile { + file, + kind: file_kind_from_stat(&stat), + dev_id: stat.st_dev as u64, + file_id: stat.st_ino as u64, + }) +} + +#[cfg(unix)] +fn file_kind_from_stat(stat: &libc::stat) -> i32 { + match stat.st_mode & libc::S_IFMT { + libc::S_IFREG => 1, + libc::S_IFDIR => 2, + libc::S_IFLNK => 3, + libc::S_IFSOCK => 4, + libc::S_IFIFO => 5, + libc::S_IFBLK => 6, + libc::S_IFCHR => 7, + _ => 0, + } +} + +#[cfg(windows)] +fn open_native_file( + filename: OsString, + access: i32, + create_mode: i32, + append: bool, + sync: i32, + _mode: i32, +) -> AsyncHostResult { + use std::os::windows::ffi::OsStrExt; + use std::os::windows::io::FromRawHandle; + use windows_sys::Win32::Foundation::{ + CloseHandle, ERROR_PIPE_BUSY, GENERIC_READ, GENERIC_WRITE, HANDLE, INVALID_HANDLE_VALUE, + }; + use windows_sys::Win32::Storage::FileSystem::{ + BY_HANDLE_FILE_INFORMATION, CREATE_ALWAYS, CREATE_NEW, CreateFileW, FILE_APPEND_DATA, + FILE_ATTRIBUTE_NORMAL, FILE_FLAG_BACKUP_SEMANTICS, FILE_FLAG_OVERLAPPED, + FILE_LIST_DIRECTORY, FILE_SHARE_DELETE, FILE_SHARE_READ, FILE_SHARE_WRITE, + GetFileInformationByHandle, OPEN_ALWAYS, OPEN_EXISTING, TRUNCATE_EXISTING, + }; + use windows_sys::Win32::System::Pipes::{NMPWAIT_WAIT_FOREVER, WaitNamedPipeW}; + + if !(0..=2).contains(&sync) { + return Err(AsyncHostError::Inval); + } + let mut access_flag = match access { + 0 => GENERIC_READ, + 1 => GENERIC_WRITE, + 2 => GENERIC_READ | GENERIC_WRITE, + 3 => FILE_LIST_DIRECTORY, + _ => return Err(AsyncHostError::Inval), + }; + let create_mode = match create_mode { + 0 => OPEN_EXISTING, + 1 => TRUNCATE_EXISTING, + 2 => OPEN_ALWAYS, + 3 => CREATE_ALWAYS, + 4 => CREATE_NEW, + _ => return Err(AsyncHostError::Inval), + }; + if append { + access_flag = (access_flag ^ GENERIC_WRITE) | FILE_APPEND_DATA; + } + let mut flags = FILE_ATTRIBUTE_NORMAL | FILE_FLAG_BACKUP_SEMANTICS; + if access == 3 { + flags |= FILE_FLAG_OVERLAPPED; + } + + let filename = filename + .as_os_str() + .encode_wide() + .chain(std::iter::once(0)) + .collect::>(); + let handle: HANDLE = loop { + let handle = unsafe { + CreateFileW( + filename.as_ptr(), + access_flag, + FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, + std::ptr::null(), + create_mode, + flags, + std::ptr::null_mut(), + ) + }; + if handle != INVALID_HANDLE_VALUE { + break handle; + } + let error = std::io::Error::last_os_error() + .raw_os_error() + .unwrap_or_else(|| AsyncHostError::Inval.errno()); + if error != ERROR_PIPE_BUSY as i32 { + return Err(AsyncHostError::Native(error)); + } + if unsafe { WaitNamedPipeW(filename.as_ptr(), NMPWAIT_WAIT_FOREVER) } == 0 { + return Err(last_native_error()); + } + }; + + let mut info = std::mem::MaybeUninit::::uninit(); + if unsafe { GetFileInformationByHandle(handle, info.as_mut_ptr()) } == 0 { + let error = last_native_error(); + unsafe { + CloseHandle(handle); + } + return Err(error); + } + let info = unsafe { info.assume_init() }; + let file = unsafe { File::from_raw_handle(handle as _) }; + Ok(OpenedFile { + file, + kind: file_kind_from_attr(info.dwFileAttributes), + dev_id: u64::from(info.dwVolumeSerialNumber), + file_id: (u64::from(info.nFileIndexHigh) << 32) | u64::from(info.nFileIndexLow), + }) +} + +#[cfg(windows)] +fn file_kind_from_attr(attrs: u32) -> i32 { + use windows_sys::Win32::Storage::FileSystem::{ + FILE_ATTRIBUTE_DIRECTORY, FILE_ATTRIBUTE_REPARSE_POINT, + }; + + if (attrs & FILE_ATTRIBUTE_REPARSE_POINT) != 0 { + 3 + } else if (attrs & FILE_ATTRIBUTE_DIRECTORY) != 0 { + 2 + } else { + 1 + } +} + +struct OpenedFile { + file: File, + kind: i32, + dev_id: u64, + file_id: u64, +} + +#[cfg(unix)] +fn read_from_native_file(file: &File, buf: &mut [u8], position: i64) -> AsyncHostResult { + use std::os::fd::AsRawFd; + + let ret = if position < 0 { + unsafe { libc::read(file.as_raw_fd(), buf.as_mut_ptr().cast(), buf.len()) } + } else { + unsafe { + libc::pread( + file.as_raw_fd(), + buf.as_mut_ptr().cast(), + buf.len(), + position as libc::off_t, + ) + } + }; + native_io_result(ret) +} + +#[cfg(unix)] +fn write_to_native_file(file: &File, data: &[u8], position: i64) -> AsyncHostResult { + use std::os::fd::AsRawFd; + + let ret = if position < 0 { + unsafe { libc::write(file.as_raw_fd(), data.as_ptr().cast(), data.len()) } + } else { + unsafe { + libc::pwrite( + file.as_raw_fd(), + data.as_ptr().cast(), + data.len(), + position as libc::off_t, + ) + } + }; + native_io_result(ret) +} + +#[cfg(unix)] +fn file_kind_by_path( + files: &mut impl HostFileTable, + parent: HostHandle, + path: OsString, + follow_symlink: bool, +) -> AsyncHostResult { + use std::os::fd::AsRawFd; + + let path = path_to_cstring(path)?; + let flags = if follow_symlink { + 0 + } else { + libc::AT_SYMLINK_NOFOLLOW + }; + let mut stat = std::mem::MaybeUninit::::uninit(); + let ret = if parent < 0 { + unsafe { libc::fstatat(libc::AT_FDCWD, path.as_ptr(), stat.as_mut_ptr(), flags) } + } else { + files.with_file_mut(parent, |file| { + Ok(unsafe { libc::fstatat(file.as_raw_fd(), path.as_ptr(), stat.as_mut_ptr(), flags) }) + })? + }; + if ret < 0 { + Err(last_native_error()) + } else { + Ok(file_kind_from_stat(&unsafe { stat.assume_init() })) + } +} + +#[cfg(unix)] +#[allow(clippy::unnecessary_cast)] +fn file_size(file: &File) -> AsyncHostResult { + use std::os::fd::AsRawFd; + + let mut stat = std::mem::MaybeUninit::::uninit(); + if unsafe { libc::fstat(file.as_raw_fd(), stat.as_mut_ptr()) } < 0 { + return Err(last_native_error()); + } + Ok(unsafe { stat.assume_init() }.st_size as i64) +} + +#[cfg(unix)] +fn file_time(file: &File) -> AsyncHostResult { + use std::os::fd::AsRawFd; + + let mut stat = std::mem::MaybeUninit::::uninit(); + if unsafe { libc::fstat(file.as_raw_fd(), stat.as_mut_ptr()) } < 0 { + return Err(last_native_error()); + } + Ok(unsafe { stat.assume_init() }) +} + +#[cfg(unix)] +fn file_time_by_path( + path: OsString, + follow_symlink: bool, +) -> AsyncHostResult { + let path = path_to_cstring(path)?; + let mut stat = std::mem::MaybeUninit::::uninit(); + let ret = if follow_symlink { + unsafe { libc::stat(path.as_ptr(), stat.as_mut_ptr()) } + } else { + unsafe { libc::lstat(path.as_ptr(), stat.as_mut_ptr()) } + }; + if ret < 0 { + return Err(last_native_error()); + } + Ok(unsafe { stat.assume_init() }) +} + +#[cfg(unix)] +fn access_native_path(path: OsString, access: i32) -> AsyncHostResult<()> { + let path = path_to_cstring(path)?; + let mode = match access { + 0 => libc::F_OK, + 1 => libc::R_OK, + 2 => libc::W_OK, + 3 => libc::X_OK, + _ => return Err(AsyncHostError::Inval), + }; + if unsafe { libc::access(path.as_ptr(), mode) } < 0 { + Err(last_native_error()) + } else { + Ok(()) + } +} + +#[cfg(unix)] +fn chmod_native_path(path: OsString, mode: i32) -> AsyncHostResult<()> { + let path = path_to_cstring(path)?; + if unsafe { libc::chmod(path.as_ptr(), mode as libc::mode_t) } < 0 { + Err(last_native_error()) + } else { + Ok(()) + } +} + +#[cfg(unix)] +fn sync_native_file(file: &File, only_data: bool) -> AsyncHostResult<()> { + use std::os::fd::AsRawFd; + + #[cfg(target_os = "macos")] + let ret = { + let _ = only_data; + unsafe { libc::fsync(file.as_raw_fd()) } + }; + + #[cfg(all(unix, not(target_os = "macos")))] + let ret = unsafe { + if only_data { + libc::fdatasync(file.as_raw_fd()) + } else { + libc::fsync(file.as_raw_fd()) + } + }; + + if ret < 0 { + Err(last_native_error()) + } else { + Ok(()) + } +} + +#[cfg(unix)] +fn lock_native_file(file: &File, exclusive: bool) -> AsyncHostResult<()> { + use std::os::fd::AsRawFd; + + let operation = if exclusive { + libc::LOCK_EX + } else { + libc::LOCK_SH + }; + if unsafe { libc::flock(file.as_raw_fd(), operation) } < 0 { + Err(last_native_error()) + } else { + Ok(()) + } +} + +#[cfg(unix)] +fn remove_native_path(path: OsString) -> AsyncHostResult<()> { + let path = path_to_cstring(path)?; + if unsafe { libc::remove(path.as_ptr()) } < 0 { + Err(last_native_error()) + } else { + Ok(()) + } +} + +#[cfg(unix)] +fn symlink_native_path( + target: OsString, + path: OsString, + _force_symlink: bool, +) -> AsyncHostResult<()> { + let target = path_to_cstring(target)?; + let path = path_to_cstring(path)?; + if unsafe { libc::symlink(target.as_ptr(), path.as_ptr()) } < 0 { + Err(last_native_error()) + } else { + Ok(()) + } +} + +#[cfg(unix)] +fn mkdir_native_path(path: OsString, mode: i32) -> AsyncHostResult<()> { + let path = path_to_cstring(path)?; + if unsafe { libc::mkdir(path.as_ptr(), mode as libc::mode_t) } < 0 { + Err(last_native_error()) + } else { + Ok(()) + } +} + +#[cfg(unix)] +fn rmdir_native_path(path: OsString) -> AsyncHostResult<()> { + let path = path_to_cstring(path)?; + if unsafe { libc::rmdir(path.as_ptr()) } < 0 { + Err(last_native_error()) + } else { + Ok(()) + } +} + +#[cfg(all(unix, target_os = "linux"))] +fn read_native_dir(file: &mut HostFile, len: usize, restart: bool) -> AsyncHostResult> { + use std::os::fd::AsRawFd; + + let fd = file.file_mut().as_raw_fd(); + if restart { + file.pending_dir_entries_mut().clear(); + if unsafe { libc::lseek(fd, 0, libc::SEEK_SET) } < 0 { + return Err(last_native_error()); + } + } + + let mut out = drain_pending_dir_entries(file, len)?; + if !out.is_empty() { + return Ok(out); + } + + let mut native = vec![0; len]; + let ret = unsafe { libc::syscall(libc::SYS_getdents64, fd, native.as_mut_ptr(), native.len()) }; + if ret < 0 { + return Err(last_native_error()); + } + if ret == 0 { + return Ok(out); + } + + let ret = usize::try_from(ret).map_err(|_| AsyncHostError::Fault)?; + let mut offset = 0usize; + while offset < ret { + let reclen_end = offset.checked_add(18).ok_or(AsyncHostError::Fault)?; + let fixed_end = offset.checked_add(19).ok_or(AsyncHostError::Fault)?; + if fixed_end > ret { + return Err(AsyncHostError::Fault); + } + + let file_id = u64::from_le_bytes( + native[offset..offset + 8] + .try_into() + .map_err(|_| AsyncHostError::Fault)?, + ); + let reclen = u16::from_le_bytes( + native[offset + 16..reclen_end] + .try_into() + .map_err(|_| AsyncHostError::Fault)?, + ) as usize; + if reclen == 0 || offset.checked_add(reclen).is_none_or(|end| end > ret) { + return Err(AsyncHostError::Fault); + } + + let d_type = native[offset + 18]; + let name_start = offset + 19; + let name_end = native[name_start..offset + reclen] + .iter() + .position(|byte| *byte == 0) + .map(|pos| name_start + pos) + .ok_or(AsyncHostError::Fault)?; + let native_name = &native[name_start..name_end]; + let name = encode_dir_name_for_wasm_os_string(native_name)?; + file.pending_dir_entries_mut() + .push_back(encode_dir_entry(EntryRecord { + is_hidden: native_name.first() == Some(&b'.'), + is_dir: match d_type { + libc::DT_UNKNOWN => -1, + libc::DT_DIR => 1, + _ => 0, + }, + name, + file_id, + })?); + + offset += reclen; + } + + out = drain_pending_dir_entries(file, len)?; + Ok(out) +} + +#[cfg(all(unix, target_os = "macos"))] +fn read_native_dir(file: &mut HostFile, len: usize, restart: bool) -> AsyncHostResult> { + use std::os::fd::AsRawFd; + + let fd = file.file_mut().as_raw_fd(); + if restart { + file.pending_dir_entries_mut().clear(); + if unsafe { libc::lseek(fd, 0, libc::SEEK_SET) } < 0 { + return Err(last_native_error()); + } + } + + let mut out = drain_pending_dir_entries(file, len)?; + if !out.is_empty() { + return Ok(out); + } + + let mut attr_spec = libc::attrlist { + bitmapcount: libc::ATTR_BIT_MAP_COUNT, + reserved: 0, + commonattr: libc::ATTR_CMN_RETURNED_ATTRS + | libc::ATTR_CMN_NAME + | libc::ATTR_CMN_OBJTYPE + | libc::ATTR_CMN_FILEID, + volattr: 0, + dirattr: 0, + fileattr: 0, + forkattr: 0, + }; + let mut native = vec![0; len]; + let ret = unsafe { + libc::getattrlistbulk( + fd, + (&mut attr_spec as *mut libc::attrlist).cast(), + native.as_mut_ptr().cast(), + native.len(), + 0, + ) + }; + if ret < 0 { + return Err(last_native_error()); + } + if ret == 0 { + return Ok(out); + } + + let mut offset = 0usize; + for _ in 0..ret { + let record_end = offset.checked_add(44).ok_or(AsyncHostError::Fault)?; + if record_end > native.len() { + return Err(AsyncHostError::Fault); + } + + let reclen = u32::from_ne_bytes( + native[offset..offset + 4] + .try_into() + .map_err(|_| AsyncHostError::Fault)?, + ) as usize; + if reclen == 0 + || offset + .checked_add(reclen) + .is_none_or(|end| end > native.len()) + { + return Err(AsyncHostError::Fault); + } + + let commonattr = u32::from_ne_bytes( + native[offset + 4..offset + 8] + .try_into() + .map_err(|_| AsyncHostError::Fault)?, + ); + let name_ref_offset = i32::from_ne_bytes( + native[offset + 24..offset + 28] + .try_into() + .map_err(|_| AsyncHostError::Fault)?, + ); + let name_len = u32::from_ne_bytes( + native[offset + 28..offset + 32] + .try_into() + .map_err(|_| AsyncHostError::Fault)?, + ); + let d_type = i32::from_ne_bytes( + native[offset + 32..offset + 36] + .try_into() + .map_err(|_| AsyncHostError::Fault)?, + ); + let file_id = u64::from_ne_bytes( + native[offset + 36..offset + 44] + .try_into() + .map_err(|_| AsyncHostError::Fault)?, + ); + + let name_ref_base = offset.checked_add(24).ok_or(AsyncHostError::Fault)?; + let name_start = name_ref_base + .checked_add(usize::try_from(name_ref_offset).map_err(|_| AsyncHostError::Fault)?) + .ok_or(AsyncHostError::Fault)?; + let name_len = usize::try_from(name_len).map_err(|_| AsyncHostError::Fault)?; + let name_len = name_len.checked_sub(1).ok_or(AsyncHostError::Fault)?; + let name_end = name_start + .checked_add(name_len) + .ok_or(AsyncHostError::Fault)?; + if name_end > offset + reclen { + return Err(AsyncHostError::Fault); + } + + let native_name = &native[name_start..name_end]; + let name = encode_dir_name_for_wasm_os_string(native_name)?; + // vnode.h defines VNON = 0 and VDIR = 2. libc does not currently expose + // these macOS constants, so keep the native stub's meaning explicit here. + let is_dir = if (commonattr & libc::ATTR_CMN_OBJTYPE) == 0 || d_type == 0 { + -1 + } else if d_type == 2 { + 1 + } else { + 0 + }; + file.pending_dir_entries_mut() + .push_back(encode_dir_entry(EntryRecord { + is_hidden: native_name.first() == Some(&b'.'), + is_dir, + name, + file_id, + })?); + + offset += reclen; + } + + out = drain_pending_dir_entries(file, len)?; + Ok(out) +} + +fn drain_pending_dir_entries(file: &mut HostFile, len: usize) -> AsyncHostResult> { + let mut out = Vec::new(); + while let Some(entry) = file.pending_dir_entries_mut().front() { + if out.len() + entry.len() > len { + if out.is_empty() { + return Err(AsyncHostError::Inval); + } + break; + } + let entry = file + .pending_dir_entries_mut() + .pop_front() + .ok_or(AsyncHostError::Inval)?; + out.extend_from_slice(&entry); + } + Ok(out) +} + +fn encode_dir_entry(entry: EntryRecord) -> AsyncHostResult> { + let mut encoded = Vec::new(); + entry.encode_into(&mut encoded)?; + Ok(encoded) +} + +#[cfg(unix)] +fn encode_dir_name_for_wasm_os_string(native_name: &[u8]) -> AsyncHostResult> { + let name = std::str::from_utf8(native_name).map_err(|_| AsyncHostError::Inval)?; + Ok(name.encode_utf16().flat_map(u16::to_le_bytes).collect()) +} + +#[cfg(windows)] +fn encode_dir_name_for_wasm_os_string(native_name: &[u8]) -> AsyncHostResult> { + if !native_name.len().is_multiple_of(2) { + return Err(AsyncHostError::Fault); + } + Ok(native_name.to_vec()) +} + +#[cfg(all(unix, target_os = "linux"))] +fn rename_native_path( + old_path: OsString, + new_path: OsString, + replace: bool, +) -> AsyncHostResult<()> { + let old_path = path_to_cstring(old_path)?; + let new_path = path_to_cstring(new_path)?; + let flags = if replace { 0 } else { libc::RENAME_NOREPLACE }; + let ret = unsafe { + libc::syscall( + libc::SYS_renameat2, + libc::AT_FDCWD, + old_path.as_ptr(), + libc::AT_FDCWD, + new_path.as_ptr(), + flags, + ) + }; + if ret < 0 { + Err(last_native_error()) + } else { + Ok(()) + } +} + +#[cfg(all(unix, target_os = "macos"))] +fn rename_native_path( + old_path: OsString, + new_path: OsString, + replace: bool, +) -> AsyncHostResult<()> { + let old_path = path_to_cstring(old_path)?; + let new_path = path_to_cstring(new_path)?; + let flags = if replace { 0 } else { libc::RENAME_EXCL }; + let ret = unsafe { + libc::renameatx_np( + libc::AT_FDCWD, + old_path.as_ptr(), + libc::AT_FDCWD, + new_path.as_ptr(), + flags, + ) + }; + if ret < 0 { + Err(last_native_error()) + } else { + Ok(()) + } +} + +#[cfg(unix)] +fn path_to_cstring(path: OsString) -> AsyncHostResult { + use std::os::unix::ffi::OsStringExt; + + std::ffi::CString::new(path.into_vec()).map_err(|_| AsyncHostError::Inval) +} + +#[cfg(unix)] +fn native_io_result(ret: libc::ssize_t) -> AsyncHostResult { + if ret < 0 { + Err(last_native_error()) + } else { + usize::try_from(ret).map_err(|_| AsyncHostError::Fault) + } +} + +#[cfg(windows)] +fn read_from_native_file(file: &File, buf: &mut [u8], position: i64) -> AsyncHostResult { + use std::os::windows::io::AsRawHandle; + use windows_sys::Win32::Foundation::{ERROR_BROKEN_PIPE, ERROR_HANDLE_EOF, HANDLE}; + use windows_sys::Win32::Storage::FileSystem::ReadFile; + use windows_sys::Win32::System::IO::OVERLAPPED; + + let overlapped = std::mem::MaybeUninit::::zeroed(); + let overlapped = unsafe { + let mut overlapped = overlapped.assume_init(); + if position > 0 { + overlapped.Anonymous.Anonymous.Offset = position as u32; + overlapped.Anonymous.Anonymous.OffsetHigh = (position >> 32) as u32; + } + overlapped + }; + let mut overlapped = overlapped; + let mut bytes_transferred = 0; + let overlapped_ptr = if position < 0 { + std::ptr::null_mut() + } else { + &mut overlapped + }; + let handle = file.as_raw_handle() as HANDLE; + // Synchronous Windows file handles can advance the current file pointer + // even when ReadFile receives an OVERLAPPED offset. Keep read_at/pread + // semantics by restoring the original pointer before returning. + let saved_position = if position < 0 { + None + } else { + Some(current_file_pointer(handle)?) + }; + let result = unsafe { + ReadFile( + handle, + buf.as_mut_ptr().cast(), + u32::try_from(buf.len()).map_err(|_| AsyncHostError::Fault)?, + &mut bytes_transferred, + overlapped_ptr, + ) + }; + let result = if result != 0 { + usize::try_from(bytes_transferred).map_err(|_| AsyncHostError::Fault) + } else { + let error = std::io::Error::last_os_error(); + let is_eof = matches!( + error.raw_os_error(), + Some(errno) if errno == ERROR_HANDLE_EOF as i32 || errno == ERROR_BROKEN_PIPE as i32 + ); + if is_eof { + Ok(0) + } else { + Err(native_io_error(error)) + } + }; + if let Some(saved_position) = saved_position + && let Err(restore_error) = restore_file_pointer(handle, saved_position) + { + return match result { + Ok(_) => Err(restore_error), + Err(error) => Err(error), + }; + } + result +} + +#[cfg(windows)] +fn write_to_native_file(file: &File, data: &[u8], position: i64) -> AsyncHostResult { + use std::os::windows::io::AsRawHandle; + use windows_sys::Win32::Foundation::HANDLE; + use windows_sys::Win32::Storage::FileSystem::WriteFile; + use windows_sys::Win32::System::IO::OVERLAPPED; + + let overlapped = std::mem::MaybeUninit::::zeroed(); + let overlapped = unsafe { + let mut overlapped = overlapped.assume_init(); + if position > 0 { + overlapped.Anonymous.Anonymous.Offset = position as u32; + overlapped.Anonymous.Anonymous.OffsetHigh = (position >> 32) as u32; + } + overlapped + }; + let mut overlapped = overlapped; + let mut bytes_transferred = 0; + let overlapped_ptr = if position < 0 { + std::ptr::null_mut() + } else { + &mut overlapped + }; + let handle = file.as_raw_handle() as HANDLE; + // See read_from_native_file: positioned writes must not alter the stream + // offset seen by following non-positioned writes. + let saved_position = if position < 0 { + None + } else { + Some(current_file_pointer(handle)?) + }; + let result = unsafe { + WriteFile( + handle, + data.as_ptr().cast(), + u32::try_from(data.len()).map_err(|_| AsyncHostError::Fault)?, + &mut bytes_transferred, + overlapped_ptr, + ) + }; + let result = if result != 0 { + usize::try_from(bytes_transferred).map_err(|_| AsyncHostError::Fault) + } else { + Err(last_native_error()) + }; + if let Some(saved_position) = saved_position + && let Err(restore_error) = restore_file_pointer(handle, saved_position) + { + return match result { + Ok(_) => Err(restore_error), + Err(error) => Err(error), + }; + } + result +} + +#[cfg(windows)] +fn current_file_pointer(handle: windows_sys::Win32::Foundation::HANDLE) -> AsyncHostResult { + use windows_sys::Win32::Storage::FileSystem::{FILE_CURRENT, SetFilePointerEx}; + + let mut current = 0; + if unsafe { SetFilePointerEx(handle, 0, &mut current, FILE_CURRENT) } == 0 { + Err(last_native_error()) + } else { + Ok(current) + } +} + +#[cfg(windows)] +fn restore_file_pointer( + handle: windows_sys::Win32::Foundation::HANDLE, + position: i64, +) -> AsyncHostResult<()> { + use windows_sys::Win32::Storage::FileSystem::{FILE_BEGIN, SetFilePointerEx}; + + if unsafe { SetFilePointerEx(handle, position, std::ptr::null_mut(), FILE_BEGIN) } == 0 { + Err(last_native_error()) + } else { + Ok(()) + } +} + +#[cfg(windows)] +fn file_kind_by_path( + files: &mut impl HostFileTable, + parent: HostHandle, + path: OsString, + follow_symlink: bool, +) -> AsyncHostResult { + use windows_sys::Win32::Foundation::{CloseHandle, HANDLE, INVALID_HANDLE_VALUE}; + use windows_sys::Win32::Storage::FileSystem::{ + BY_HANDLE_FILE_INFORMATION, CreateFileW, FILE_ATTRIBUTE_NORMAL, FILE_FLAG_BACKUP_SEMANTICS, + FILE_FLAG_OPEN_REPARSE_POINT, FILE_READ_ATTRIBUTES, FILE_SHARE_DELETE, FILE_SHARE_READ, + FILE_SHARE_WRITE, GetFileInformationByHandle, OPEN_EXISTING, + }; + + let mut flags = FILE_ATTRIBUTE_NORMAL | FILE_FLAG_BACKUP_SEMANTICS; + if !follow_symlink { + flags |= FILE_FLAG_OPEN_REPARSE_POINT; + } + let path = resolve_windows_path_for_parent(files, parent, path)?; + let path = path_to_wide(path); + let handle: HANDLE = unsafe { + CreateFileW( + path.as_ptr(), + FILE_READ_ATTRIBUTES, + FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, + std::ptr::null(), + OPEN_EXISTING, + flags, + std::ptr::null_mut(), + ) + }; + if handle == INVALID_HANDLE_VALUE { + return Err(last_native_error()); + } + let mut info = std::mem::MaybeUninit::::uninit(); + if unsafe { GetFileInformationByHandle(handle, info.as_mut_ptr()) } == 0 { + let error = last_native_error(); + unsafe { + CloseHandle(handle); + } + return Err(error); + } + unsafe { + CloseHandle(handle); + } + Ok(file_kind_from_attr( + unsafe { info.assume_init() }.dwFileAttributes, + )) +} + +#[cfg(windows)] +fn resolve_windows_path_for_parent( + files: &mut impl HostFileTable, + parent: HostHandle, + path: OsString, +) -> AsyncHostResult { + if parent < 0 || std::path::Path::new(&path).is_absolute() { + return Ok(path); + } + + files.with_file_mut(parent, |file| { + let mut parent_path = std::path::PathBuf::from(final_path_from_handle(file)?); + parent_path.push(path); + Ok(parent_path.into_os_string()) + }) +} + +#[cfg(windows)] +fn final_path_from_handle(file: &File) -> AsyncHostResult { + use std::os::windows::ffi::OsStringExt; + use std::os::windows::io::AsRawHandle; + use windows_sys::Win32::Storage::FileSystem::{ + FILE_NAME_NORMALIZED, GetFinalPathNameByHandleW, VOLUME_NAME_DOS, + }; + + let mut buffer = vec![0u16; 260]; + loop { + let len = unsafe { + GetFinalPathNameByHandleW( + file.as_raw_handle(), + buffer.as_mut_ptr(), + u32::try_from(buffer.len()).map_err(|_| AsyncHostError::Fault)?, + FILE_NAME_NORMALIZED | VOLUME_NAME_DOS, + ) + }; + if len == 0 { + return Err(last_native_error()); + } + let len = usize::try_from(len).map_err(|_| AsyncHostError::Fault)?; + if len < buffer.len() { + buffer.truncate(len); + return Ok(OsString::from_wide(&buffer)); + } + buffer.resize(len + 1, 0); + } +} + +#[cfg(windows)] +fn file_size(file: &File) -> AsyncHostResult { + use std::os::windows::io::AsRawHandle; + use windows_sys::Win32::Foundation::HANDLE; + use windows_sys::Win32::Storage::FileSystem::GetFileSizeEx; + + let mut size = std::mem::MaybeUninit::::uninit(); + let result = unsafe { GetFileSizeEx(file.as_raw_handle() as HANDLE, size.as_mut_ptr()) }; + if result == 0 { + Err(last_native_error()) + } else { + Ok(unsafe { size.assume_init() }) + } +} + +#[cfg(windows)] +fn file_time(file: &File) -> AsyncHostResult { + use std::os::windows::io::AsRawHandle; + use windows_sys::Win32::Foundation::HANDLE; + use windows_sys::Win32::Storage::FileSystem::{ + FILE_BASIC_INFO, FileBasicInfo, GetFileInformationByHandleEx, + }; + + let mut info = std::mem::MaybeUninit::::uninit(); + let ok = unsafe { + GetFileInformationByHandleEx( + file.as_raw_handle() as HANDLE, + FileBasicInfo, + info.as_mut_ptr().cast(), + std::mem::size_of::() as u32, + ) + }; + if ok == 0 { + return Err(last_native_error()); + } + Ok(unsafe { info.assume_init() }) +} + +#[cfg(windows)] +fn file_time_by_path( + path: OsString, + follow_symlink: bool, +) -> AsyncHostResult { + use windows_sys::Win32::Foundation::{CloseHandle, HANDLE, INVALID_HANDLE_VALUE}; + use windows_sys::Win32::Storage::FileSystem::{ + CreateFileW, FILE_ATTRIBUTE_NORMAL, FILE_BASIC_INFO, FILE_FLAG_BACKUP_SEMANTICS, + FILE_FLAG_OPEN_REPARSE_POINT, FILE_READ_ATTRIBUTES, FILE_SHARE_DELETE, FILE_SHARE_READ, + FILE_SHARE_WRITE, FileBasicInfo, GetFileInformationByHandleEx, OPEN_EXISTING, + }; + + let mut flags = FILE_ATTRIBUTE_NORMAL | FILE_FLAG_BACKUP_SEMANTICS; + if !follow_symlink { + flags |= FILE_FLAG_OPEN_REPARSE_POINT; + } + let path = path_to_wide(path); + let handle: HANDLE = unsafe { + CreateFileW( + path.as_ptr(), + FILE_READ_ATTRIBUTES, + FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, + std::ptr::null(), + OPEN_EXISTING, + flags, + std::ptr::null_mut(), + ) + }; + if handle == INVALID_HANDLE_VALUE { + return Err(last_native_error()); + } + + let mut info = std::mem::MaybeUninit::::uninit(); + let ok = unsafe { + GetFileInformationByHandleEx( + handle, + FileBasicInfo, + info.as_mut_ptr().cast(), + std::mem::size_of::() as u32, + ) + }; + unsafe { + CloseHandle(handle); + } + if ok == 0 { + return Err(last_native_error()); + } + Ok(unsafe { info.assume_init() }) +} + +#[cfg(windows)] +fn access_native_path(path: OsString, access: i32) -> AsyncHostResult<()> { + use windows_sys::Win32::Foundation::{ + CloseHandle, GENERIC_READ, GENERIC_WRITE, INVALID_HANDLE_VALUE, + }; + use windows_sys::Win32::Storage::FileSystem::{ + CreateFileW, FILE_ATTRIBUTE_NORMAL, FILE_EXECUTE, FILE_FLAG_BACKUP_SEMANTICS, + FILE_SHARE_DELETE, FILE_SHARE_READ, FILE_SHARE_WRITE, OPEN_EXISTING, + }; + + let access_mode = match access { + 0 => 0, + 1 => GENERIC_READ, + 2 => GENERIC_WRITE, + 3 => FILE_EXECUTE, + _ => return Err(AsyncHostError::Inval), + }; + let path = path_to_wide(path); + let handle = unsafe { + CreateFileW( + path.as_ptr(), + access_mode, + FILE_SHARE_DELETE | FILE_SHARE_READ | FILE_SHARE_WRITE, + std::ptr::null(), + OPEN_EXISTING, + FILE_ATTRIBUTE_NORMAL | FILE_FLAG_BACKUP_SEMANTICS, + std::ptr::null_mut(), + ) + }; + if handle == INVALID_HANDLE_VALUE { + Err(last_native_error()) + } else { + unsafe { + CloseHandle(handle); + } + Ok(()) + } +} + +#[cfg(windows)] +fn chmod_native_path(_path: OsString, _mode: i32) -> AsyncHostResult<()> { + Err(AsyncHostError::NotSupported) +} + +#[cfg(windows)] +fn sync_native_file(file: &File, _only_data: bool) -> AsyncHostResult<()> { + use std::os::windows::io::AsRawHandle; + use windows_sys::Win32::Foundation::HANDLE; + use windows_sys::Win32::Storage::FileSystem::FlushFileBuffers; + + if unsafe { FlushFileBuffers(file.as_raw_handle() as HANDLE) } == 0 { + Err(last_native_error()) + } else { + Ok(()) + } +} + +#[cfg(windows)] +fn lock_native_file(file: &File, exclusive: bool) -> AsyncHostResult<()> { + use std::mem::zeroed; + use std::os::windows::io::AsRawHandle; + use windows_sys::Win32::Storage::FileSystem::{LOCKFILE_EXCLUSIVE_LOCK, LockFileEx}; + use windows_sys::Win32::System::IO::OVERLAPPED; + + let mut overlapped: OVERLAPPED = unsafe { zeroed() }; + // Keep parity with thread_pool.c: lock a one-byte sentinel range beyond + // ordinary file data so Windows mandatory locks approximate advisory locks. + overlapped.Anonymous.Anonymous.Offset = 0xfffffffe; + overlapped.Anonymous.Anonymous.OffsetHigh = 0xffffffff; + let flags = if exclusive { + LOCKFILE_EXCLUSIVE_LOCK + } else { + 0 + }; + if unsafe { LockFileEx(file.as_raw_handle(), flags, 0, 1, 0, &mut overlapped) } == 0 { + Err(last_native_error()) + } else { + Ok(()) + } +} + +#[cfg(windows)] +fn remove_native_path(path: OsString) -> AsyncHostResult<()> { + use windows_sys::Win32::Storage::FileSystem::{ + DeleteFileW, FILE_ATTRIBUTE_DIRECTORY, FILE_ATTRIBUTE_REPARSE_POINT, GetFileAttributesW, + INVALID_FILE_ATTRIBUTES, RemoveDirectoryW, + }; + + let path = path_to_wide(path); + let attrs = unsafe { GetFileAttributesW(path.as_ptr()) }; + if attrs == INVALID_FILE_ATTRIBUTES { + return Err(last_native_error()); + } + + let is_directory_link = + (attrs & FILE_ATTRIBUTE_DIRECTORY) != 0 && (attrs & FILE_ATTRIBUTE_REPARSE_POINT) != 0; + let ok = if is_directory_link { + // Windows removes directory symlinks and junctions through RemoveDirectoryW, + // not DeleteFileW, even though they are reparse points rather than real dirs. + unsafe { RemoveDirectoryW(path.as_ptr()) } + } else { + unsafe { DeleteFileW(path.as_ptr()) } + }; + if ok == 0 { + Err(last_native_error()) + } else { + Ok(()) + } +} + +#[cfg(windows)] +fn symlink_native_path( + target: OsString, + path: OsString, + force_symlink: bool, +) -> AsyncHostResult<()> { + use windows_sys::Win32::Foundation::ERROR_INVALID_PARAMETER; + use windows_sys::Win32::Storage::FileSystem::{ + CreateSymbolicLinkW, FILE_ATTRIBUTE_DIRECTORY, GetFileAttributesW, INVALID_FILE_ATTRIBUTES, + SYMBOLIC_LINK_FLAG_DIRECTORY, + }; + + let target_wide = path_to_wide(target.clone()); + let path_wide = path_to_wide(path.clone()); + let attrs = unsafe { GetFileAttributesW(target_wide.as_ptr()) }; + let is_directory = attrs != INVALID_FILE_ATTRIBUTES && (attrs & FILE_ATTRIBUTE_DIRECTORY) != 0; + + if !force_symlink && is_directory && std::path::Path::new(target.as_os_str()).is_absolute() { + match create_junction_native_path(target.clone(), path.clone()) { + Ok(()) => return Ok(()), + Err(AsyncHostError::Native(error)) if error == ERROR_INVALID_PARAMETER as i32 => {} + Err(error) => return Err(error), + } + } + + let flags = if is_directory { + SYMBOLIC_LINK_FLAG_DIRECTORY + } else { + 0 + }; + if unsafe { CreateSymbolicLinkW(path_wide.as_ptr(), target_wide.as_ptr(), flags) } != 0 { + Ok(()) + } else { + Err(last_native_error()) + } +} + +#[cfg(windows)] +fn create_junction_native_path(target: OsString, path: OsString) -> AsyncHostResult<()> { + use windows_sys::Win32::Foundation::{CloseHandle, INVALID_HANDLE_VALUE}; + use windows_sys::Win32::Storage::FileSystem::{ + CreateDirectoryW, CreateFileW, FILE_FLAG_BACKUP_SEMANTICS, FILE_FLAG_OPEN_REPARSE_POINT, + FILE_SHARE_DELETE, FILE_SHARE_READ, FILE_SHARE_WRITE, FILE_WRITE_ATTRIBUTES, OPEN_EXISTING, + RemoveDirectoryW, + }; + use windows_sys::Win32::System::IO::DeviceIoControl; + use windows_sys::Win32::System::Ioctl::FSCTL_SET_REPARSE_POINT; + + let data = junction_reparse_buffer(target)?; + let path_wide = path_to_wide(path); + if unsafe { CreateDirectoryW(path_wide.as_ptr(), std::ptr::null()) } == 0 { + return Err(last_native_error()); + } + + let handle = unsafe { + CreateFileW( + path_wide.as_ptr(), + FILE_WRITE_ATTRIBUTES, + FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, + std::ptr::null(), + OPEN_EXISTING, + FILE_FLAG_OPEN_REPARSE_POINT | FILE_FLAG_BACKUP_SEMANTICS, + std::ptr::null_mut(), + ) + }; + if handle == INVALID_HANDLE_VALUE { + let error = last_native_error(); + unsafe { + RemoveDirectoryW(path_wide.as_ptr()); + } + return Err(error); + } + + let mut bytes_returned = 0; + let ok = unsafe { + DeviceIoControl( + handle, + FSCTL_SET_REPARSE_POINT, + data.as_ptr().cast(), + data.len() as u32, + std::ptr::null_mut(), + 0, + &mut bytes_returned, + std::ptr::null_mut(), + ) + }; + unsafe { + CloseHandle(handle); + } + if ok == 0 { + let error = last_native_error(); + unsafe { + RemoveDirectoryW(path_wide.as_ptr()); + } + Err(error) + } else { + Ok(()) + } +} + +#[cfg(windows)] +fn junction_reparse_buffer(target: OsString) -> AsyncHostResult> { + use windows_sys::Win32::System::SystemServices::IO_REPARSE_TAG_MOUNT_POINT; + + const MOUNT_POINT_HEADER_LEN: usize = 8; + const WCHAR_SIZE: usize = 2; + const UNICODE_NULL_SIZE: usize = WCHAR_SIZE; + const NON_INTERPRETED_PATH_PREFIX: [u16; 4] = + ['\\' as u16, '?' as u16, '?' as u16, '\\' as u16]; + + let mut print_name = os_string_to_wide(target); + if print_name.starts_with(&NON_INTERPRETED_PATH_PREFIX) + || print_name.starts_with(&['\\' as u16, '\\' as u16, '?' as u16, '\\' as u16]) + { + print_name.drain(0..NON_INTERPRETED_PATH_PREFIX.len()); + } + for code_unit in &mut print_name { + if *code_unit == '/' as u16 { + *code_unit = '\\' as u16; + } + } + + let mut substitute = Vec::from(NON_INTERPRETED_PATH_PREFIX); + substitute.extend(print_name.iter().copied()); + let substitute_len = substitute + .len() + .checked_mul(WCHAR_SIZE) + .ok_or(AsyncHostError::Inval)?; + let substitute_len = u16::try_from(substitute_len).map_err(|_| AsyncHostError::Inval)?; + let print_name_len = print_name + .len() + .checked_mul(WCHAR_SIZE) + .ok_or(AsyncHostError::Inval)?; + let print_name_len = u16::try_from(print_name_len).map_err(|_| AsyncHostError::Inval)?; + let print_name_offset = substitute_len + .checked_add(UNICODE_NULL_SIZE as u16) + .ok_or(AsyncHostError::Inval)?; + let reparse_data_len = (MOUNT_POINT_HEADER_LEN as u16) + .checked_add(print_name_offset) + .and_then(|len| len.checked_add(print_name_len)) + .and_then(|len| len.checked_add(UNICODE_NULL_SIZE as u16)) + .ok_or(AsyncHostError::Inval)?; + + let mut data = Vec::with_capacity(8 + usize::from(reparse_data_len)); + data.extend_from_slice(&IO_REPARSE_TAG_MOUNT_POINT.to_le_bytes()); + data.extend_from_slice(&reparse_data_len.to_le_bytes()); + data.extend_from_slice(&0u16.to_le_bytes()); + data.extend_from_slice(&0u16.to_le_bytes()); + data.extend_from_slice(&substitute_len.to_le_bytes()); + data.extend_from_slice(&print_name_offset.to_le_bytes()); + data.extend_from_slice(&print_name_len.to_le_bytes()); + for code_unit in substitute { + data.extend_from_slice(&code_unit.to_le_bytes()); + } + data.extend_from_slice(&0u16.to_le_bytes()); + for code_unit in print_name { + data.extend_from_slice(&code_unit.to_le_bytes()); + } + data.extend_from_slice(&0u16.to_le_bytes()); + Ok(data) +} + +#[cfg(windows)] +fn mkdir_native_path(path: OsString, _mode: i32) -> AsyncHostResult<()> { + use windows_sys::Win32::Storage::FileSystem::CreateDirectoryW; + + let path = path_to_wide(path); + if unsafe { CreateDirectoryW(path.as_ptr(), std::ptr::null()) } == 0 { + Err(last_native_error()) + } else { + Ok(()) + } +} + +#[cfg(windows)] +fn rmdir_native_path(path: OsString) -> AsyncHostResult<()> { + use windows_sys::Win32::Storage::FileSystem::RemoveDirectoryW; + + let path = path_to_wide(path); + if unsafe { RemoveDirectoryW(path.as_ptr()) } == 0 { + Err(last_native_error()) + } else { + Ok(()) + } +} + +#[cfg(windows)] +fn read_native_dir(file: &mut HostFile, len: usize, restart: bool) -> AsyncHostResult> { + use std::os::windows::io::AsRawHandle; + use windows_sys::Win32::Foundation::{ERROR_NO_MORE_FILES, HANDLE}; + use windows_sys::Win32::Storage::FileSystem::{ + FILE_ATTRIBUTE_DIRECTORY, FILE_ATTRIBUTE_HIDDEN, FILE_ATTRIBUTE_REPARSE_POINT, + FILE_ID_BOTH_DIR_INFO, FileIdBothDirectoryInfo, FileIdBothDirectoryRestartInfo, + GetFileInformationByHandleEx, + }; + + if restart { + file.pending_dir_entries_mut().clear(); + } + + let mut out = drain_pending_dir_entries(file, len)?; + if !out.is_empty() { + return Ok(out); + } + + let mut native = vec![0; len]; + let info_class = if restart { + FileIdBothDirectoryRestartInfo + } else { + FileIdBothDirectoryInfo + }; + let ok = unsafe { + GetFileInformationByHandleEx( + file.file_mut().as_raw_handle() as HANDLE, + info_class, + native.as_mut_ptr().cast(), + u32::try_from(native.len()).map_err(|_| AsyncHostError::Fault)?, + ) + }; + if ok == 0 { + let error = std::io::Error::last_os_error() + .raw_os_error() + .unwrap_or_else(|| AsyncHostError::Inval.errno()); + if error == ERROR_NO_MORE_FILES as i32 { + return Ok(out); + } + return Err(AsyncHostError::Native(error)); + } + + let mut offset = 0usize; + loop { + let fixed_end = offset + .checked_add(std::mem::size_of::()) + .ok_or(AsyncHostError::Fault)?; + if fixed_end > native.len() { + return Err(AsyncHostError::Fault); + } + + let entry = unsafe { + std::ptr::read_unaligned(native.as_ptr().add(offset).cast::()) + }; + let name_len = usize::try_from(entry.FileNameLength).map_err(|_| AsyncHostError::Fault)?; + let name_start = offset + std::mem::offset_of!(FILE_ID_BOTH_DIR_INFO, FileName); + let name_end = name_start + .checked_add(name_len) + .ok_or(AsyncHostError::Fault)?; + if name_end > native.len() { + return Err(AsyncHostError::Fault); + } + + let name = encode_dir_name_for_wasm_os_string(&native[name_start..name_end])?; + let is_dir = if (entry.FileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) == 0 + && (entry.FileAttributes & FILE_ATTRIBUTE_DIRECTORY) != 0 + { + 1 + } else { + 0 + }; + file.pending_dir_entries_mut() + .push_back(encode_dir_entry(EntryRecord { + is_hidden: (entry.FileAttributes & FILE_ATTRIBUTE_HIDDEN) != 0, + is_dir, + name, + file_id: entry.FileId as u64, + })?); + + let next = usize::try_from(entry.NextEntryOffset).map_err(|_| AsyncHostError::Fault)?; + if next == 0 { + break; + } + offset = offset.checked_add(next).ok_or(AsyncHostError::Fault)?; + } + + out = drain_pending_dir_entries(file, len)?; + Ok(out) +} + +#[cfg(windows)] +fn rename_native_path( + old_path: OsString, + new_path: OsString, + replace: bool, +) -> AsyncHostResult<()> { + use windows_sys::Win32::Foundation::{ + CloseHandle, ERROR_INVALID_PARAMETER, HANDLE, INVALID_HANDLE_VALUE, + }; + use windows_sys::Win32::Storage::FileSystem::{ + CreateFileW, DELETE, FILE_ATTRIBUTE_NORMAL, FILE_FLAG_BACKUP_SEMANTICS, FILE_RENAME_INFO, + FILE_SHARE_DELETE, FILE_SHARE_READ, FILE_SHARE_WRITE, FileRenameInfoEx, + MOVEFILE_COPY_ALLOWED, MOVEFILE_REPLACE_EXISTING, MoveFileExW, OPEN_EXISTING, + SetFileInformationByHandle, + }; + + let old_path = path_to_wide(old_path); + let new_path = path_to_wide(new_path); + let handle: HANDLE = unsafe { + CreateFileW( + old_path.as_ptr(), + DELETE, + FILE_SHARE_DELETE | FILE_SHARE_READ | FILE_SHARE_WRITE, + std::ptr::null(), + OPEN_EXISTING, + FILE_ATTRIBUTE_NORMAL | FILE_FLAG_BACKUP_SEMANTICS, + std::ptr::null_mut(), + ) + }; + if handle == INVALID_HANDLE_VALUE { + return Err(last_native_error()); + } + + let filename_len = new_path.len().checked_sub(1).ok_or(AsyncHostError::Inval)?; + let buffer_size = std::mem::size_of::() + .checked_add(filename_len * std::mem::size_of::()) + .ok_or(AsyncHostError::Fault)?; + let mut buffer = vec![0u8; buffer_size]; + let info = buffer.as_mut_ptr().cast::(); + unsafe { + (*info).Anonymous.Flags = if replace { 3 } else { 0 }; + (*info).RootDirectory = std::ptr::null_mut(); + (*info).FileNameLength = u32::try_from(filename_len * std::mem::size_of::()) + .map_err(|_| AsyncHostError::Fault)?; + std::ptr::copy_nonoverlapping( + new_path.as_ptr(), + (*info).FileName.as_mut_ptr(), + filename_len, + ); + } + + let ret = unsafe { + SetFileInformationByHandle(handle, FileRenameInfoEx, info.cast(), buffer_size as u32) + }; + unsafe { + CloseHandle(handle); + } + if ret != 0 { + return Ok(()); + } + + let error = std::io::Error::last_os_error() + .raw_os_error() + .unwrap_or_else(|| AsyncHostError::Inval.errno()); + if error != ERROR_INVALID_PARAMETER as i32 { + return Err(AsyncHostError::Native(error)); + } + + let flags = MOVEFILE_COPY_ALLOWED + | if replace { + MOVEFILE_REPLACE_EXISTING + } else { + 0 + }; + if unsafe { MoveFileExW(old_path.as_ptr(), new_path.as_ptr(), flags) } == 0 { + Err(last_native_error()) + } else { + Ok(()) + } +} + +#[cfg(windows)] +fn path_to_wide(path: OsString) -> Vec { + os_string_to_wide(path) + .into_iter() + .chain(std::iter::once(0)) + .collect() +} + +#[cfg(windows)] +fn os_string_to_wide(path: OsString) -> Vec { + use std::os::windows::ffi::OsStrExt; + + path.as_os_str().encode_wide().collect() +} + +fn last_native_error() -> AsyncHostError { + AsyncHostError::Native( + std::io::Error::last_os_error() + .raw_os_error() + .unwrap_or_else(|| AsyncHostError::Inval.errno()), + ) +} + +#[cfg(windows)] +fn native_io_error(error: std::io::Error) -> AsyncHostError { + AsyncHostError::Native( + error + .raw_os_error() + .unwrap_or_else(|| AsyncHostError::Inval.errno()), + ) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[cfg(unix)] + #[test] + fn dir_name_for_wasm_os_string_encodes_utf16le_on_unix() { + assert_eq!( + encode_dir_name_for_wasm_os_string("file".as_bytes()).unwrap(), + [b'f', 0, b'i', 0, b'l', 0, b'e', 0] + ); + assert_eq!( + encode_dir_name_for_wasm_os_string("\u{6587}".as_bytes()).unwrap(), + [0x87, 0x65] + ); + } + + #[cfg(unix)] + #[test] + fn dir_name_for_wasm_os_string_rejects_non_utf8_unix_name() { + assert_eq!( + encode_dir_name_for_wasm_os_string(b"\xff"), + Err(AsyncHostError::Inval) + ); + } + + #[cfg(windows)] + #[test] + fn dir_name_for_wasm_os_string_preserves_windows_utf16_bytes() { + assert_eq!( + encode_dir_name_for_wasm_os_string(&[b'f', 0, b'i', 0]).unwrap(), + [b'f', 0, b'i', 0] + ); + } + + #[cfg(windows)] + #[test] + fn dir_name_for_wasm_os_string_rejects_odd_windows_name_bytes() { + assert_eq!( + encode_dir_name_for_wasm_os_string(b"f"), + Err(AsyncHostError::Fault) + ); + } + + #[cfg(windows)] + #[test] + fn junction_reparse_buffer_encodes_substitute_and_print_names() { + let data = junction_reparse_buffer(OsString::from("C:/target")).unwrap(); + let reparse_data_len = u16::from_le_bytes([data[4], data[5]]) as usize; + let substitute_len = u16::from_le_bytes([data[10], data[11]]) as usize; + let print_name_offset = u16::from_le_bytes([data[12], data[13]]) as usize; + let print_name_len = u16::from_le_bytes([data[14], data[15]]) as usize; + + assert_eq!(data.len(), 8 + reparse_data_len); + assert_eq!(print_name_offset, substitute_len + 2); + + let path_buffer = &data[16..]; + let substitute = path_buffer[..substitute_len] + .chunks_exact(2) + .map(|chunk| u16::from_le_bytes([chunk[0], chunk[1]])) + .collect::>(); + let print_name = path_buffer[print_name_offset..print_name_offset + print_name_len] + .chunks_exact(2) + .map(|chunk| u16::from_le_bytes([chunk[0], chunk[1]])) + .collect::>(); + + assert_eq!(String::from_utf16(&substitute).unwrap(), r"\??\C:\target"); + assert_eq!(String::from_utf16(&print_name).unwrap(), r"C:\target"); + } +} diff --git a/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/jobs.rs b/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/jobs.rs new file mode 100644 index 000000000..c3afcb56c --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/jobs.rs @@ -0,0 +1,438 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use std::ffi::OsString; + +use crate::async_host::{AsyncHostError, AsyncHostResult}; +use crate::async_sys::ported_fns; + +use super::process::HostProcess; +use super::types::{GuestBuffer, HostHandle, Job, JobPayload, OpenJobResult, platform}; + +ported_fns! { + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_get_platform" + )] + pub(crate) fn get_platform() -> i32 { + platform() + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_job_get_ret" + )] + pub(crate) fn job_get_ret(job: &Job) -> i64 { + job.ret() + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_job_get_err" + )] + pub(crate) fn job_get_err(job: &Job) -> i32 { + job.err() + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_errno_is_cancelled" + )] + pub(crate) fn errno_is_cancelled(errno: i32) -> bool { + #[cfg(windows)] + { + use windows_sys::Win32::Foundation::ERROR_OPERATION_ABORTED; + errno == ERROR_OPERATION_ABORTED as i32 + } + #[cfg(unix)] + { + errno == libc::EINTR + } + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_make_sleep_job" + )] + pub(crate) fn make_sleep_job(ms: i32) -> Job { + Job::new(JobPayload::Sleep { duration_ms: ms }) + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_make_read_job" + )] + pub(crate) fn make_read_job( + fd: HostHandle, + ptr: i32, + offset: i32, + len: i32, + position: i64, + ) -> Job { + Job::new(JobPayload::Read { + fd, + dst: GuestBuffer::new(ptr, offset, len), + position, + result: None, + }) + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_make_write_job" + )] + pub(crate) fn make_write_job(fd: HostHandle, data: Vec, position: i64) -> Job { + Job::new(JobPayload::Write { fd, data, position }) + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_make_open_job" + )] + pub(crate) fn make_open_job( + filename: OsString, + access: i32, + create_mode: i32, + append: bool, + sync: i32, + mode: i32, + ) -> Job { + Job::new(JobPayload::Open { + filename, + access, + create_mode, + append, + sync, + mode, + result: None, + }) + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_make_file_kind_by_path_job" + )] + pub(crate) fn make_file_kind_by_path_job( + parent: HostHandle, + path: OsString, + follow_symlink: bool, + ) -> Job { + Job::new(JobPayload::FileKindByPath { + parent, + path, + follow_symlink, + }) + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_open_job_get_fd" + )] + pub(crate) fn open_job_get_fd(result: &OpenJobResult) -> HostHandle { + result.fd + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_open_job_get_kind" + )] + pub(crate) fn open_job_get_kind(result: &OpenJobResult) -> i32 { + result.kind + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_open_job_get_dev_id" + )] + pub(crate) fn open_job_get_dev_id(result: &OpenJobResult) -> u64 { + result.dev_id + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_open_job_get_file_id" + )] + pub(crate) fn open_job_get_file_id(result: &OpenJobResult) -> u64 { + result.file_id + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_make_file_size_job" + )] + pub(crate) fn make_file_size_job(fd: HostHandle) -> Job { + Job::new(JobPayload::FileSize { fd, result: 0 }) + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_make_file_time_job" + )] + pub(crate) fn make_file_time_job(fd: HostHandle, out: i32, out_len: i32) -> Job { + Job::new(JobPayload::FileTime { + fd, + out: GuestBuffer::new(out, 0, out_len), + result: None, + }) + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_make_file_time_by_path_job" + )] + pub(crate) fn make_file_time_by_path_job( + path: OsString, + out: i32, + out_len: i32, + follow_symlink: bool, + ) -> Job { + Job::new(JobPayload::FileTimeByPath { + path, + out: GuestBuffer::new(out, 0, out_len), + follow_symlink, + result: None, + }) + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_make_access_job" + )] + pub(crate) fn make_access_job(path: OsString, access: i32) -> Job { + Job::new(JobPayload::Access { path, access }) + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_make_chmod_job" + )] + pub(crate) fn make_chmod_job(path: OsString, mode: i32) -> Job { + Job::new(JobPayload::Chmod { path, mode }) + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_get_file_size_result" + )] + pub(crate) fn get_file_size_result(job: &Job) -> AsyncHostResult { + match job.payload() { + JobPayload::FileSize { result, .. } => Ok(*result), + _ => Err(AsyncHostError::Badf), + } + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_make_fsync_job" + )] + pub(crate) fn make_fsync_job(fd: HostHandle, only_data: bool) -> Job { + Job::new(JobPayload::Fsync { fd, only_data }) + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_make_flock_job" + )] + pub(crate) fn make_flock_job(fd: HostHandle, exclusive: bool) -> Job { + Job::new(JobPayload::Flock { fd, exclusive }) + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_make_remove_job" + )] + pub(crate) fn make_remove_job(path: OsString) -> Job { + Job::new(JobPayload::Remove { path }) + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_make_rename_job" + )] + pub(crate) fn make_rename_job(old_path: OsString, new_path: OsString, replace: bool) -> Job { + Job::new(JobPayload::Rename { + old_path, + new_path, + replace, + }) + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_make_symlink_job" + )] + pub(crate) fn make_symlink_job(target: OsString, path: OsString, force_symlink: bool) -> Job { + Job::new(JobPayload::Symlink { + target, + path, + force_symlink, + }) + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_make_mkdir_job" + )] + pub(crate) fn make_mkdir_job(path: OsString, mode: i32) -> Job { + Job::new(JobPayload::Mkdir { path, mode }) + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_make_rmdir_job" + )] + pub(crate) fn make_rmdir_job(path: OsString) -> Job { + Job::new(JobPayload::Rmdir { path }) + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_make_readdir_job" + )] + pub(crate) fn make_readdir_job(dir: HostHandle, ptr: i32, len: i32, restart: bool) -> Job { + Job::new(JobPayload::Readdir { + dir, + dst: GuestBuffer::new(ptr, 0, len), + restart, + result: None, + }) + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_make_wait_for_process_job" + )] + pub(crate) fn make_wait_for_process_job(process: HostProcess) -> Job { + Job::new(JobPayload::WaitForProcess { process }) + } +} + +pub(crate) fn open_job_result(job: &Job) -> AsyncHostResult<&OpenJobResult> { + match job.payload() { + JobPayload::Open { + result: Some(result), + .. + } => Ok(result), + JobPayload::Open { .. } => Err(AsyncHostError::Inval), + _ => Err(AsyncHostError::Badf), + } +} + +#[cfg(test)] +mod tests { + use std::fs::File; + + use super::*; + use crate::async_sys::internal::event_loop::thread_pool::{ + HostFile, HostFileTable, run_host_job, + }; + + struct NoFiles; + + impl HostFileTable for NoFiles { + fn insert_file(&mut self, _file: File) -> AsyncHostResult { + unreachable!("sleep jobs do not access files") + } + + fn with_file_mut( + &mut self, + _handle: HostHandle, + _f: impl FnOnce(&mut File) -> AsyncHostResult, + ) -> AsyncHostResult { + unreachable!("sleep jobs do not access files") + } + + fn with_host_file_mut( + &mut self, + _handle: HostHandle, + _f: impl FnOnce(&mut HostFile) -> AsyncHostResult, + ) -> AsyncHostResult { + unreachable!("sleep jobs do not access files") + } + } + + #[test] + fn sleep_job_initial_result_matches_native_job_header() { + let job = make_sleep_job(0); + + assert_eq!(job_get_ret(&job), 0); + assert_eq!(job_get_err(&job), 0); + } + + #[test] + fn read_job_carries_host_handle_and_guest_buffer_payload() { + let job = make_read_job(7, 100, 2, 8, -1); + + match job.payload() { + JobPayload::Read { + fd, + dst, + position, + result: None, + } => { + assert_eq!(*fd, 7); + assert_eq!(*dst, GuestBuffer::new(100, 2, 8)); + assert_eq!(*position, -1); + } + other => panic!("unexpected payload: {other:?}"), + } + } + + #[test] + fn open_job_carries_owned_path_and_open_flags() { + let job = make_open_job(OsString::from("/tmp/example"), 2, 3, true, 1, 0o644); + + match job.payload() { + JobPayload::Open { + filename, + access, + create_mode, + append, + sync, + mode, + result: None, + } => { + assert_eq!(filename, &OsString::from("/tmp/example")); + assert_eq!(*access, 2); + assert_eq!(*create_mode, 3); + assert!(*append); + assert_eq!(*sync, 1); + assert_eq!(*mode, 0o644); + } + other => panic!("unexpected payload: {other:?}"), + } + } + + #[test] + fn sleep_job_runs_without_error() { + let mut job = make_sleep_job(0); + let mut files = NoFiles; + + run_host_job(&mut job, &mut files); + + assert_eq!(job_get_ret(&job), 0); + assert_eq!(job_get_err(&job), 0); + } + + #[cfg(unix)] + #[test] + fn unix_errno_is_cancelled_matches_async_stub() { + assert!(errno_is_cancelled(libc::EINTR)); + assert!(!errno_is_cancelled(libc::EINVAL)); + } +} diff --git a/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/mod.rs b/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/mod.rs new file mode 100644 index 000000000..0f8fc602b --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/mod.rs @@ -0,0 +1,55 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +mod fs; +mod jobs; +mod process; +mod runner; +mod sleep; +mod types; +mod worker; + +pub(crate) use jobs::{ + errno_is_cancelled, get_file_size_result, get_platform, job_get_err, job_get_ret, + make_access_job, make_chmod_job, make_file_kind_by_path_job, make_file_size_job, + make_file_time_by_path_job, make_file_time_job, make_flock_job, make_fsync_job, make_mkdir_job, + make_open_job, make_read_job, make_readdir_job, make_remove_job, make_rename_job, + make_rmdir_job, make_sleep_job, make_symlink_job, make_write_job, open_job_get_dev_id, + open_job_get_fd, open_job_get_file_id, open_job_get_kind, open_job_result, +}; +pub(crate) use process::{ + HostProcess, HostProcessTable, make_wait_for_process_job_from_handle, spawn_process, +}; +pub(crate) use runner::{complete_guest_job, run_host_job}; +#[cfg(test)] +pub(crate) use types::JobPayload; +pub(crate) use types::{ + FileTimeResult, GuestBuffer, HostFile, HostFileTable, HostHandle, Job, OpenJobResult, +}; +pub(crate) use worker::{ + HostWorkerHandle, HostWorkerJob, cancel_worker, free_worker, spawn_worker, wake_worker, + worker_enter_idle, +}; + +#[cfg(test)] +pub(crate) fn ported_symbols() -> Vec { + let mut symbols = Vec::new(); + symbols.extend_from_slice(jobs::PORTED_SYMBOLS); + symbols.extend_from_slice(worker::PORTED_SYMBOLS); + symbols +} diff --git a/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/process.rs b/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/process.rs new file mode 100644 index 000000000..af8528599 --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/process.rs @@ -0,0 +1,479 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use std::sync::{Arc, Mutex}; + +use crate::async_host::{AsyncHostError, AsyncHostResult}; + +use super::jobs; +use super::types::{HostFileTable, HostHandle, Job}; + +#[derive(Debug, Clone)] +pub(crate) struct HostProcess { + process: Arc>>, +} + +impl PartialEq for HostProcess { + fn eq(&self, other: &Self) -> bool { + Arc::ptr_eq(&self.process, &other.process) + } +} + +impl Eq for HostProcess {} + +impl HostProcess { + fn new(process: NativeProcess) -> Self { + Self { + process: Arc::new(Mutex::new(Some(process))), + } + } + + fn wait(&self) -> AsyncHostResult { + let mut process = self.process.lock().unwrap(); + let Some(process) = process.take() else { + return Err(AsyncHostError::Badf); + }; + process.wait() + } +} + +#[cfg(unix)] +#[derive(Debug, PartialEq, Eq)] +struct NativeProcess { + pid: libc::pid_t, +} + +#[cfg(unix)] +impl NativeProcess { + fn wait(self) -> AsyncHostResult { + let mut status = 0; + let ret = unsafe { libc::waitpid(self.pid, &mut status, 0) }; + if ret == self.pid { + Ok(libc::WEXITSTATUS(status)) + } else { + Err(last_native_error()) + } + } +} + +#[cfg(windows)] +#[derive(Debug, PartialEq, Eq)] +struct NativeProcess { + handle: isize, +} + +#[cfg(windows)] +impl NativeProcess { + fn handle(&self) -> windows_sys::Win32::Foundation::HANDLE { + self.handle as windows_sys::Win32::Foundation::HANDLE + } + + fn wait(self) -> AsyncHostResult { + use windows_sys::Win32::Foundation::WAIT_FAILED; + use windows_sys::Win32::System::Threading::{ + GetExitCodeProcess, INFINITE, WaitForSingleObject, + }; + + if unsafe { WaitForSingleObject(self.handle(), INFINITE) } == WAIT_FAILED { + return Err(last_native_error()); + } + let mut exit_code = 0; + if unsafe { GetExitCodeProcess(self.handle(), &mut exit_code) } == 0 { + return Err(last_native_error()); + } + Ok(exit_code as i32) + } +} + +#[cfg(windows)] +impl Drop for NativeProcess { + fn drop(&mut self) { + unsafe { + windows_sys::Win32::Foundation::CloseHandle(self.handle()); + } + } +} + +pub(crate) trait HostProcessTable { + fn insert_process(&mut self, process: HostProcess) -> AsyncHostResult; + + fn take_process(&mut self, handle: HostHandle) -> AsyncHostResult; +} + +pub(crate) fn spawn_process( + files: &mut impl HostFileTable, + processes: &mut impl HostProcessTable, + command: String, + args: Vec, + stdin: HostHandle, + stdout: HostHandle, + stderr: HostHandle, +) -> AsyncHostResult { + spawn_native_process(files, processes, command, args, stdin, stdout, stderr) +} + +#[cfg(unix)] +fn spawn_native_process( + files: &mut impl HostFileTable, + processes: &mut impl HostProcessTable, + command: String, + args: Vec, + stdin: HostHandle, + stdout: HostHandle, + stderr: HostHandle, +) -> AsyncHostResult { + use std::ffi::CString; + + let command_c = CString::new(command.clone()).map_err(|_| AsyncHostError::Inval)?; + let mut argv_c = Vec::with_capacity(args.len() + 1); + argv_c.push(CString::new(command.as_str()).map_err(|_| AsyncHostError::Inval)?); + for arg in args { + argv_c.push(CString::new(arg).map_err(|_| AsyncHostError::Inval)?); + } + let mut argv = argv_c + .iter() + .map(|arg| arg.as_ptr() as *mut libc::c_char) + .collect::>(); + argv.push(std::ptr::null_mut()); + + let env_c = current_environment(); + let mut envp = env_c + .iter() + .map(|env| env.as_ptr() as *mut libc::c_char) + .collect::>(); + envp.push(std::ptr::null_mut()); + + let stdio = [ + raw_fd_for_stdio(files, stdin)?, + raw_fd_for_stdio(files, stdout)?, + raw_fd_for_stdio(files, stderr)?, + ]; + + let mut file_actions = std::mem::MaybeUninit::::uninit(); + let ret = unsafe { libc::posix_spawn_file_actions_init(file_actions.as_mut_ptr()) }; + if ret != 0 { + return Err(AsyncHostError::Native(ret)); + } + let mut file_actions = unsafe { file_actions.assume_init() }; + + let mut attr = std::mem::MaybeUninit::::uninit(); + let ret = unsafe { libc::posix_spawnattr_init(attr.as_mut_ptr()) }; + if ret != 0 { + unsafe { + libc::posix_spawn_file_actions_destroy(&mut file_actions); + } + return Err(AsyncHostError::Native(ret)); + } + let mut attr = unsafe { attr.assume_init() }; + + let result = (|| { + for (target, fd) in stdio.into_iter().enumerate() { + if let Some(fd) = fd { + let ret = unsafe { + libc::posix_spawn_file_actions_adddup2( + &mut file_actions, + fd, + target as libc::c_int, + ) + }; + if ret != 0 { + return Err(AsyncHostError::Native(ret)); + } + } + } + + configure_spawn_attributes(&mut attr)?; + + let mut pid = 0; + let ret = if command_c.as_bytes().contains(&b'/') { + unsafe { + libc::posix_spawn( + &mut pid, + command_c.as_ptr(), + &file_actions, + &attr, + argv.as_mut_ptr(), + envp.as_mut_ptr(), + ) + } + } else { + unsafe { + libc::posix_spawnp( + &mut pid, + command_c.as_ptr(), + &file_actions, + &attr, + argv.as_mut_ptr(), + envp.as_mut_ptr(), + ) + } + }; + if ret != 0 { + return Err(AsyncHostError::Native(ret)); + } + processes.insert_process(HostProcess::new(NativeProcess { pid })) + })(); + + unsafe { + libc::posix_spawnattr_destroy(&mut attr); + libc::posix_spawn_file_actions_destroy(&mut file_actions); + } + result +} + +#[cfg(unix)] +fn raw_fd_for_stdio( + files: &mut impl HostFileTable, + handle: HostHandle, +) -> AsyncHostResult> { + use std::os::fd::AsRawFd; + + if handle < 0 { + return Ok(None); + } + files.with_file_mut(handle, |file| Ok(Some(file.as_raw_fd()))) +} + +#[cfg(all(unix, target_os = "linux"))] +unsafe fn current_environ() -> *mut *mut libc::c_char { + unsafe extern "C" { + static mut environ: *mut *mut libc::c_char; + } + + unsafe { environ } +} + +#[cfg(all(unix, target_os = "macos"))] +unsafe fn current_environ() -> *mut *mut libc::c_char { + unsafe { *libc::_NSGetEnviron() } +} + +#[cfg(unix)] +fn current_environment() -> Vec { + let mut env = Vec::new(); + let mut cursor = unsafe { current_environ() }; + if cursor.is_null() { + return env; + } + while unsafe { !(*cursor).is_null() } { + env.push(unsafe { std::ffi::CStr::from_ptr(*cursor).to_owned() }); + cursor = unsafe { cursor.add(1) }; + } + env +} + +#[cfg(unix)] +fn configure_spawn_attributes(attr: &mut libc::posix_spawnattr_t) -> AsyncHostResult<()> { + let flags = (libc::POSIX_SPAWN_SETSIGMASK | libc::POSIX_SPAWN_SETSIGDEF) as _; + let ret = unsafe { libc::posix_spawnattr_setflags(attr, flags) }; + if ret != 0 { + return Err(AsyncHostError::Native(ret)); + } + + let mut current_mask = std::mem::MaybeUninit::::uninit(); + let ret = unsafe { + libc::pthread_sigmask( + libc::SIG_SETMASK, + std::ptr::null(), + current_mask.as_mut_ptr(), + ) + }; + if ret != 0 { + return Err(AsyncHostError::Native(ret)); + } + let current_mask = unsafe { current_mask.assume_init() }; + let ret = unsafe { libc::posix_spawnattr_setsigmask(attr, ¤t_mask) }; + if ret != 0 { + return Err(AsyncHostError::Native(ret)); + } + + let mut all_signals = std::mem::MaybeUninit::::uninit(); + if unsafe { libc::sigfillset(all_signals.as_mut_ptr()) } != 0 { + return Err(last_native_error()); + } + let all_signals = unsafe { all_signals.assume_init() }; + let ret = unsafe { libc::posix_spawnattr_setsigdefault(attr, &all_signals) }; + if ret != 0 { + return Err(AsyncHostError::Native(ret)); + } + Ok(()) +} + +#[cfg(windows)] +fn spawn_native_process( + files: &mut impl HostFileTable, + processes: &mut impl HostProcessTable, + command: String, + args: Vec, + stdin: HostHandle, + stdout: HostHandle, + stderr: HostHandle, +) -> AsyncHostResult { + use windows_sys::Win32::Foundation::INVALID_HANDLE_VALUE; + use windows_sys::Win32::System::Console::{ + STD_ERROR_HANDLE, STD_INPUT_HANDLE, STD_OUTPUT_HANDLE, + }; + use windows_sys::Win32::System::Threading::{ + CREATE_NEW_PROCESS_GROUP, CREATE_UNICODE_ENVIRONMENT, CreateProcessW, PROCESS_INFORMATION, + STARTF_USESTDHANDLES, STARTUPINFOW, + }; + + let stdio = [ + raw_handle_for_stdio(files, stdin, STD_INPUT_HANDLE)?, + raw_handle_for_stdio(files, stdout, STD_OUTPUT_HANDLE)?, + raw_handle_for_stdio(files, stderr, STD_ERROR_HANDLE)?, + ]; + for handle in stdio { + if handle == INVALID_HANDLE_VALUE as isize { + return Err(last_native_error()); + } + if unsafe { + windows_sys::Win32::Foundation::SetHandleInformation( + handle as _, + windows_sys::Win32::Foundation::HANDLE_FLAG_INHERIT, + windows_sys::Win32::Foundation::HANDLE_FLAG_INHERIT, + ) + } == 0 + { + return Err(last_native_error()); + } + } + + let mut command_line = windows_command_line(command, args) + .encode_utf16() + .chain(std::iter::once(0)) + .collect::>(); + let startup_info = STARTUPINFOW { + cb: std::mem::size_of::() as u32, + dwFlags: STARTF_USESTDHANDLES, + hStdInput: stdio[0] as _, + hStdOutput: stdio[1] as _, + hStdError: stdio[2] as _, + ..unsafe { std::mem::zeroed() } + }; + let mut process_info = unsafe { std::mem::zeroed::() }; + let ok = unsafe { + CreateProcessW( + std::ptr::null(), + command_line.as_mut_ptr(), + std::ptr::null(), + std::ptr::null(), + 1, + CREATE_NEW_PROCESS_GROUP | CREATE_UNICODE_ENVIRONMENT, + std::ptr::null(), + std::ptr::null(), + &startup_info, + &mut process_info, + ) + }; + if ok == 0 { + return Err(last_native_error()); + } + unsafe { + windows_sys::Win32::Foundation::CloseHandle(process_info.hThread); + } + processes.insert_process(HostProcess::new(NativeProcess { + handle: process_info.hProcess as isize, + })) +} + +#[cfg(windows)] +fn raw_handle_for_stdio( + files: &mut impl HostFileTable, + handle: HostHandle, + default: u32, +) -> AsyncHostResult { + use std::os::windows::io::AsRawHandle; + + if handle < 0 { + return Ok(unsafe { windows_sys::Win32::System::Console::GetStdHandle(default) } as isize); + } + files.with_file_mut(handle, |file| Ok(file.as_raw_handle() as isize)) +} + +#[cfg(windows)] +fn windows_command_line(command: String, args: Vec) -> String { + let command = if command.ends_with(".exe") { + command + } else { + format!("{command}.exe") + }; + let mut line = String::new(); + push_windows_arg(&mut line, &command); + for arg in args { + line.push(' '); + push_windows_arg(&mut line, &arg); + } + line +} + +#[cfg(windows)] +fn push_windows_arg(out: &mut String, arg: &str) { + let need_quote = arg.chars().any(|ch| matches!(ch, ' ' | '\t' | '"')); + if !need_quote { + out.push_str(arg); + return; + } + + out.push('"'); + let mut backslashes = 0; + for ch in arg.chars() { + match ch { + '\\' => backslashes += 1, + '"' => { + for _ in 0..(backslashes * 2 + 1) { + out.push('\\'); + } + out.push('"'); + backslashes = 0; + } + _ => { + for _ in 0..backslashes { + out.push('\\'); + } + out.push(ch); + backslashes = 0; + } + } + } + for _ in 0..(backslashes * 2) { + out.push('\\'); + } + out.push('"'); +} + +pub(crate) fn make_wait_for_process_job_from_handle( + processes: &mut impl HostProcessTable, + process: HostHandle, +) -> AsyncHostResult { + Ok(jobs::make_wait_for_process_job( + processes.take_process(process)?, + )) +} + +pub(super) fn run_wait_for_process_job(process: &HostProcess) -> AsyncHostResult { + process.wait().map(i64::from) +} + +fn last_native_error() -> AsyncHostError { + AsyncHostError::Native( + std::io::Error::last_os_error() + .raw_os_error() + .unwrap_or_else(|| AsyncHostError::Inval.errno()), + ) +} diff --git a/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/runner.rs b/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/runner.rs new file mode 100644 index 000000000..81643ccaa --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/runner.rs @@ -0,0 +1,157 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_host::{AsyncHostError, AsyncHostResult, GuestMemory}; +use crate::async_sys::internal::fd_util; + +use super::fs::{ + run_access_job, run_chmod_job, run_file_kind_by_path_job, run_file_size_job, + run_file_time_by_path_job, run_file_time_job, run_flock_job, run_fsync_job, run_mkdir_job, + run_open_job, run_read_job, run_readdir_job, run_remove_job, run_rename_job, run_rmdir_job, + run_symlink_job, run_write_job, +}; +use super::process::run_wait_for_process_job; +use super::sleep::run_sleep_job; +use super::types::{HostFileTable, Job, JobPayload}; + +pub(crate) fn run_host_job(job: &mut Job, files: &mut impl HostFileTable) { + job.set_ret(0); + + let result = match job.payload_mut() { + JobPayload::Sleep { duration_ms } => { + run_sleep_job(*duration_ms); + Ok(0) + } + JobPayload::Open { + filename, + access, + create_mode, + append, + sync, + mode, + result, + } => run_open_job( + files, + result, + filename.clone(), + *access, + *create_mode, + *append, + *sync, + *mode, + ), + JobPayload::Read { + fd, + dst, + position, + result, + } => run_read_job(files, *fd, *dst, *position, result), + JobPayload::Write { fd, data, position } => run_write_job(files, *fd, data, *position), + JobPayload::FileKindByPath { + parent, + path, + follow_symlink, + } => run_file_kind_by_path_job(files, *parent, path.clone(), *follow_symlink), + JobPayload::FileSize { fd, result } => run_file_size_job(files, *fd, result), + JobPayload::FileTime { fd, result, .. } => run_file_time_job(files, *fd, result), + JobPayload::FileTimeByPath { + path, + follow_symlink, + result, + .. + } => run_file_time_by_path_job(path.clone(), *follow_symlink, result), + JobPayload::Access { path, access } => run_access_job(path.clone(), *access), + JobPayload::Chmod { path, mode } => run_chmod_job(path.clone(), *mode), + JobPayload::Fsync { fd, only_data } => run_fsync_job(files, *fd, *only_data), + JobPayload::Flock { fd, exclusive } => run_flock_job(files, *fd, *exclusive), + JobPayload::Remove { path } => run_remove_job(path.clone()), + JobPayload::Rename { + old_path, + new_path, + replace, + } => run_rename_job(old_path.clone(), new_path.clone(), *replace), + JobPayload::Symlink { + target, + path, + force_symlink, + } => run_symlink_job(target.clone(), path.clone(), *force_symlink), + JobPayload::Mkdir { path, mode } => run_mkdir_job(path.clone(), *mode), + JobPayload::Rmdir { path } => run_rmdir_job(path.clone()), + JobPayload::Readdir { + dir, + dst, + restart, + result, + } => run_readdir_job(files, *dir, *dst, *restart, result), + JobPayload::WaitForProcess { process } => run_wait_for_process_job(process), + }; + + match result { + Ok(ret) => job.set_ret(ret), + Err(error) => job.set_err(error.errno()), + } +} + +pub(crate) fn complete_guest_job( + job: &mut Job, + memory: &mut (impl GuestMemory + ?Sized), +) -> AsyncHostResult<()> { + if let JobPayload::Read { + dst, + result: Some(result), + .. + } + | JobPayload::Readdir { + dst, + result: Some(result), + .. + } = job.payload_mut() + { + let dst_offset = dst + .ptr + .checked_add(dst.offset) + .ok_or(AsyncHostError::Fault)?; + memory.write_with_capacity(dst_offset, dst.len, result)?; + } + if let JobPayload::FileTime { + out, + result: Some(result), + .. + } + | JobPayload::FileTimeByPath { + out, + result: Some(result), + .. + } = job.payload_mut() + { + let file_time = result.as_native(); + let mut record = [0; 48]; + record[0..8].copy_from_slice(&fd_util::stub::get_atime_sec(file_time).to_le_bytes()); + record[8..12].copy_from_slice(&fd_util::stub::get_atime_nsec(file_time).to_le_bytes()); + record[16..24].copy_from_slice(&fd_util::stub::get_mtime_sec(file_time).to_le_bytes()); + record[24..28].copy_from_slice(&fd_util::stub::get_mtime_nsec(file_time).to_le_bytes()); + record[32..40].copy_from_slice(&fd_util::stub::get_ctime_sec(file_time).to_le_bytes()); + record[40..44].copy_from_slice(&fd_util::stub::get_ctime_nsec(file_time).to_le_bytes()); + let dst_offset = out + .ptr + .checked_add(out.offset) + .ok_or(AsyncHostError::Fault)?; + memory.write_with_capacity(dst_offset, out.len, &record)?; + } + Ok(()) +} diff --git a/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/sleep.rs b/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/sleep.rs new file mode 100644 index 000000000..9d1bfa587 --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/sleep.rs @@ -0,0 +1,63 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +pub(super) fn run_sleep_job(duration_ms: i32) { + #[cfg(windows)] + { + // Match the native stub's `Sleep(((struct sleep_job*)job)->duration)`. + unsafe { windows_sys::Win32::System::Threading::Sleep(duration_ms as u32) }; + } + #[cfg(all(unix, target_os = "macos"))] + { + run_sleep_job_with_kqueue(duration_ms); + } + #[cfg(all(unix, not(target_os = "macos")))] + { + run_sleep_job_with_nanosleep(duration_ms); + } +} + +#[cfg(all(unix, target_os = "macos"))] +fn run_sleep_job_with_kqueue(duration_ms: i32) { + let kqfd = unsafe { libc::kqueue() }; + let duration = sleep_job_timespec(duration_ms); + let mut event = std::mem::MaybeUninit::::uninit(); + + // Native async intentionally uses kqueue as a timeout-only sleeper on + // macOS because nanosleep was too imprecise on CI runners. + unsafe { + libc::kevent(kqfd, std::ptr::null(), 0, event.as_mut_ptr(), 1, &duration); + libc::close(kqfd); + } +} + +#[cfg(all(unix, not(target_os = "macos")))] +fn run_sleep_job_with_nanosleep(duration_ms: i32) { + let duration = sleep_job_timespec(duration_ms); + unsafe { + libc::nanosleep(&duration, std::ptr::null_mut()); + } +} + +#[cfg(unix)] +fn sleep_job_timespec(duration_ms: i32) -> libc::timespec { + libc::timespec { + tv_sec: (duration_ms / 1000) as libc::time_t, + tv_nsec: ((duration_ms % 1000) * 1_000_000) as libc::c_long, + } +} diff --git a/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/types.rs b/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/types.rs new file mode 100644 index 000000000..50b2b8ae0 --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/types.rs @@ -0,0 +1,270 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use std::collections::VecDeque; +use std::ffi::OsString; +use std::fs::File; + +use super::process::HostProcess; +use crate::async_host::AsyncHostResult; +use crate::async_sys::internal::fd_util; + +pub(crate) type HostHandle = i32; + +#[derive(Debug)] +pub(crate) struct HostFile { + file: File, + pending_dir_entries: VecDeque>, + #[cfg(windows)] + lock_file: Option, +} + +impl HostFile { + pub(crate) fn new(file: File) -> Self { + Self { + file, + pending_dir_entries: VecDeque::new(), + #[cfg(windows)] + lock_file: None, + } + } + + pub(crate) fn file_mut(&mut self) -> &mut File { + &mut self.file + } + + pub(crate) fn pending_dir_entries_mut(&mut self) -> &mut VecDeque> { + &mut self.pending_dir_entries + } + + #[cfg(windows)] + pub(crate) fn set_lock_file(&mut self, file: File) { + self.lock_file = Some(file); + } + + #[cfg(windows)] + pub(crate) fn take_lock_file(&mut self) -> Option { + self.lock_file.take() + } +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) struct GuestBuffer { + pub(crate) ptr: i32, + pub(crate) offset: i32, + pub(crate) len: i32, +} + +impl GuestBuffer { + pub(crate) fn new(ptr: i32, offset: i32, len: i32) -> Self { + Self { ptr, offset, len } + } +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) struct OpenJobResult { + pub(crate) fd: HostHandle, + pub(crate) kind: i32, + pub(crate) dev_id: u64, + pub(crate) file_id: u64, +} + +#[derive(Clone, Copy)] +pub(crate) struct FileTimeResult(fd_util::stub::FileTime); + +impl FileTimeResult { + pub(crate) fn new(file_time: fd_util::stub::FileTime) -> Self { + Self(file_time) + } + + pub(crate) fn as_native(&self) -> &fd_util::stub::FileTime { + &self.0 + } +} + +impl std::fmt::Debug for FileTimeResult { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_tuple("FileTimeResult").finish_non_exhaustive() + } +} + +#[derive(Debug, Clone)] +pub(crate) struct Job { + ret: i64, + err: i32, + payload: JobPayload, +} + +impl Job { + pub(super) fn new(payload: JobPayload) -> Self { + Self { + ret: 0, + err: 0, + payload, + } + } + + pub(crate) fn payload(&self) -> &JobPayload { + &self.payload + } + + pub(crate) fn payload_mut(&mut self) -> &mut JobPayload { + &mut self.payload + } + + pub(crate) fn ret(&self) -> i64 { + self.ret + } + + pub(crate) fn err(&self) -> i32 { + self.err + } + + pub(crate) fn set_ret(&mut self, ret: i64) { + self.ret = ret; + self.err = 0; + } + + pub(crate) fn set_err(&mut self, err: i32) { + self.ret = -1; + self.err = err; + } +} + +#[derive(Debug, Clone)] +pub(crate) enum JobPayload { + Sleep { + duration_ms: i32, + }, + Read { + fd: HostHandle, + dst: GuestBuffer, + position: i64, + result: Option>, + }, + Write { + fd: HostHandle, + data: Vec, + position: i64, + }, + Open { + filename: OsString, + access: i32, + create_mode: i32, + append: bool, + sync: i32, + mode: i32, + result: Option, + }, + FileKindByPath { + parent: HostHandle, + path: OsString, + follow_symlink: bool, + }, + FileSize { + fd: HostHandle, + result: i64, + }, + FileTime { + fd: HostHandle, + out: GuestBuffer, + result: Option, + }, + FileTimeByPath { + path: OsString, + out: GuestBuffer, + follow_symlink: bool, + result: Option, + }, + Access { + path: OsString, + access: i32, + }, + Chmod { + path: OsString, + mode: i32, + }, + Fsync { + fd: HostHandle, + only_data: bool, + }, + Flock { + fd: HostHandle, + exclusive: bool, + }, + Remove { + path: OsString, + }, + Rename { + old_path: OsString, + new_path: OsString, + replace: bool, + }, + Symlink { + target: OsString, + path: OsString, + force_symlink: bool, + }, + Mkdir { + path: OsString, + mode: i32, + }, + Rmdir { + path: OsString, + }, + Readdir { + dir: HostHandle, + dst: GuestBuffer, + restart: bool, + result: Option>, + }, + WaitForProcess { + process: HostProcess, + }, +} + +pub(crate) trait HostFileTable { + fn insert_file(&mut self, file: File) -> AsyncHostResult; + + fn with_file_mut( + &mut self, + handle: HostHandle, + f: impl FnOnce(&mut File) -> AsyncHostResult, + ) -> AsyncHostResult; + + fn with_host_file_mut( + &mut self, + handle: HostHandle, + f: impl FnOnce(&mut HostFile) -> AsyncHostResult, + ) -> AsyncHostResult; +} + +pub(crate) fn platform() -> i32 { + #[cfg(windows)] + { + 2 + } + #[cfg(target_os = "macos")] + { + 1 + } + #[cfg(target_os = "linux")] + { + 0 + } +} diff --git a/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/worker.rs b/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/worker.rs new file mode 100644 index 000000000..439517b62 --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/event_loop/thread_pool/worker.rs @@ -0,0 +1,327 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use std::sync::{Arc, Condvar, Mutex}; +use std::thread::JoinHandle; + +use crate::async_sys::ported_fns; + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) struct HostWorkerJob { + pub(crate) job_id: i32, + pub(crate) job_handle: i32, +} + +#[derive(Debug)] +struct HostWorkerState { + job: Option, + waiting: bool, +} + +#[derive(Debug)] +struct HostWorkerShared { + state: Mutex, + wakeup: Condvar, +} + +// MoonBit owns the pool scheduler. Each host worker handle owns one long-lived +// OS thread and follows thread_pool.c's worker state machine: run current job, +// publish completion, wait until MoonBit either assigns another job or parks it. +pub(crate) struct HostWorkerHandle { + shared: Arc, + thread: Option>, + #[cfg(unix)] + thread_id: Arc>>, +} + +impl std::fmt::Debug for HostWorkerHandle { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("HostWorkerHandle") + .field( + "alive", + &self + .thread + .as_ref() + .is_some_and(|thread| !thread.is_finished()), + ) + .field("state", &self.shared.state.lock().ok()) + .finish() + } +} + +#[cfg(unix)] +fn init_worker_signal_handler() { + static INIT: std::sync::Once = std::sync::Once::new(); + + extern "C" fn nop_signal_handler(_: i32) {} + + INIT.call_once(|| unsafe { + libc::signal(libc::SIGPIPE, libc::SIG_IGN); + let mut action = std::mem::zeroed::(); + action.sa_sigaction = nop_signal_handler as usize; + libc::sigemptyset(&mut action.sa_mask); + action.sa_flags = 0; + libc::sigaction(libc::SIGUSR2, &action, std::ptr::null_mut()); + }); +} + +impl HostWorkerHandle { + pub(crate) fn spawn( + init_job: HostWorkerJob, + mut run_job: impl FnMut(HostWorkerJob) + Send + 'static, + mut complete_job: impl FnMut(HostWorkerJob) + Send + 'static, + ) -> Self { + #[cfg(unix)] + init_worker_signal_handler(); + + let shared = Arc::new(HostWorkerShared { + state: Mutex::new(HostWorkerState { + job: Some(init_job), + waiting: false, + }), + wakeup: Condvar::new(), + }); + let worker_shared = Arc::clone(&shared); + #[cfg(unix)] + let thread_id = Arc::new(Mutex::new(None)); + #[cfg(unix)] + let worker_thread_id = Arc::clone(&thread_id); + let thread = std::thread::spawn(move || { + #[cfg(unix)] + { + *worker_thread_id.lock().unwrap() = Some(unsafe { libc::pthread_self() }); + } + + loop { + let Some(job) = worker_shared.state.lock().unwrap().job else { + break; + }; + run_job(job); + + { + let mut state = worker_shared.state.lock().unwrap(); + state.waiting = true; + } + complete_job(job); + + let mut state = worker_shared.state.lock().unwrap(); + while state.waiting { + state = worker_shared.wakeup.wait(state).unwrap(); + } + } + + #[cfg(unix)] + { + *worker_thread_id.lock().unwrap() = None; + } + }); + Self { + shared, + thread: Some(thread), + #[cfg(unix)] + thread_id, + } + } + + pub(crate) fn wake(&self, job: Option) { + let mut state = self.shared.state.lock().unwrap(); + state.job = job; + state.waiting = false; + self.shared.wakeup.notify_one(); + } + + pub(crate) fn enter_idle(&self) { + self.shared.state.lock().unwrap().job = None; + } + + pub(crate) fn cancel(&self) -> i32 { + if self.shared.state.lock().unwrap().waiting { + return 1; + } + + #[cfg(unix)] + { + if let Some(thread_id) = *self.thread_id.lock().unwrap() { + unsafe { + libc::pthread_kill(thread_id, libc::SIGUSR2); + } + } + 0 + } + + #[cfg(windows)] + { + use std::os::windows::io::AsRawHandle; + use windows_sys::Win32::Foundation::{ERROR_NOT_FOUND, GetLastError}; + use windows_sys::Win32::System::IO::CancelSynchronousIo; + + let Some(thread) = &self.thread else { + return -1; + }; + if unsafe { CancelSynchronousIo(thread.as_raw_handle()) } != 0 { + 1 + } else if unsafe { GetLastError() } == ERROR_NOT_FOUND { + 0 + } else { + -1 + } + } + } + + pub(crate) fn join(&mut self) { + if let Some(thread) = self.thread.take() { + self.wake(None); + let _ = thread.join(); + } + } +} + +ported_fns! { + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_spawn_worker" + )] + pub(crate) fn spawn_worker( + init_job: HostWorkerJob, + run_job: impl FnMut(HostWorkerJob) + Send + 'static, + complete_job: impl FnMut(HostWorkerJob) + Send + 'static, + ) -> HostWorkerHandle { + HostWorkerHandle::spawn(init_job, run_job, complete_job) + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_wake_worker" + )] + pub(crate) fn wake_worker(worker: &HostWorkerHandle, job: HostWorkerJob) { + worker.wake(Some(job)); + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_worker_enter_idle" + )] + pub(crate) fn worker_enter_idle(worker: &HostWorkerHandle) { + worker.enter_idle(); + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_cancel_worker" + )] + pub(crate) fn cancel_worker(worker: &HostWorkerHandle) -> i32 { + worker.cancel() + } + + #[ported( + source = "src/internal/event_loop/thread_pool.c", + original = "moonbitlang_async_free_worker" + )] + pub(crate) fn free_worker(mut worker: HostWorkerHandle) { + worker.join(); + } +} + +#[cfg(test)] +mod tests { + use super::*; + use std::sync::mpsc; + + #[test] + fn host_worker_runs_initial_job_then_waits_for_wake() { + let (sender, receiver) = mpsc::channel(); + let (completion_sender, completion_receiver) = mpsc::channel(); + let worker = spawn_worker( + HostWorkerJob { + job_id: 7, + job_handle: 11, + }, + move |job| sender.send(job).unwrap(), + move |job| completion_sender.send(job).unwrap(), + ); + + assert_eq!( + receiver.recv().unwrap(), + HostWorkerJob { + job_id: 7, + job_handle: 11 + } + ); + assert_eq!( + completion_receiver.recv().unwrap(), + HostWorkerJob { + job_id: 7, + job_handle: 11 + } + ); + assert_eq!(cancel_worker(&worker), 1); + + wake_worker( + &worker, + HostWorkerJob { + job_id: 13, + job_handle: 17, + }, + ); + assert_eq!( + receiver.recv().unwrap(), + HostWorkerJob { + job_id: 13, + job_handle: 17 + } + ); + assert_eq!( + completion_receiver.recv().unwrap(), + HostWorkerJob { + job_id: 13, + job_handle: 17 + } + ); + free_worker(worker); + } + + #[test] + fn worker_enter_idle_parks_until_next_wake() { + let (sender, receiver) = mpsc::channel(); + let (completion_sender, completion_receiver) = mpsc::channel(); + let worker = spawn_worker( + HostWorkerJob { + job_id: 1, + job_handle: 2, + }, + move |job| sender.send(job).unwrap(), + move |job| completion_sender.send(job).unwrap(), + ); + + assert_eq!(receiver.recv().unwrap().job_id, 1); + assert_eq!(completion_receiver.recv().unwrap().job_id, 1); + + worker_enter_idle(&worker); + wake_worker( + &worker, + HostWorkerJob { + job_id: 3, + job_handle: 4, + }, + ); + + assert_eq!(receiver.recv().unwrap().job_id, 3); + assert_eq!(completion_receiver.recv().unwrap().job_id, 3); + free_worker(worker); + } +} diff --git a/crates/moonrun/src/async_sys/internal/fd_util/mod.rs b/crates/moonrun/src/async_sys/internal/fd_util/mod.rs new file mode 100644 index 000000000..fa380a27b --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/fd_util/mod.rs @@ -0,0 +1,19 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +pub(crate) mod stub; diff --git a/crates/moonrun/src/async_sys/internal/fd_util/stub.rs b/crates/moonrun/src/async_sys/internal/fd_util/stub.rs new file mode 100644 index 000000000..21434eb00 --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/fd_util/stub.rs @@ -0,0 +1,448 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_host::{AsyncHostError, AsyncHostResult}; +use crate::async_sys::internal::event_loop::thread_pool::HostFileTable; +use crate::async_sys::ported_fns; + +#[cfg(unix)] +pub(crate) type RawFd = std::os::fd::RawFd; + +#[cfg(windows)] +pub(crate) type RawFd = windows_sys::Win32::Foundation::HANDLE; + +#[cfg(windows)] +pub(crate) type FileTime = windows_sys::Win32::Storage::FileSystem::FILE_BASIC_INFO; + +#[cfg(unix)] +pub(crate) type FileTime = libc::stat; + +#[cfg(windows)] +const WINDOWS_TICKS_PER_SECOND: i64 = 10_000_000; +#[cfg(windows)] +const WINDOWS_TO_UNIX_EPOCH_SECONDS: i64 = 11_644_473_600; + +#[cfg(windows)] +fn windows_filetime_to_unix_seconds(ticks: i64) -> i64 { + ticks / WINDOWS_TICKS_PER_SECOND - WINDOWS_TO_UNIX_EPOCH_SECONDS +} + +#[cfg(windows)] +fn windows_filetime_to_nanoseconds(ticks: i64) -> i32 { + ((ticks % WINDOWS_TICKS_PER_SECOND) * 100) as i32 +} + +ported_fns! { + #[ported( + source = "src/internal/fd_util/stub.c", + original = "moonbitlang_async_get_invalid_handle" + )] + #[cfg(windows)] + #[allow(dead_code)] + pub(crate) fn get_invalid_handle() -> RawFd { + windows_sys::Win32::Foundation::INVALID_HANDLE_VALUE + } + + #[ported( + source = "src/internal/fd_util/stub.c", + original = "moonbitlang_async_close_fd" + )] + #[cfg(windows)] + #[allow(dead_code)] + pub(crate) fn close_fd(fd: RawFd, is_socket: bool) -> AsyncHostResult<()> { + use windows_sys::Win32::Foundation::CloseHandle; + use windows_sys::Win32::Networking::WinSock::{SOCKET, closesocket}; + + let ok = if is_socket { + unsafe { closesocket(fd as SOCKET) == 0 } + } else { + unsafe { CloseHandle(fd) != 0 } + }; + if ok { Ok(()) } else { Err(last_native_error()) } + } + + #[ported( + source = "src/internal/fd_util/stub.c", + original = "moonbitlang_async_fd_is_nonblocking" + )] + #[cfg(unix)] + #[allow(dead_code)] + pub(crate) fn fd_is_nonblocking(fd: RawFd) -> AsyncHostResult { + let flags = fcntl_getfl(fd)?; + Ok((flags & libc::O_NONBLOCK) > 0) + } + + #[ported( + source = "src/internal/fd_util/stub.c", + original = "moonbitlang_async_set_blocking" + )] + #[cfg(unix)] + #[allow(dead_code)] + pub(crate) fn set_blocking(fd: RawFd) -> AsyncHostResult<()> { + let flags = fcntl_getfl(fd)?; + if (flags & libc::O_NONBLOCK) != 0 { + fcntl_setfl(fd, flags & !libc::O_NONBLOCK)?; + } + Ok(()) + } + + #[ported( + source = "src/internal/fd_util/stub.c", + original = "moonbitlang_async_set_nonblocking" + )] + #[cfg(unix)] + #[allow(dead_code)] + pub(crate) fn set_nonblocking(fd: RawFd) -> AsyncHostResult<()> { + let flags = fcntl_getfl(fd)?; + if (flags & libc::O_NONBLOCK) == 0 { + fcntl_setfl(fd, flags | libc::O_NONBLOCK)?; + } + Ok(()) + } + + #[ported( + source = "src/internal/fd_util/stub.c", + original = "moonbitlang_async_set_cloexec" + )] + #[cfg(unix)] + #[allow(dead_code)] + pub(crate) fn set_cloexec(fd: RawFd) -> AsyncHostResult<()> { + let flags = unsafe { libc::fcntl(fd, libc::F_GETFD) }; + if flags < 0 { + return Err(last_native_error()); + } + if (flags & libc::FD_CLOEXEC) == 0 { + let ret = unsafe { libc::fcntl(fd, libc::F_SETFD, flags | libc::FD_CLOEXEC) }; + if ret < 0 { + return Err(last_native_error()); + } + } + Ok(()) + } + + #[ported( + source = "src/internal/fd_util/stub.c", + original = "moonbitlang_async_create_named_pipe_server" + )] + #[cfg(windows)] + #[allow(dead_code)] + pub(crate) fn create_named_pipe_server(name: &std::ffi::OsStr, is_async: bool) -> RawFd { + use std::os::windows::ffi::OsStrExt; + use windows_sys::Win32::Storage::FileSystem::{ + FILE_FLAG_FIRST_PIPE_INSTANCE, FILE_FLAG_OVERLAPPED, PIPE_ACCESS_OUTBOUND, + }; + use windows_sys::Win32::System::Pipes::{ + CreateNamedPipeW, PIPE_READMODE_BYTE, PIPE_TYPE_BYTE, PIPE_UNLIMITED_INSTANCES, + PIPE_WAIT, + }; + + let mut name: Vec = name.encode_wide().collect(); + name.push(0); + let flags = PIPE_ACCESS_OUTBOUND + | FILE_FLAG_FIRST_PIPE_INSTANCE + | if is_async { FILE_FLAG_OVERLAPPED } else { 0 }; + unsafe { + CreateNamedPipeW( + name.as_ptr(), + flags, + PIPE_TYPE_BYTE | PIPE_READMODE_BYTE | PIPE_WAIT, + PIPE_UNLIMITED_INSTANCES, + 1024, + 1024, + 0, + std::ptr::null(), + ) + } + } + + #[ported( + source = "src/internal/fd_util/stub.c", + original = "moonbitlang_async_create_named_pipe_client" + )] + #[cfg(windows)] + #[allow(dead_code)] + pub(crate) fn create_named_pipe_client(name: &std::ffi::OsStr, is_async: bool) -> RawFd { + use std::os::windows::ffi::OsStrExt; + use windows_sys::Win32::Foundation::GENERIC_READ; + use windows_sys::Win32::Storage::FileSystem::{ + CreateFileW, FILE_FLAG_OVERLAPPED, OPEN_EXISTING, + }; + + let mut name: Vec = name.encode_wide().collect(); + name.push(0); + unsafe { + CreateFileW( + name.as_ptr(), + GENERIC_READ, + 0, + std::ptr::null(), + OPEN_EXISTING, + if is_async { FILE_FLAG_OVERLAPPED } else { 0 }, + std::ptr::null_mut(), + ) + } + } + + #[ported( + source = "src/internal/fd_util/stub.c", + original = "moonbitlang_async_pipe" + )] + #[cfg(unix)] + #[allow(dead_code)] + pub(crate) fn pipe() -> AsyncHostResult<[RawFd; 2]> { + let mut fds = [0, 0]; + if unsafe { libc::pipe(fds.as_mut_ptr()) } < 0 { + return Err(last_native_error()); + } + for fd in fds { + if let Err(error) = set_cloexec(fd) { + unsafe { + libc::close(fds[0]); + libc::close(fds[1]); + } + return Err(error); + } + } + Ok(fds) + } + + #[ported( + source = "src/internal/fd_util/stub.c", + original = "moonbitlang_async_sizeof_file_time" + )] + #[allow(dead_code)] + pub(crate) fn sizeof_file_time() -> i32 { + std::mem::size_of::() as i32 + } + + #[ported( + source = "src/internal/fd_util/stub.c", + original = "moonbitlang_async_get_atime_sec" + )] + #[allow(dead_code)] + #[allow(clippy::unnecessary_cast)] + pub(crate) fn get_atime_sec(file_time: &FileTime) -> i64 { + #[cfg(windows)] + { + windows_filetime_to_unix_seconds(file_time.LastAccessTime) + } + #[cfg(unix)] + { + file_time.st_atime as i64 + } + } + + #[ported( + source = "src/internal/fd_util/stub.c", + original = "moonbitlang_async_get_atime_nsec" + )] + #[allow(dead_code)] + pub(crate) fn get_atime_nsec(file_time: &FileTime) -> i32 { + #[cfg(windows)] + { + windows_filetime_to_nanoseconds(file_time.LastAccessTime) + } + #[cfg(unix)] + { + file_time.st_atime_nsec as i32 + } + } + + #[ported( + source = "src/internal/fd_util/stub.c", + original = "moonbitlang_async_get_mtime_sec" + )] + #[allow(dead_code)] + #[allow(clippy::unnecessary_cast)] + pub(crate) fn get_mtime_sec(file_time: &FileTime) -> i64 { + #[cfg(windows)] + { + windows_filetime_to_unix_seconds(file_time.LastWriteTime) + } + #[cfg(unix)] + { + file_time.st_mtime as i64 + } + } + + #[ported( + source = "src/internal/fd_util/stub.c", + original = "moonbitlang_async_get_mtime_nsec" + )] + #[allow(dead_code)] + pub(crate) fn get_mtime_nsec(file_time: &FileTime) -> i32 { + #[cfg(windows)] + { + windows_filetime_to_nanoseconds(file_time.LastWriteTime) + } + #[cfg(unix)] + { + file_time.st_mtime_nsec as i32 + } + } + + #[ported( + source = "src/internal/fd_util/stub.c", + original = "moonbitlang_async_get_ctime_sec" + )] + #[allow(dead_code)] + #[allow(clippy::unnecessary_cast)] + pub(crate) fn get_ctime_sec(file_time: &FileTime) -> i64 { + #[cfg(windows)] + { + windows_filetime_to_unix_seconds(file_time.ChangeTime) + } + #[cfg(unix)] + { + file_time.st_ctime as i64 + } + } + + #[ported( + source = "src/internal/fd_util/stub.c", + original = "moonbitlang_async_get_ctime_nsec" + )] + #[allow(dead_code)] + pub(crate) fn get_ctime_nsec(file_time: &FileTime) -> i32 { + #[cfg(windows)] + { + windows_filetime_to_nanoseconds(file_time.ChangeTime) + } + #[cfg(unix)] + { + file_time.st_ctime_nsec as i32 + } + } +} + +pub(crate) fn pipe_host_files(files: &mut impl HostFileTable) -> AsyncHostResult<[i32; 2]> { + #[cfg(unix)] + { + use std::fs::File; + use std::os::fd::FromRawFd; + + let fds = pipe()?; + let read = unsafe { File::from_raw_fd(fds[0]) }; + let write = unsafe { File::from_raw_fd(fds[1]) }; + let read = files.insert_file(read)?; + let write = files.insert_file(write)?; + Ok([read, write]) + } + + #[cfg(windows)] + { + use std::ffi::OsString; + use std::os::windows::io::FromRawHandle; + use std::sync::atomic::{AtomicU64, Ordering}; + use windows_sys::Win32::Foundation::{CloseHandle, INVALID_HANDLE_VALUE}; + use windows_sys::Win32::System::Threading::GetCurrentProcessId; + + static PIPE_ID: AtomicU64 = AtomicU64::new(0); + + let pipe_id = PIPE_ID.fetch_add(1, Ordering::Relaxed) + 1; + let name = OsString::from(format!( + r"\\.\pipe\moonbitlang_async.{}.{}", + unsafe { GetCurrentProcessId() }, + pipe_id + )); + let write = create_named_pipe_server(&name, false); + if write == INVALID_HANDLE_VALUE { + return Err(last_native_error()); + } + let read = create_named_pipe_client(&name, false); + if read == INVALID_HANDLE_VALUE { + unsafe { + CloseHandle(write); + } + return Err(last_native_error()); + } + + let read = unsafe { std::fs::File::from_raw_handle(read as _) }; + let write = unsafe { std::fs::File::from_raw_handle(write as _) }; + let read = files.insert_file(read)?; + let write = files.insert_file(write)?; + Ok([read, write]) + } +} + +#[cfg(unix)] +fn fcntl_getfl(fd: RawFd) -> AsyncHostResult { + let flags = unsafe { libc::fcntl(fd, libc::F_GETFL) }; + if flags < 0 { + Err(last_native_error()) + } else { + Ok(flags) + } +} + +#[cfg(unix)] +fn fcntl_setfl(fd: RawFd, flags: i32) -> AsyncHostResult<()> { + if unsafe { libc::fcntl(fd, libc::F_SETFL, flags) } < 0 { + Err(last_native_error()) + } else { + Ok(()) + } +} + +fn last_native_error() -> AsyncHostError { + AsyncHostError::Native( + std::io::Error::last_os_error() + .raw_os_error() + .unwrap_or_else(|| AsyncHostError::Inval.errno()), + ) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn sizeof_file_time_matches_platform_stat_buffer() { + assert_eq!(sizeof_file_time(), std::mem::size_of::() as i32); + } + + #[cfg(windows)] + #[test] + fn windows_filetime_seconds_use_unix_epoch() { + let unix_epoch = WINDOWS_TO_UNIX_EPOCH_SECONDS * WINDOWS_TICKS_PER_SECOND; + + assert_eq!(windows_filetime_to_unix_seconds(unix_epoch), 0); + assert_eq!( + windows_filetime_to_unix_seconds(unix_epoch + WINDOWS_TICKS_PER_SECOND), + 1 + ); + assert_eq!(windows_filetime_to_nanoseconds(unix_epoch + 123), 12_300); + } + + #[cfg(unix)] + #[test] + fn unix_set_nonblocking_and_blocking_match_native_stub() { + let fds = pipe().unwrap(); + + assert!(!fd_is_nonblocking(fds[0]).unwrap()); + set_nonblocking(fds[0]).unwrap(); + assert!(fd_is_nonblocking(fds[0]).unwrap()); + set_blocking(fds[0]).unwrap(); + assert!(!fd_is_nonblocking(fds[0]).unwrap()); + + unsafe { + libc::close(fds[0]); + libc::close(fds[1]); + } + } +} diff --git a/crates/moonrun/src/async_sys/internal/mod.rs b/crates/moonrun/src/async_sys/internal/mod.rs new file mode 100644 index 000000000..9fc1d3668 --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/mod.rs @@ -0,0 +1,24 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +pub(crate) mod c_buffer; +pub(crate) mod env_util; +pub(crate) mod event_loop; +pub(crate) mod fd_util; +pub(crate) mod os_string; +pub(crate) mod time; diff --git a/crates/moonrun/src/async_sys/internal/os_string/mod.rs b/crates/moonrun/src/async_sys/internal/os_string/mod.rs new file mode 100644 index 000000000..fa380a27b --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/os_string/mod.rs @@ -0,0 +1,19 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +pub(crate) mod stub; diff --git a/crates/moonrun/src/async_sys/internal/os_string/stub.rs b/crates/moonrun/src/async_sys/internal/os_string/stub.rs new file mode 100644 index 000000000..b4e4c9727 --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/os_string/stub.rs @@ -0,0 +1,68 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_host::{AsyncHostError, AsyncHostResult}; +use crate::async_sys::ported_fns; + +ported_fns! { + #[ported( + source = "src/internal/os_string/stub.c", + original = "moonbitlang_async_c_buffer_as_string" + )] + #[allow(dead_code)] + pub(crate) fn c_buffer_as_string(buffer: &[u8], len: i32) -> AsyncHostResult { + let bytes = if len == -1 { + let nul = buffer + .chunks_exact(2) + .position(|unit| unit == [0, 0]) + .ok_or(AsyncHostError::Fault)?; + &buffer[..nul * 2] + } else { + let len = usize::try_from(len).map_err(|_| AsyncHostError::Fault)?; + buffer.get(..len).ok_or(AsyncHostError::Fault)? + }; + + if bytes.len() % 2 != 0 { + return Err(AsyncHostError::Inval); + } + + let units = bytes + .chunks_exact(2) + .map(|bytes| u16::from_ne_bytes([bytes[0], bytes[1]])); + String::from_utf16(&units.collect::>()).map_err(|_| AsyncHostError::Inval) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn c_buffer_as_string_decodes_explicit_byte_length() { + let bytes = [b'a', 0, 0x34, 0xd8, 0x1e, 0xdd, 0, 0]; + + assert_eq!(c_buffer_as_string(&bytes, 6).unwrap(), "a\u{1d11e}"); + } + + #[test] + fn c_buffer_as_string_uses_nul_terminated_length() { + let bytes = [b'o', 0, b'k', 0, 0, 0, b'x', 0]; + + assert_eq!(c_buffer_as_string(&bytes, -1).unwrap(), "ok"); + } +} diff --git a/crates/moonrun/src/async_sys/internal/time/clock.rs b/crates/moonrun/src/async_sys/internal/time/clock.rs new file mode 100644 index 000000000..267d04377 --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/time/clock.rs @@ -0,0 +1,53 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use std::sync::OnceLock; +use std::time::Instant; + +use crate::async_sys::ported_fns; + +static MONOTONIC_CLOCK_ORIGIN: OnceLock = OnceLock::new(); + +ported_fns! { + #[ported( + source = "src/internal/time/time.c", + original = "moonbitlang_async_get_ms_since_epoch" + )] + pub(crate) fn get_ms_since_epoch() -> i64 { + // The async timer code only relies on elapsed differences in + // millisecond precision. The absolute value and epoch are not part of + // the async contract, so the wasm host uses an arbitrary process-local + // monotonic origin instead of reproducing the native C stubs' wall + // clock epochs. + let origin = MONOTONIC_CLOCK_ORIGIN.get_or_init(Instant::now); + i64::try_from(origin.elapsed().as_millis()).unwrap_or(i64::MAX) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn clock_is_monotonic_from_arbitrary_origin() { + let first = get_ms_since_epoch(); + let second = get_ms_since_epoch(); + + assert!(second >= first); + } +} diff --git a/crates/moonrun/src/async_sys/internal/time/mod.rs b/crates/moonrun/src/async_sys/internal/time/mod.rs new file mode 100644 index 000000000..3ea1f2b85 --- /dev/null +++ b/crates/moonrun/src/async_sys/internal/time/mod.rs @@ -0,0 +1,19 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +pub(crate) mod clock; diff --git a/crates/moonrun/src/async_sys/mod.rs b/crates/moonrun/src/async_sys/mod.rs new file mode 100644 index 000000000..cfd66f362 --- /dev/null +++ b/crates/moonrun/src/async_sys/mod.rs @@ -0,0 +1,77 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +pub(crate) mod fs; +pub(crate) mod internal; +pub(crate) mod os_error; + +#[cfg(test)] +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)] +pub(crate) struct PortedSymbol { + pub(crate) rust_module: &'static str, + pub(crate) rust_symbol: &'static str, + pub(crate) native_symbol: &'static str, + pub(crate) source: &'static str, +} + +macro_rules! ported_fns { + ($( + #[ported(source = $source:literal, original = $original:literal)] + $(#[$meta:meta])* + $vis:vis fn $name:ident($($args:tt)*) $(-> $ret:ty)? $body:block + )+) => { + #[cfg(test)] + pub(crate) const PORTED_SYMBOLS: &[crate::async_sys::PortedSymbol] = &[ + $( + crate::async_sys::PortedSymbol { + rust_module: module_path!(), + rust_symbol: stringify!($name), + native_symbol: $original, + source: $source, + }, + )+ + ]; + + $( + $(#[$meta])* + $vis fn $name($($args)*) $(-> $ret)? $body + )* + }; +} + +pub(crate) use ported_fns; + +#[cfg(test)] +pub(crate) fn ported_symbols() -> Vec { + let mut symbols = Vec::new(); + symbols.extend_from_slice(internal::c_buffer::stub::PORTED_SYMBOLS); + symbols.extend_from_slice(internal::env_util::stub::PORTED_SYMBOLS); + symbols.extend_from_slice(internal::fd_util::stub::PORTED_SYMBOLS); + #[cfg(any(target_os = "linux", target_os = "macos"))] + symbols.extend_from_slice(internal::event_loop::io_unix::PORTED_SYMBOLS); + #[cfg(windows)] + symbols.extend_from_slice(internal::event_loop::io_windows::PORTED_SYMBOLS); + symbols.extend_from_slice(internal::event_loop::poll::PORTED_SYMBOLS); + symbols.extend(internal::event_loop::thread_pool::ported_symbols()); + symbols.extend_from_slice(internal::os_string::stub::PORTED_SYMBOLS); + symbols.extend_from_slice(internal::time::clock::PORTED_SYMBOLS); + symbols.extend_from_slice(fs::dir::PORTED_SYMBOLS); + symbols.extend_from_slice(fs::stub::PORTED_SYMBOLS); + symbols.extend_from_slice(os_error::stub::PORTED_SYMBOLS); + symbols +} diff --git a/crates/moonrun/src/async_sys/os_error/mod.rs b/crates/moonrun/src/async_sys/os_error/mod.rs new file mode 100644 index 000000000..fa380a27b --- /dev/null +++ b/crates/moonrun/src/async_sys/os_error/mod.rs @@ -0,0 +1,19 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +pub(crate) mod stub; diff --git a/crates/moonrun/src/async_sys/os_error/stub.rs b/crates/moonrun/src/async_sys/os_error/stub.rs new file mode 100644 index 000000000..980bfc5af --- /dev/null +++ b/crates/moonrun/src/async_sys/os_error/stub.rs @@ -0,0 +1,159 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +use crate::async_host::AsyncHost; +use crate::async_sys::ported_fns; + +ported_fns! { + #[ported( + source = "src/os_error/stub.c", + original = "moonbitlang_async_get_errno" + )] + pub(crate) fn get_errno(host: &AsyncHost) -> i32 { + host.get_errno() + } + + #[ported( + source = "src/os_error/stub.c", + original = "moonbitlang_async_is_nonblocking_io_error" + )] + pub(crate) fn is_nonblocking_io_error(errno: i32) -> bool { + #[cfg(unix)] + { + errno == libc::EAGAIN || errno == libc::EINPROGRESS || errno == libc::EWOULDBLOCK + } + #[cfg(windows)] + { + use windows_sys::Win32::Foundation::{ERROR_IO_INCOMPLETE, ERROR_IO_PENDING}; + errno == ERROR_IO_INCOMPLETE as i32 || errno == ERROR_IO_PENDING as i32 + } + } + + #[ported( + source = "src/os_error/stub.c", + original = "moonbitlang_async_is_EINTR" + )] + pub(crate) fn is_eintr(errno: i32) -> bool { + #[cfg(unix)] + { + errno == libc::EINTR + } + #[cfg(windows)] + { + let _ = errno; + false + } + } + + #[ported( + source = "src/os_error/stub.c", + original = "moonbitlang_async_is_ENOENT" + )] + pub(crate) fn is_enoent(errno: i32) -> bool { + #[cfg(unix)] + { + errno == libc::ENOENT + } + #[cfg(windows)] + { + use windows_sys::Win32::Foundation::{ERROR_FILE_NOT_FOUND, ERROR_PATH_NOT_FOUND}; + errno == ERROR_FILE_NOT_FOUND as i32 || errno == ERROR_PATH_NOT_FOUND as i32 + } + } + + #[ported( + source = "src/os_error/stub.c", + original = "moonbitlang_async_is_EEXIST" + )] + pub(crate) fn is_eexist(errno: i32) -> bool { + #[cfg(unix)] + { + errno == libc::EEXIST + } + #[cfg(windows)] + { + use windows_sys::Win32::Foundation::{ERROR_ALREADY_EXISTS, ERROR_FILE_EXISTS}; + errno == ERROR_FILE_EXISTS as i32 || errno == ERROR_ALREADY_EXISTS as i32 + } + } + + #[ported( + source = "src/os_error/stub.c", + original = "moonbitlang_async_is_EACCES" + )] + pub(crate) fn is_eacces(errno: i32) -> bool { + #[cfg(unix)] + { + errno == libc::EACCES + } + #[cfg(windows)] + { + use windows_sys::Win32::Foundation::ERROR_ACCESS_DENIED; + errno == ERROR_ACCESS_DENIED as i32 + } + } + + #[ported( + source = "src/os_error/stub.c", + original = "moonbitlang_async_is_ECONNREFUSED" + )] + pub(crate) fn is_econnrefused(errno: i32) -> bool { + #[cfg(unix)] + { + errno == libc::ECONNREFUSED + } + #[cfg(windows)] + { + use windows_sys::Win32::Foundation::ERROR_CONNECTION_REFUSED; + errno == ERROR_CONNECTION_REFUSED as i32 + } + } + + #[ported( + source = "src/os_error/stub.c", + original = "moonbitlang_async_is_ERROR_NOTIFY_ENUM_DIR" + )] + pub(crate) fn is_error_notify_enum_dir(errno: i32) -> bool { + #[cfg(windows)] + { + use windows_sys::Win32::Foundation::ERROR_NOTIFY_ENUM_DIR; + errno == ERROR_NOTIFY_ENUM_DIR as i32 + } + #[cfg(unix)] + { + let _ = errno; + false + } + } + + #[ported( + source = "src/os_error/stub.c", + original = "moonbitlang_async_get_ENOTDIR" + )] + pub(crate) fn get_enotdir() -> i32 { + #[cfg(unix)] + { + libc::ENOTDIR + } + #[cfg(windows)] + { + use windows_sys::Win32::Foundation::ERROR_DIRECTORY; + ERROR_DIRECTORY as i32 + } + } +} diff --git a/crates/moonrun/src/main.rs b/crates/moonrun/src/main.rs index bb9b97446..e1d6a0ad3 100644 --- a/crates/moonrun/src/main.rs +++ b/crates/moonrun/src/main.rs @@ -23,6 +23,9 @@ use std::path::Path; use std::{cell::Cell, io::Read, path::PathBuf, time::Instant}; use v8::V8::set_flags_from_string; +mod async_api; +mod async_host; +mod async_sys; mod backtrace_api; mod demangle_js_template; mod fs_api_temp; @@ -383,6 +386,11 @@ fn init_env( time.set_func(scope, "now", now); } + { + let async_runtime = global_proxy.child(scope, async_api::MOONBIT_V0_MODULE); + async_api::init_env(async_runtime, scope, dtors); + } + { let wasi = global_proxy.child(scope, "__moonbit_wasi_unstable"); wasi_api::init_env(wasi, scope, wasm_file_name, args, dtors); diff --git a/crates/moonrun/src/template/js_glue.js b/crates/moonrun/src/template/js_glue.js index 5886bcd1b..33899bcda 100644 --- a/crates/moonrun/src/template/js_glue.js +++ b/crates/moonrun/src/template/js_glue.js @@ -143,6 +143,7 @@ const __moonbit_backtrace_runtime = globalThis.__moonbit_backtrace_runtime || { })(__moonbit_fs_unstable, __moonbit_run_env); const __moonbit_wasi_unstable = globalThis.__moonbit_wasi_unstable || {}; +const moonbit_v0 = globalThis.moonbit_v0 || {}; delete globalThis.__moonbit_run_env; delete globalThis.__moonbit_backtrace_runtime; @@ -436,6 +437,7 @@ const spectest = { __moonbit_io_unstable: __moonbit_io_unstable, __moonbit_sys_unstable: __moonbit_sys_unstable, __moonbit_time_unstable: __moonbit_time_unstable, + moonbit_v0: moonbit_v0, wasi_snapshot_preview1: wasi_snapshot_preview1, moonbit: { string_to_js_string() { @@ -502,6 +504,13 @@ function setWasiMemory(memory) { wasiMemoryInitialized = true; } +function setAsyncMemory(memory) { + if (!(memory instanceof WebAssembly.Memory)) { + throw new Error("moonbit_v0 requires an exported `memory`"); + } + moonbit_v0.memory = memory; +} + try { if (typeof bytes === 'undefined') { bytes = read_file_to_bytes(module_name); @@ -514,12 +523,20 @@ try { const moduleExports = WebAssembly.Module.exports(module); const usesWasiSnapshotPreview1 = moduleImports .some(importItem => importItem.module === "wasi_snapshot_preview1"); + const usesMoonBitV0 = moduleImports + .some(importItem => importItem.module === "moonbit_v0"); if (usesWasiSnapshotPreview1) { const importedMemory = findImportedMemory(moduleImports, moduleExports, spectest); if (importedMemory instanceof WebAssembly.Memory) { setWasiMemory(importedMemory); } } + if (usesMoonBitV0) { + const importedMemory = findImportedMemory(moduleImports, moduleExports, spectest); + if (importedMemory instanceof WebAssembly.Memory) { + setAsyncMemory(importedMemory); + } + } let instance = new WebAssembly.Instance(module, spectest); if (usesWasiSnapshotPreview1) { const memory = instance.exports.memory; @@ -531,6 +548,16 @@ try { ); } } + if (usesMoonBitV0) { + const memory = instance.exports.memory; + if (memory instanceof WebAssembly.Memory) { + setAsyncMemory(memory); + } else if (!(moonbit_v0.memory instanceof WebAssembly.Memory)) { + throw new Error( + "moonbit_v0 requires an exported or imported WebAssembly.Memory" + ); + } + } if (test_mode) { for (param of testParams) { try { diff --git a/crates/moonrun/tests/test.rs b/crates/moonrun/tests/test.rs index ddf15070e..0fc81a4c1 100644 --- a/crates/moonrun/tests/test.rs +++ b/crates/moonrun/tests/test.rs @@ -310,6 +310,47 @@ fn test_moon_run_with_is_windows() { }); } +#[test] +fn test_moon_run_with_async_host_imports() { + let dir = TestDir::new("test_async_host.in"); + + moon_cmd() + .current_dir(&dir) + .args(["build", "--target", "wasm"]) + .assert() + .success(); + + let wasm_file = dir.join("_build/wasm/debug/build/main/main.wasm"); + + snapbox::cmd::Command::new(snapbox::cmd::cargo_bin!("moonrun")) + .arg(&wasm_file) + .assert() + .success() + .stdout_eq("ok\n"); +} + +#[test] +fn test_moon_run_with_async_host_invalid_c_buffer_traps() { + let dir = TestDir::new("test_async_host_invalid_c_buffer.in"); + + moon_cmd() + .current_dir(&dir) + .args(["build", "--target", "wasm"]) + .assert() + .success(); + + let wasm_file = dir.join("_build/wasm/debug/build/main/main.wasm"); + + snapbox::cmd::Command::new(snapbox::cmd::cargo_bin!("moonrun")) + .arg(&wasm_file) + .assert() + .failure() + .stderr_eq(snapbox::str![[r#" +Error: moonbit_v0.c_buffer/c_buffer_get failed: Fault +[..] +"#]]); +} + #[test] fn test_moon_fmt_skips_prebuild_output() { // Prepare a temp copy of the test case diff --git a/crates/moonrun/tests/test_cases/test_async_host.in/main/main.mbt b/crates/moonrun/tests/test_cases/test_async_host.in/main/main.mbt new file mode 100644 index 000000000..fbbbb8f7a --- /dev/null +++ b/crates/moonrun/tests/test_cases/test_async_host.in/main/main.mbt @@ -0,0 +1,107 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +fn get_platform() -> Int = "moonbit_v0" "runtime/get_platform" + +///| +fn ms_since_epoch() -> Int64 = "moonbit_v0" "time/get_ms_since_epoch" + +///| +fn sleep_ms(duration : Int) -> Int = "moonbit_v0" "time/sleep_ms" + +///| +fn get_errno() -> Int = "moonbit_v0" "os_error/get_errno" + +///| +fn get_tmp_path_len() -> Int = "moonbit_v0" "fs/get_tmp_path_len" + +///| +fn get_tmp_path(dst : Int, len : Int) -> Int = "moonbit_v0" "fs/get_tmp_path" + +///| +#borrow(bytes) +extern "wasm" fn bytes_to_ptr(bytes : Bytes) -> Int = + #|(func (param i32) (result i32) local.get 0) + +///| +#borrow(str) +extern "wasm" fn string_to_ptr(str : String) -> Int = + #|(func (param i32) (result i32) local.get 0) + +///| +fn blit_to_c(dst : Int, src : Int, offset : Int, len : Int) -> Unit = "moonbit_v0" "c_buffer/blit_to_c" + +///| +fn blit_from_c(src : Int, dst : Int, offset : Int, len : Int) -> Unit = "moonbit_v0" "c_buffer/blit_from_c" + +///| +fn c_buffer_get(buf : Int, index : Int) -> Byte = "moonbit_v0" "c_buffer/c_buffer_get" + +///| +fn strlen(buf : Int) -> Int = "moonbit_v0" "c_buffer/strlen" + +///| +fn pointer_is_null(ptr : Int) -> Bool = "moonbit_v0" "c_buffer/pointer_is_null" + +///| +fn null_pointer() -> Int = "moonbit_v0" "c_buffer/null_pointer" + +///| +fn errno_is_lock_violation(errno : Int) -> Bool = "moonbit_v0" "fs/errno_is_lock_violation" + +///| +fn make_tcp_socket(family : Int) -> Int = "moonbit_v0" "socket/make_tcp_socket" + +///| +fn assert_true(cond : Bool, message : String) -> Unit { + if !cond { + abort(message) + } +} + +///| +fn main { + let platform = get_platform() + assert_true(platform >= 0 && platform <= 2, "invalid platform") + let before = ms_since_epoch() + assert_true(sleep_ms(1) == 0, "sleep failed") + let after = ms_since_epoch() + assert_true(after >= before, "time moved backwards") + let tmp_path_len = get_tmp_path_len() + assert_true(tmp_path_len > 0, "tmp path size query failed") + assert_true(get_tmp_path(0, 0) == -1, "tmp path fill accepted a missing buffer") + let tmp_path = String::make(tmp_path_len, '\u{0}') + let tmp_path_ptr = string_to_ptr(tmp_path) + assert_true(get_tmp_path(tmp_path_ptr, tmp_path_len) == 0, "tmp path fill failed") + assert_true(tmp_path != String::make(tmp_path_len, '\u{0}'), "tmp path fill wrote an empty path") + let src = b"abcdef\x00" + let dst = Bytes::make(6, b'_') + blit_to_c(bytes_to_ptr(dst), bytes_to_ptr(src), 2, 3) + assert_true(dst[0] == b'c' && dst[1] == b'd' && dst[2] == b'e', "blit_to_c failed") + let dst2 = Bytes::make(6, b'_') + blit_from_c(bytes_to_ptr(src), bytes_to_ptr(dst2), 2, 3) + assert_true(dst2[2] == b'a' && dst2[3] == b'b' && dst2[4] == b'c', "blit_from_c failed") + assert_true(c_buffer_get(bytes_to_ptr(src), 1) == b'b', "c_buffer_get failed") + assert_true(strlen(bytes_to_ptr(src)) == 6, "strlen failed") + assert_true(null_pointer() == 0, "null pointer value changed") + assert_true(pointer_is_null(0), "null pointer check failed") + assert_true(!errno_is_lock_violation(0), "zero errno is a lock violation") + assert_true(make_tcp_socket(0) == -1, "unsupported socket stub succeeded") + assert_true(get_errno() != 0, "unsupported socket stub did not set errno") + println("ok") +} diff --git a/crates/moonrun/tests/test_cases/test_async_host.in/main/moon.pkg.json b/crates/moonrun/tests/test_cases/test_async_host.in/main/moon.pkg.json new file mode 100644 index 000000000..b29a60c86 --- /dev/null +++ b/crates/moonrun/tests/test_cases/test_async_host.in/main/moon.pkg.json @@ -0,0 +1,3 @@ +{ + "is-main": true +} diff --git a/crates/moonrun/tests/test_cases/test_async_host.in/moon.mod.json b/crates/moonrun/tests/test_cases/test_async_host.in/moon.mod.json new file mode 100644 index 000000000..90ee495d4 --- /dev/null +++ b/crates/moonrun/tests/test_cases/test_async_host.in/moon.mod.json @@ -0,0 +1,3 @@ +{ + "name": "username/async-host" +} diff --git a/crates/moonrun/tests/test_cases/test_async_host_invalid_c_buffer.in/main/main.mbt b/crates/moonrun/tests/test_cases/test_async_host_invalid_c_buffer.in/main/main.mbt new file mode 100644 index 000000000..8e7925460 --- /dev/null +++ b/crates/moonrun/tests/test_cases/test_async_host_invalid_c_buffer.in/main/main.mbt @@ -0,0 +1,24 @@ +// moon: The build system and package manager for MoonBit. +// Copyright (C) 2024 International Digital Economy Academy +// +// This program is free software: you can redistribute it and/or modify +// it under the terms of the GNU Affero General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU Affero General Public License for more details. +// +// You should have received a copy of the GNU Affero General Public License +// along with this program. If not, see . +// +// For inquiries, you can contact us via e-mail at jichuruanjian@idea.edu.cn. + +fn c_buffer_get(buf : Int, index : Int) -> Byte = "moonbit_v0" "c_buffer/c_buffer_get" + +///| +fn main { + ignore(c_buffer_get(2147483647, 0)) +} diff --git a/crates/moonrun/tests/test_cases/test_async_host_invalid_c_buffer.in/main/moon.pkg.json b/crates/moonrun/tests/test_cases/test_async_host_invalid_c_buffer.in/main/moon.pkg.json new file mode 100644 index 000000000..b29a60c86 --- /dev/null +++ b/crates/moonrun/tests/test_cases/test_async_host_invalid_c_buffer.in/main/moon.pkg.json @@ -0,0 +1,3 @@ +{ + "is-main": true +} diff --git a/crates/moonrun/tests/test_cases/test_async_host_invalid_c_buffer.in/moon.mod.json b/crates/moonrun/tests/test_cases/test_async_host_invalid_c_buffer.in/moon.mod.json new file mode 100644 index 000000000..215525c12 --- /dev/null +++ b/crates/moonrun/tests/test_cases/test_async_host_invalid_c_buffer.in/moon.mod.json @@ -0,0 +1,3 @@ +{ + "name": "username/async-host-invalid-c-buffer" +} diff --git a/docs/adr/0001-async-wasm-host-boundary.md b/docs/adr/0001-async-wasm-host-boundary.md new file mode 100644 index 000000000..cadcf4b8f --- /dev/null +++ b/docs/adr/0001-async-wasm-host-boundary.md @@ -0,0 +1,22 @@ +# Async Wasm Host Boundary + +We will support `moonbitlang/async` on the wasm backend through `#cfg(target="wasm")` bindings to a `moonbit_v0` host module in `moonrun`, without compiler changes or JS backend changes. The host follows the semantic contract of async's native C stubs, but wasm resources are represented as host handles and guest-memory ranges rather than native MoonBit runtime objects or pinned pointers; this keeps V8 as the first adapter while allowing the V8-free host core to be reused by future wasm runtimes. + +## Decisions + +- Use a semantic C-stub boundary: `moonbit_v0` symbols map from `moonbitlang_async_*` symbols by stripping that prefix. +- Keep source provenance next to import registration and V8-free host implementation. Each mapped import declares the async source file and native symbol it tracks, each active Rust port declares the same origin through the port provenance macro, and tests verify the registry and implementation entries stay in sync. V8 callback modules are adapters only and should not own port provenance. +- Keep `AsyncHost` focused on shared runtime state, resource tables, guest-memory helpers, and shared ABI representation types. Ported operation behavior belongs in `async_sys`, with `AsyncHost` passed in only when the operation needs shared state. +- Use monotonic elapsed milliseconds with an unspecified process-local origin for wasm async time. The async timer contract uses differences between readings; the absolute epoch is not meaningful. +- Support Unix-family and Windows hosts first. Other host families are deliberately compile-time unsupported until the async C-stub parity target is defined for them. +- Store wasm memory as `moonbit_v0.memory` in the JS glue. The Rust adapter reads that property on each memory-using import instead of registering a separate async `set_memory` callback. +- Never retain raw wasm-memory pointers after an import returns. Host state may store handles, guest offsets, lengths, and host-owned buffers. +- Pass async path arguments as borrowed MoonBit `String` pointers plus UTF-16 code-unit lengths. The guest must not encode `OsString` paths to UTF-8 `Bytes` before calling `moonbit_v0`; `moonrun` owns conversion from MoonBit string data into Rust `OsString` and then into the native OS call form. +- Treat V8 memory growth as a reason to reacquire memory every call. OS APIs that need stable pointers must use host-owned memory and copy to or from wasm memory. +- Keep worker threads out of wasm guest memory. Worker jobs may compute host-owned results, but guest-memory writes happen only during guest-thread imports such as `fetch_completion`, where the V8 adapter can reacquire the current memory view while notifying MoonBit that jobs are ready. +- Treat wasm `FileTime` as a portable 48-byte record, not as a native `stat` or `FILE_BASIC_INFO` buffer. The wasm layout is little-endian `{ atime_sec: i64, atime_nsec: i32, mtime_sec: i64, mtime_nsec: i32, ctime_sec: i64, ctime_nsec: i32 }` with 4 bytes of padding after each nanosecond field, matching the WIT canonical record layout for those fields on wasm32. +- Keep unsupported MVP symbols registered when they are part of the mapped boundary, but make them return native-style unsupported errors instead of causing missing-import instantiation failures. + +## Deferred + +Executor design, cancellation semantics, broad core FS, sockets, process, TLS, file watching, arbitrary external fd/HANDLE adoption, Wasmtime/WasmEdge adapters, and wasm-gc support remain follow-up work. diff --git a/licenserc.toml b/licenserc.toml index 6d9b7c6f0..75d57dfc0 100644 --- a/licenserc.toml +++ b/licenserc.toml @@ -44,6 +44,7 @@ excludes = [ "crates/moon/tests/test_cases/**", "!crates/moon/tests/test_cases/mod.rs", "crates/moonbuild/template/**", + "third_party/**", ".justfile", "moon.test", "*.js", diff --git a/third_party/moonbitlang_async b/third_party/moonbitlang_async new file mode 160000 index 000000000..f7d37085c --- /dev/null +++ b/third_party/moonbitlang_async @@ -0,0 +1 @@ +Subproject commit f7d37085ce1e7d29b72102bdc65347db78e66e50