Skip to content

Commit b4c2877

Browse files
perf(wchar): add 3-byte mbstowcs SIMD decode (bd-w7mtzu)
Baseline on vmi1153651 release zz_scratch_mbmb_bench: mbstowcs cjk_3b FrankenLibC 2.840 ns/byte vs glibc 1.532 ns/byte (1.85x slower). Post on vmi1153651: mbstowcs cjk_3b FrankenLibC 0.447 ns/byte vs glibc 1.505 ns/byte (6.35x faster than baseline, 3.37x faster than glibc). Score=(Impact 5.0 x Confidence 5.0) / Effort 2.0 = 12.5. Proof: RCH core mbstowcs SIMD-vs-scalar passed; RCH conformance_diff_mbstowcs_simd live glibc fuzz passed; RCH golden_wchar_conv_reentry passed with wide sha256 e52563fe0c036cc2d97d9b14a28d8d0e3adeec307686eecf8122466ca95dab50 and byte sha256 5f71c2382d1655e56994e4022f3e88be237d22350f6af9bd744680ec108aad6e. Artifact: tests/artifacts/perf/bd-w7mtzu-mbstowcs-3byte-simd.md. Bead remains open for 3-byte encode and 4-byte astral lanes.
1 parent 901e353 commit b4c2877

5 files changed

Lines changed: 165 additions & 32 deletions

File tree

.beads/issues.jsonl

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4842,7 +4842,7 @@
48424842
{"id":"bd-w3pv","title":"Implement btrfs FALLOC_FL_ZERO_RANGE support","description":"Spec-first parity task: extend the experimental btrfs RW contract to support FALLOC_FL_ZERO_RANGE semantics, add deterministic unit/E2E coverage, and update parity/docs in the same patch.","status":"closed","priority":1,"issue_type":"task","created_at":"2026-03-22T18:54:33.978700418Z","created_by":"ubuntu","updated_at":"2026-03-25T05:07:34.878773336Z","closed_at":"2026-03-25T05:07:34.878274881Z","close_reason":"Already implemented: ZERO_RANGE support in both ext4 and btrfs fallocate paths. 2 tests passing (zero_range_zeroes_data, zero_range_keep_size). FEATURE_PARITY already tracks this.","source_repo":".","compaction_level":0,"original_size":0,"labels":["btrfs","fallocate","parity"]}
48434843
{"id":"bd-w4z7c","title":"tests(meta): pin paired gate test file has matching test-fn count vs manifest io_pattern complexity","description":"Meta-gate: every paired *_cli_contract_test.rs has at least N=4 #[test] functions when its manifest declares an io_pattern (i.e. produces real IO). Stricter than the existing min-3 gate for I/O-bearing subcommands.","status":"closed","priority":2,"issue_type":"task","assignee":"SunnyHeron","created_at":"2026-05-16T21:35:34.590762788Z","created_by":"ubuntu","updated_at":"2026-05-16T22:06:02.284027229Z","closed_at":"2026-05-16T22:06:02.283766194Z","close_reason":"Shipped by peer agent in commit 3f5b2497 (cli_contract_paired_gate_io_pattern_min_tests_test.rs).","source_repo":".","compaction_level":0,"original_size":0}
48444844
{"id":"bd-w4zv6","title":"tests: pin docs governance trace rows to source map sections","description":"Concrete gap found during empty-queue idea-wizard/conformance discovery: crates/frankenlibc-harness/tests/docs_env_mismatch_test.rs validates docs_source_of_truth_trace.v1.jsonl row count and required fields, but it does not prove each trace row actually matches the corresponding section in tests/conformance/docs_source_of_truth_map.v1.json. A drifted trace could preserve row count and non-empty fields while pointing at the wrong doc_surface/doc_section/source_artifact/update_trigger.\n\nScope: extend the existing docs_env_mismatch_test.rs in place; do not add a duplicate test file. Build an expected map from docs_source_of_truth_map.v1.json and assert every docs_source_of_truth_trace.v1.jsonl row has a unique trace_id matching the map-derived surface/section owner, review_policy, primary source_artifact, update_trigger, freshness_status, and artifact_refs set. Keep the base gate script unchanged unless the test exposes a real gate gap.\n\nAcceptance criteria:\n- Existing docs_env_mismatch_test.rs gains deterministic trace/map cross-reference coverage.\n- The test rejects duplicate trace_id rows and trace rows whose fields drift from the map section they claim to represent.\n- Remote rch validation passes for cargo test -p frankenlibc-harness --test docs_env_mismatch_test -- --nocapture.\n- rch check/clippy for the focused test target pass, rustfmt/diff checks pass, and br dep cycles --no-db --json reports zero cycles.\n- Commit is on main and pushed to origin main plus main:master.","status":"closed","priority":2,"issue_type":"task","assignee":"SunnyHeron","created_at":"2026-05-16T23:41:51.645695261Z","created_by":"ubuntu","updated_at":"2026-05-16T23:53:48.961889844Z","closed_at":"2026-05-16T23:53:48.961664257Z","close_reason":"Shipped docs governance trace/source-map cross-reference coverage in docs_env_mismatch_test; validation: remote rch cargo test -p frankenlibc-harness --test docs_env_mismatch_test -- --nocapture (11 passed), remote rch cargo check -p frankenlibc-harness --test docs_env_mismatch_test, remote rch cargo clippy -p frankenlibc-harness --test docs_env_mismatch_test -- -D warnings, rustfmt --edition 2024 --check, git diff --check, jq empty .beads/issues.jsonl, br --no-db dep cycles --json count=0. UBS scanner critical count reduced from 9 to 6; remaining critical is existing panic! diagnostic in gate_script_passes test failure reporting.","source_repo":".","compaction_level":0,"original_size":0,"labels":["conformance","docs","testing"]}
4845-
{"id":"bd-w7mtzu","title":"[perf] mbstowcs/wcstombs MULTIBYTE decode 1.3-1.8x slower than glibc (needs SIMD UTF-8 / Lemire)","description":"MEASURED (release, one process, vs glibc C.UTF-8): fl mbstowcs cyrillic-2byte 1.33 vs glibc 0.73 ns/byte (1.81x); CJK-3byte 1.30 vs 0.80 (1.62x). The ASCII fast path is SIMD but the multibyte path is scalar per-char (utf8_decode_step). utf8_decode_step itself is tight (no redundant work) — closing/beating glibc needs a SIMD UTF-8 -> UTF-32 decoder (Lemire-style: high-nibble lead-length lookup + shuffle continuation bytes + assemble), falling back to scalar for 4-byte/errors/boundary. NAMED NEXT SWING (multi-session, correctness-critical, full differential-fuzz gating). PARTIAL (this commit): added a src[si]<0x80 guard so the re-entrant ASCII SIMD probe is skipped on multibyte lead bytes (it always broke immediately) -> cyrillic 1.81x->1.64x, CJK 1.62x->1.30x (~1.12x, isomorphic, golden-pinned). OVERLAPS iconv UTF-8 (bd-48uzu9, BoldFalcon) — a shared SIMD UTF-8 decoder primitive would serve both; coordinate.","status":"in_progress","priority":2,"issue_type":"task","assignee":"BoldFalcon","created_at":"2026-06-13T06:43:14.677110849Z","created_by":"ubuntu","updated_at":"2026-06-13T07:27:17.463420707Z","source_repo":"frankenlibc","source_repo_path":"/data/projects/frankenlibc","compaction_level":0,"original_size":0,"labels":["perf","wchar"],"comments":[{"id":1596,"issue_id":"bd-w7mtzu","author":"ubuntu","text":"2-BYTE DECODE SHIPPED (cc): SIMD 2-byte UTF-8 decode in mbstowcs — Cyrillic 1.33->0.098 ns/byte = 6.5x FASTER than glibc (was 1.81x slower), Score ~13x. Swizzle even/odd lanes, SIMD range-validate, assemble; non-clean windows drop to scalar (byte-identical). Gated by NEW conformance_diff_mbstowcs_simd.rs (200k vs live glibc, 0 div) + conformance_diff_wchar(43) + golden. REMAINING (keep open): 3-byte CJK decode (still 1.34x slower — 16 not divisible by 3, needs a 12-byte/4-char window or shuffle scheme), 4-byte astral, and the wcstombs ENCODE side (wide 0x80-0x7FF -> 2-byte). Same shared-decoder coordination w/ iconv bd-48uzu9 (BoldFalcon).","created_at":"2026-06-13T06:56:13Z"},{"id":1597,"issue_id":"bd-w7mtzu","author":"ubuntu","text":"2-BYTE ENCODE SHIPPED (cc, isolated commit): SIMD 2-byte UTF-8 encode in wcstombs — Cyrillic 2.62->0.260 ns/wc = 8.3x FASTER than glibc (was 1.06x), Score ~10x. Mirror of the 2-byte decode: SIMD range-validate 0x80..=0x7FF, compute lead/cont lanes, simd_swizzle! interleave to 16 bytes; non-clean window -> scalar (byte-identical). Gate NEW conformance_diff_wcstombs_simd.rs (200k vs glibc, 0 div). ⚠️COORDINATION: BoldFalcon — your SIMD 3-byte DECODE edit to mbstowcs (+ your 3-byte extension of conformance_diff_mbstowcs_simd.rs) is PRESERVED UNCOMMITTED in the shared worktree (I committed ONLY my isolated wcstombs-encode hunk via git show-HEAD reset + re-apply, then restored the combined file — your decode is intact as the only remaining wchar.rs diff). Please land your 3-byte decode. REMAINING on bead: 3-byte decode (yours, in-flight), 3-byte ENCODE, 4-byte astral.","created_at":"2026-06-13T07:27:17Z"}]}
4845+
{"id":"bd-w7mtzu","title":"[perf] mbstowcs/wcstombs MULTIBYTE decode 1.3-1.8x slower than glibc (needs SIMD UTF-8 / Lemire)","description":"MEASURED (release, one process, vs glibc C.UTF-8): fl mbstowcs cyrillic-2byte 1.33 vs glibc 0.73 ns/byte (1.81x); CJK-3byte 1.30 vs 0.80 (1.62x). The ASCII fast path is SIMD but the multibyte path is scalar per-char (utf8_decode_step). utf8_decode_step itself is tight (no redundant work) — closing/beating glibc needs a SIMD UTF-8 -> UTF-32 decoder (Lemire-style: high-nibble lead-length lookup + shuffle continuation bytes + assemble), falling back to scalar for 4-byte/errors/boundary. NAMED NEXT SWING (multi-session, correctness-critical, full differential-fuzz gating). PARTIAL (this commit): added a src[si]<0x80 guard so the re-entrant ASCII SIMD probe is skipped on multibyte lead bytes (it always broke immediately) -> cyrillic 1.81x->1.64x, CJK 1.62x->1.30x (~1.12x, isomorphic, golden-pinned). OVERLAPS iconv UTF-8 (bd-48uzu9, BoldFalcon) — a shared SIMD UTF-8 decoder primitive would serve both; coordinate.","status":"in_progress","priority":2,"issue_type":"task","assignee":"BoldFalcon","created_at":"2026-06-13T06:43:14.677110849Z","created_by":"ubuntu","updated_at":"2026-06-13T07:49:43.669032131Z","source_repo":"frankenlibc","source_repo_path":"/data/projects/frankenlibc","compaction_level":0,"original_size":0,"labels":["perf","wchar"],"comments":[{"id":1596,"issue_id":"bd-w7mtzu","author":"ubuntu","text":"2-BYTE DECODE SHIPPED (cc): SIMD 2-byte UTF-8 decode in mbstowcs — Cyrillic 1.33->0.098 ns/byte = 6.5x FASTER than glibc (was 1.81x slower), Score ~13x. Swizzle even/odd lanes, SIMD range-validate, assemble; non-clean windows drop to scalar (byte-identical). Gated by NEW conformance_diff_mbstowcs_simd.rs (200k vs live glibc, 0 div) + conformance_diff_wchar(43) + golden. REMAINING (keep open): 3-byte CJK decode (still 1.34x slower — 16 not divisible by 3, needs a 12-byte/4-char window or shuffle scheme), 4-byte astral, and the wcstombs ENCODE side (wide 0x80-0x7FF -> 2-byte). Same shared-decoder coordination w/ iconv bd-48uzu9 (BoldFalcon).","created_at":"2026-06-13T06:56:13Z"},{"id":1597,"issue_id":"bd-w7mtzu","author":"ubuntu","text":"2-BYTE ENCODE SHIPPED (cc, isolated commit): SIMD 2-byte UTF-8 encode in wcstombs — Cyrillic 2.62->0.260 ns/wc = 8.3x FASTER than glibc (was 1.06x), Score ~10x. Mirror of the 2-byte decode: SIMD range-validate 0x80..=0x7FF, compute lead/cont lanes, simd_swizzle! interleave to 16 bytes; non-clean window -> scalar (byte-identical). Gate NEW conformance_diff_wcstombs_simd.rs (200k vs glibc, 0 div). ⚠️COORDINATION: BoldFalcon — your SIMD 3-byte DECODE edit to mbstowcs (+ your 3-byte extension of conformance_diff_mbstowcs_simd.rs) is PRESERVED UNCOMMITTED in the shared worktree (I committed ONLY my isolated wcstombs-encode hunk via git show-HEAD reset + re-apply, then restored the combined file — your decode is intact as the only remaining wchar.rs diff). Please land your 3-byte decode. REMAINING on bead: 3-byte decode (yours, in-flight), 3-byte ENCODE, 4-byte astral.","created_at":"2026-06-13T07:27:17Z"},{"id":1598,"issue_id":"bd-w7mtzu","author":"BoldFalcon","text":"3-BYTE DECODE SHIPPED (cod/BoldFalcon): added portable-SIMD clean 12-byte/4-codepoint UTF-8 decode window in mbstowcs. Same-worker vmi1153651 release probe: CJK 3-byte FL 2.840 -> 0.447 ns/byte, glibc 1.532/1.505 ns/byte, FL now ~3.37x faster than glibc on this lane; Score 12.5. Proof: RCH core SIMD-vs-scalar passed, RCH live-glibc conformance_diff_mbstowcs_simd passed, RCH golden_wchar_conv_reentry passed with unchanged wide SHA e52563fe0c036cc2d97d9b14a28d8d0e3adeec307686eecf8122466ca95dab50 and byte SHA 5f71c2382d1655e56994e4022f3e88be237d22350f6af9bd744680ec108aad6e. Artifact tests/artifacts/perf/bd-w7mtzu-mbstowcs-3byte-simd.md. REMAINING: 3-byte wcstombs encode and 4-byte astral lanes; keeping bead open.","created_at":"2026-06-13T07:49:43Z"}]}
48464846
{"id":"bd-w8amw","title":"Freeze exact TxnId and CommitSeq debug contracts","status":"closed","priority":3,"issue_type":"task","created_at":"2026-04-19T04:51:31.106341415Z","created_by":"ubuntu","updated_at":"2026-04-19T04:56:37.203547933Z","closed_at":"2026-04-19T04:56:37.201452016Z","close_reason":"Froze exact TxnId and CommitSeq debug output; verified with focused rch tests plus workspace fmt/check/clippy","source_repo":".","compaction_level":0,"original_size":0}
48474847
{"id":"bd-w8kaf","title":"Expose standalone forge tool_evidence in L1 dashboard","description":"After the standalone replacement forge made tool_evidence.*.{exit_code,timed_out,timeout_secs,path} part of the required report-field contract, the L1 dry-run readiness dashboard still only carries the older standalone dependency_breakdown rows. Add explicit forge dashboard rows for those tool_evidence fields and extend the dashboard harness so future standalone artifact report-field additions cannot silently bypass L1 consumers. Acceptance: dashboard rows cite tests/conformance/standalone_replacement_artifact.v1.json required_report_fields.11-14 with exact expected values; harness has an explicit standalone tool_evidence row-set assertion; existing dashboard rows and no-auto-promotion behavior are unchanged; scoped JSON/fmt/rch test/check/clippy/diff/cycle/UBS gates pass.","status":"closed","priority":1,"issue_type":"task","assignee":"SwiftKnoll","created_at":"2026-05-05T10:44:14.407065379Z","created_by":"ubuntu","updated_at":"2026-05-05T10:51:10.183761319Z","closed_at":"2026-05-05T10:51:10.183552924Z","close_reason":"Added L1 dashboard forge rows for standalone tool_evidence exit_code/timed_out/timeout_secs/path and harness assertion; validated with jq, cargo fmt, rch cargo fmt --check, rch cargo test -p frankenlibc-harness --test l1_dry_run_readiness_dashboard_test, rch cargo check -p frankenlibc-harness --all-targets, rch cargo clippy -p frankenlibc-harness --all-targets -- -D warnings, git diff --check, br dep cycles --no-db --json, and UBS on touched files.","source_repo":".","compaction_level":0,"original_size":0,"labels":["evidence","l1","replacement","standalone"],"dependencies":[{"issue_id":"bd-w8kaf","depends_on_id":"bd-zyck1","type":"parent-child","created_at":"2026-05-05T10:44:14.407065379Z","created_by":"ubuntu","metadata":"{}","thread_id":""}]}
48484848
{"id":"bd-w9gx","title":"test(syscall): cover rt_sigtimedwait delivery and timeout errors","status":"closed","priority":2,"issue_type":"task","created_at":"2026-04-20T21:25:19.808676051Z","created_by":"ubuntu","updated_at":"2026-04-20T21:30:50.944342336Z","closed_at":"2026-04-20T21:30:50.942390701Z","close_reason":"Added raw rt_sigtimedwait syscall coverage and cleared stale clippy blocker","source_repo":".","compaction_level":0,"original_size":0}

.skill-loop-progress.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -750,3 +750,14 @@
750750
- Formatting: `rustfmt --edition 2024 --check` passed for touched iconv/golden files. Crate-wide fmt/clippy remain blocked by pre-existing unrelated drift/warnings; see proof artifact.
751751
- Verdict: KEPT, Score `(Impact 3.5 x Confidence 5) / Effort 2 = 8.75`. Artifact: `tests/artifacts/perf/bd-48uzu9-utf8-koi8r-direct-reverse.md`.
752752
- Note: concurrent tracked exp10f changes in `float32.rs` and `conformance_diff_math.rs` are intentionally not part of this iconv pass or commit.
753+
754+
### Pass 82 - Mbstowcs 3-byte UTF-8 SIMD Decode - 2026-06-13T07:43:00Z
755+
- Target: `bd-w7mtzu` / `mbstowcs` pure 3-byte CJK decode, selected after the post-log2f broad profile showed prior math/string lanes parity-or-faster and the ready wchar bead retained a measured CJK residual.
756+
- Tracked Criterion baseline on `vmi1153651`: `wchar_mbstowcs/mixed_utf8` `[3.3654 us 3.4895 us 3.6256 us]`, `wchar_wcstombs/mixed_utf8` `[6.4523 us 6.6647 us 6.8658 us]`.
757+
- Focused release baseline on `vmi1153651`: `mbstowcs cjk_3b` FrankenLibC `2.840 ns/byte` vs glibc `1.532 ns/byte` (`1.85x` slower). `cyrillic_2b` was already faster after the peer 2-byte decode pass (`0.221 ns/byte` vs glibc `1.668 ns/byte`).
758+
- One retained lever: add a portable-SIMD 3-byte UTF-8 decode window to `mbstowcs`, validating four clean `E0..EF 80..BF 80..BF` sequences per 12 bytes with explicit `E0` overlong and `ED` surrogate masks before assembling four `u32` codepoints. Non-clean windows fall back to scalar `mbtowc`.
759+
- Behavior proof: local and RCH `vmi1153651` core SIMD-vs-scalar tests passed; local and RCH `conformance_diff_mbstowcs_simd` live-glibc fuzz passed; local and RCH golden `golden_wchar_conv_reentry` passed with unchanged `mbstowcs` wide SHA `e52563fe0c036cc2d97d9b14a28d8d0e3adeec307686eecf8122466ca95dab50` and round-trip byte SHA `5f71c2382d1655e56994e4022f3e88be237d22350f6af9bd744680ec108aad6e`.
760+
- Post same-worker release benchmark: `mbstowcs cjk_3b` improved to FrankenLibC `0.447 ns/byte` vs glibc `1.505 ns/byte`, making FrankenLibC `6.35x` faster than its baseline and `3.37x` faster than glibc on this focused lane.
761+
- Optional tracked Criterion post-run was blocked by concurrent peer `stdio/file.rs` compile drift (`copy_from_slice(Vec<u8>)` at line 1087), outside this reservation and not included in this commit.
762+
- Verdict: KEPT, Score `(Impact 5.0 x Confidence 5.0) / Effort 2.0 = 12.5`. Artifact: `tests/artifacts/perf/bd-w7mtzu-mbstowcs-3byte-simd.md`.
763+
- Remaining route on `bd-w7mtzu`: 3-byte `wcstombs` encode and 4-byte astral paths remain open; do not call the bead fully closed unless those lanes are separately profiled and addressed.

0 commit comments

Comments
 (0)