Skip to content

Commit d3f45e1

Browse files
dschuffdschuff
authored
Add CI script for syncing selected WebAssembly proposal tests (#59714)
* Prototype script for adding WebAssembly proposal tests The current update-wasm-tests workflow updates the wasm core tests by checking out the spec repo and regenerating them. This PR also adds support for updating proposals that have not yet been merged into the main spec. It does this by checking out the proposal repo, determining which tests are updated compared to the main spec, building the tests (with the fork's interpreter), and copying just the updated tests into WPT as tentative tests. * add stack switching * filter out non-wast files * fix trailing whitespace * address some of the review feedback * Record udpated and merged proposals for PR body * Simplify Wasm test update workflow and improve PR body details - Update `update-proposals.py` to include repository URLs and commit hashes in the proposals summary. - Refactor `update-wasm-tests.yml` workflow to: - Dynamically determine if core spec tests, proposal tests, or both have changed. - Simplify the commit message to "Update Wasm tests" with a suffix indicating the changes (core, proposals, or both). - Combine the PR body generation and PR creation steps. - Use `gh pr create --body-file` to pass the PR body, eliminating the complex multi-line output block. - Include detailed repository and revision information for both core spec and proposals in the PR body. * remove trailing whitespace
1 parent 4120ac0 commit d3f45e1

4 files changed

Lines changed: 212 additions & 8 deletions

File tree

‎.github/workflows/update-wasm-tests.yml‎

Lines changed: 54 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,9 @@ jobs:
3535
# Replace wasm/core entirely, but preserve WEB_FEATURES.yml files.
3636
run: |
3737
rsync -a --delete --exclude 'WEB_FEATURES.yml' wasm-spec/out/ wpt/wasm/core/
38+
- name: Update Wasm proposal tests
39+
run: |
40+
python3 wpt/wasm/tools/update-proposals.py
3841
- name: Commit changes
3942
id: commit
4043
continue-on-error: true
@@ -45,13 +48,36 @@ jobs:
4548
git config user.name "$GIT_AUTHOR_NAME"
4649
git config user.email "$GIT_AUTHOR_EMAIL"
4750
git checkout -b $BRANCH_NAME
48-
git add wasm/core/
51+
git add wasm/core/ wasm/proposals
52+
53+
CORE_CHANGED=false
54+
git diff --cached --quiet -- wasm/core/ || CORE_CHANGED=true
55+
PROPOSALS_CHANGED=false
56+
git diff --cached --quiet -- wasm/proposals/ || PROPOSALS_CHANGED=true
57+
58+
echo "CORE_CHANGED=$CORE_CHANGED" >> $GITHUB_ENV
59+
echo "PROPOSALS_CHANGED=$PROPOSALS_CHANGED" >> $GITHUB_ENV
60+
4961
UPSTREAM_COMMIT=$(cd ../wasm-spec && git rev-parse HEAD)
5062
UPSTREAM_SHORT=$(cd ../wasm-spec && git -c core.abbrev=auto rev-parse --short HEAD)
5163
echo "UPSTREAM_COMMIT=$UPSTREAM_COMMIT" >> $GITHUB_ENV
5264
echo "UPSTREAM_SHORT=$UPSTREAM_SHORT" >> $GITHUB_ENV
53-
git commit -m "Update Wasm tests from $UPSTREAM_REPO@$UPSTREAM_SHORT" \
54-
--trailer "Upstream: https://github.com/$UPSTREAM_REPO/commit/$UPSTREAM_COMMIT"
65+
66+
SUFFIX=""
67+
if [ "$CORE_CHANGED" = "true" ] && [ "$PROPOSALS_CHANGED" = "true" ]; then
68+
SUFFIX=" (core, proposals)"
69+
elif [ "$CORE_CHANGED" = "true" ]; then
70+
SUFFIX=" (core)"
71+
elif [ "$PROPOSALS_CHANGED" = "true" ]; then
72+
SUFFIX=" (proposals)"
73+
fi
74+
75+
if [ -n "$SUFFIX" ]; then
76+
git commit -m "Update Wasm tests$SUFFIX"
77+
else
78+
echo "No changes to commit"
79+
false
80+
fi
5581
env:
5682
GIT_AUTHOR_NAME: "wpt-pr-bot"
5783
GIT_AUTHOR_EMAIL: "wpt-pr-bot@users.noreply.github.com"
@@ -61,10 +87,31 @@ jobs:
6187
if: ${{ steps.commit.outcome == 'success' }}
6288
run: |
6389
cd wpt
90+
BODY_FILE="pr-body.txt"
91+
92+
echo "Scheduled weekly update." > "$BODY_FILE"
93+
echo "" >> "$BODY_FILE"
94+
echo "Auto-generated by the '$GITHUB_WORKFLOW' workflow." >> "$BODY_FILE"
95+
96+
if [ "$CORE_CHANGED" = "true" ] || [ "$PROPOSALS_CHANGED" = "true" ]; then
97+
echo "" >> "$BODY_FILE"
98+
echo "Source information:" >> "$BODY_FILE"
99+
100+
if [ "$CORE_CHANGED" = "true" ]; then
101+
echo "Core spec: https://github.com/$UPSTREAM_REPO/commit/$UPSTREAM_COMMIT" >> "$BODY_FILE"
102+
fi
103+
104+
if [ "$PROPOSALS_CHANGED" = "true" ] && [ -f wasm/proposals-summary.txt ]; then
105+
if [ "$CORE_CHANGED" = "true" ]; then
106+
echo "" >> "$BODY_FILE"
107+
fi
108+
cat wasm/proposals-summary.txt >> "$BODY_FILE"
109+
fi
110+
fi
111+
64112
git push --set-upstream origin $BRANCH_NAME
65-
gh pr create --title "Update Wasm tests" --body "$PR_BODY"
113+
gh pr create --title "Update Wasm tests" --body-file "$BODY_FILE"
114+
115+
rm -f "$BODY_FILE"
66116
env:
67117
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
68-
PR_BODY: "Scheduled weekly update from https://github.com/${{ env.UPSTREAM_REPO }}/commit/${{ env.UPSTREAM_COMMIT }}.
69-
70-
Auto-generated by the '${{ github.workflow }}' workflow."

‎CODEOWNERS‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,9 @@
99
tools/docker/Dockerfile @web-platform-tests/wpt-core-team @web-platform-tests/admins
1010
.taskcluster.yml @web-platform-tests/wpt-core-team @web-platform-tests/admins
1111

12-
# Prevent accidentally touching wasm/core which is updated by a workflow
12+
# Prevent accidentally touching wasm/core and wasm/proposals which are updated by a workflow
1313
wasm/core/ @web-platform-tests/wpt-core-team
14+
wasm/proposals @web-platform-tests/wpt-core-team
1415

1516
# GENERATED: interop-tests
1617
css/css-anchor-position/anchor-as-multicol-crash.html @web-platform-tests/interop

‎wasm/proposals.json‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
[
2+
"stack-switching"
3+
]

‎wasm/tools/update-proposals.py‎

Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
#!/usr/bin/env python3
2+
import os
3+
import json
4+
import subprocess
5+
import shutil
6+
import sys
7+
8+
def run_cmd(cmd, cwd=None):
9+
print(f"Running: {' '.join(cmd)} in {cwd or os.getcwd()}")
10+
return subprocess.run(cmd, cwd=cwd, capture_output=True, text=True, check=True).stdout
11+
12+
13+
def main():
14+
wpt_root = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", ".."))
15+
proposals_json_path = os.path.join(wpt_root, "wasm", "proposals.json")
16+
17+
if not os.path.exists(proposals_json_path):
18+
raise FileNotFoundError(f"Proposals config file not found at {proposals_json_path}")
19+
20+
with open(proposals_json_path, "r") as f:
21+
proposals = json.load(f)
22+
23+
if not isinstance(proposals, list):
24+
raise TypeError("Proposals config must be a list of strings.")
25+
26+
if not proposals:
27+
print("Proposals list is empty.")
28+
return
29+
30+
# 1. Clear out target proposals directory entirely to prune removed proposals
31+
proposals_dir = os.path.join(wpt_root, "wasm", "proposals")
32+
if os.path.exists(proposals_dir):
33+
shutil.rmtree(proposals_dir)
34+
os.makedirs(proposals_dir, exist_ok=True)
35+
36+
tmp_dir = os.path.join(os.getcwd(), "tmp_proposals")
37+
if os.path.exists(tmp_dir):
38+
shutil.rmtree(tmp_dir)
39+
os.makedirs(tmp_dir)
40+
41+
updated_proposals = []
42+
merged_proposals = []
43+
44+
try:
45+
for name in proposals:
46+
if not name or not isinstance(name, str):
47+
print(f"Invalid proposal name: {name}")
48+
continue
49+
50+
print(f"\n--- Processing proposal: {name} ---")
51+
proposal_dir = os.path.join(tmp_dir, name)
52+
53+
# 1. Clone proposal repo (start with main branch)
54+
run_cmd(["git", "clone", "--branch", "main", f"https://github.com/WebAssembly/{name}.git", proposal_dir])
55+
56+
# 2. Add upstream remote to find fork point
57+
run_cmd(["git", "remote", "add", "upstream", "https://github.com/WebAssembly/spec.git"], cwd=proposal_dir)
58+
run_cmd(["git", "fetch", "upstream", "main"], cwd=proposal_dir)
59+
60+
# 3. Find the merge base (fork point) commit
61+
try:
62+
base_commit = run_cmd(["git", "merge-base", "upstream/main", "HEAD"], cwd=proposal_dir).strip()
63+
print(f"Fork point commit: {base_commit}")
64+
except subprocess.CalledProcessError as e:
65+
print(f"Failed to find merge-base against upstream/main: {e}")
66+
print("Falling back to diffing against upstream/main directly.")
67+
base_commit = "upstream/main"
68+
69+
# 4. Find changed or added files under test/core
70+
diff_out = run_cmd(["git", "diff", "--name-status", base_commit, "HEAD", "--", "test/core"], cwd=proposal_dir)
71+
72+
changed_wast_files = set()
73+
for line in diff_out.splitlines():
74+
if not line.strip():
75+
continue
76+
parts = line.split(maxsplit=1)
77+
if len(parts) < 2:
78+
continue
79+
status, path = parts
80+
# We care about Added, Modified, Renamed, etc. (anything except Deleted 'D')
81+
if 'D' not in status:
82+
if path.startswith("test/core/") and path.endswith(".wast"):
83+
rel_path = path[len("test/core/"):]
84+
changed_wast_files.add(rel_path)
85+
86+
if not changed_wast_files:
87+
print(f"No added or modified tests found in test/core for proposal {name}.")
88+
merged_proposals.append(name)
89+
continue
90+
91+
print(f"Found {len(changed_wast_files)} changed/added test files.")
92+
93+
# 5. Build the proposal's interpreter
94+
interpreter_dir = os.path.join(proposal_dir, "interpreter")
95+
assert os.path.exists(interpreter_dir)
96+
print("Building proposal interpreter...")
97+
run_cmd(["opam", "exec", "make"], cwd=interpreter_dir)
98+
99+
# 6. Convert WAST tests using proposal's build script
100+
out_dir = os.path.join(proposal_dir, "out")
101+
os.makedirs(out_dir, exist_ok=True)
102+
build_script = os.path.join(proposal_dir, "test", "build.py")
103+
assert os.path.exists(build_script)
104+
print("Converting WAST tests to WPT format...")
105+
run_cmd([build_script, "--dont-recompile", "--html", out_dir], cwd=proposal_dir)
106+
107+
# 7. Create target directory in WPT proposals
108+
target_dir = os.path.join(wpt_root, "wasm", "proposals", name, "core")
109+
os.makedirs(target_dir, exist_ok=True)
110+
111+
# 8. Copy only the files that were changed/added by the proposal
112+
copied_count = 0
113+
for rel_wast in changed_wast_files:
114+
print(f"Changed wast file: {rel_wast}")
115+
src_file = os.path.join(out_dir, rel_wast + ".js.html")
116+
dst_file = os.path.join(target_dir, rel_wast + ".js.tentative.html")
117+
118+
assert os.path.exists(src_file)
119+
os.makedirs(os.path.dirname(dst_file), exist_ok=True)
120+
shutil.copy2(src_file, dst_file)
121+
copied_count += 1
122+
123+
if copied_count > 0:
124+
proposal_repo = f"https://github.com/WebAssembly/{name}"
125+
proposal_commit = run_cmd(["git", "rev-parse", "HEAD"], cwd=proposal_dir).strip()
126+
updated_proposals.append((name, proposal_repo, proposal_commit))
127+
128+
print(f"Successfully copied {copied_count} test files to {target_dir}")
129+
130+
finally:
131+
# 9. Write execution summary
132+
summary_path = os.path.join(wpt_root, "wasm", "proposals-summary.txt")
133+
if updated_proposals or merged_proposals:
134+
with open(summary_path, "w") as f:
135+
if updated_proposals:
136+
f.write("The following Wasm proposals were successfully updated:\n")
137+
for name, repo, commit in updated_proposals:
138+
f.write(f"- {name} ({repo}/commit/{commit})\n")
139+
if updated_proposals and merged_proposals:
140+
f.write("\n")
141+
if merged_proposals:
142+
f.write("Note: The following proposals have 0 differences from the main spec and are likely fully merged. Please consider removing them from `wasm/proposals.json`:\n")
143+
for name in merged_proposals:
144+
f.write(f"- {name}\n")
145+
print(f"\nWritten proposals summary to {summary_path}")
146+
elif os.path.exists(summary_path):
147+
os.remove(summary_path)
148+
149+
if os.path.exists(tmp_dir):
150+
shutil.rmtree(tmp_dir)
151+
152+
if __name__ == "__main__":
153+
main()

0 commit comments

Comments
 (0)