Add pyodide-build as a separate build frontend and update its build verbosity handling

[agriyakhetarpal]

See #2605

cc: @AndrewAnnex

See summary by Copilot below, AI-generated, which I edited manually to be more concise 🤖

This PR makes pyodide-build a distinguished build frontend, which is now the only allowed one for the Pyodide platform:

• The schema has been updated
• A documentation section has been added, and the build frontends' verbosity table has been updated
• pyodide-build isn't selectable on any other platform and raises configuration errors if so
• Tests updated to accommodate the new frontend

Pyodide build frontend enforcement and support:

• Added pyodide-build as a valid value for BuildFrontendName and implemented logic to enforce that the Pyodide platform only supports the pyodide-build frontend, raising configuration errors if mismatched.
• Updated the default configuration and documentation to set pyodide-build as the default and only supported frontend for Pyodide, including a new documentation section explaining this restriction.

Platform-specific frontend restrictions:

• Explicitly disallowed pyodide-build as a build frontend on all non-Pyodide platforms, raising fatal errors if attempted.

Verbosity and CLI argument handling:

• Implemented support for verbosity flags for the pyodide-build frontend, capping at -vv since pyodide build does not support higher verbosity. Updated documentation tables accordingly.

Testing and validation:

• Updated and expanded tests to verify that only pyodide-build is accepted on Pyodide, that it is rejected on other platforms, and that CLI arguments and verbosity are handled correctly.

Cleanup and removal of obsolete logic:

• Removed obsolete checks for unsupported frontends (like pip) on Pyodide, as only pyodide-build is now allowed.

[agriyakhetarpal]

I don't think this is the right approach at the moment, and I should try another way. What I think happens here is that _get_verbosity_flags will get wrapped in get_build_frontend_extra_flags, which will be called at the time of Pyodide builds. However, the finalisation of the build frontend flags happens outside and before the pyodide build command is called, which means the PYODIDE environment variable is set during the wheel builds and not when cibuildwheel performs its own preparation for the build, which means that this will fail to remove the verbosity-related flags.

[henryiii]

What's the status here?

[joerick]

I thought myself and Agriya talked about this but I can't see the convo. As I remember, the environment variable approach won't work due to when/where it's set and read. My feeling is that we should actually split out pyodide build as a separate build frontend option from build, as they actually do have different command line frontends. Once we've done that, this kind of change becomes easy too.

[agriyakhetarpal]

I recently implemented build verbosity handling in pyodide-build: pyodide/pyodide-build#363. It landed in 0.35.0, which we already have in our constraints. So this is now easy.

[freakboy3742]

The parts touching the iOS backend all make sense to me.

The only question this raises for me is the long-term goal of handling cross-platform builds. pyodide-build is obviously the pragmatic approach for pyodide right now; but longer term, there's a lot of shared needs between pyodide, iOS and Android builds, and I think there's an opportunity for us to coordinate those efforts.

I've got a very early stage prototype in the form of xbuild. My intention is to get this working for iOS at least, and submit it for use in cibuildwheel (as a replacement for the vendored shims added in #2857); in my ideal world, it would be usable for Android and Pyodide as well.

It's also been deliberately built as an extension to build so that there's a potential path to eventually upstream the changes into build itself (presumably once we've proven it works cross platform and we've got a PEP detailing what "cross platform builds" actually mean).

I mention this not as a blocker to this PR, but as an indicator of a direction that I'm interested in pursuing, where I think the needs of Pyodide, Android and iOS are aligned.

[agriyakhetarpal]

Thanks @freakboy3742. Yes, I believe this is an effort worth coordinating on our side for Pyodide-related aspects. I would be happy to help out with contributing or testing things when it's experimentable. Right now, pyodide-build just passes all the build frontend work to pypa/build, and the WASM sysconfigdata comes pre-registered as a _sysconfigdata__emscripten_wasm32-emscripten.py file within the Pyodide cross-build environments, which are separately distributed and installed onto the packager's copy when building, instead of being monkeypatched at interpreter startup based on the targeted platform.

In that sense, the xvenv is analogous to the Pyodide xbuildenvs, but with some differences, in that (i) the semantics of the cross-build environment are not exposed to the user in pyodide-build, and (ii) pyodide-build is called from a conventional system Python. The user does not need to set up a cross-build environment on their own when building a WASM wheel. I recognise that I might be biased by the work we have built to make pyodide-build more mature, but I have developed a propensity for this kind of "plug-and-play" convenience¹.

One thing that will remain difficult to port to xbuild is pyodide-build's pywasmcross.py file. That is the crux of our cross-compilation needs and allows us to compile all the packages we've been able to compile thanks to: our special handling of GCC/Clang flags that Emscripten doesn't support, pyemscripten ABI needs, and our terrible pile of hacks for SciPy to compile to WASM (though that's getting easier for me nowadays, over the years). It has turned out to be quite the success story—not in the sense that it is well-written or robust or anything—but that PEP 783 was accepted and it still exists :) That sort of thing will have to stick for longer, as the UX for cross-compiling a Pyodide/WASM wheel will be terrible without it. It could be restructured into something better and more declarative, though, for sure. IIRC, PEP 739 is the closest we are to improving cross-compilation, a precursor of sorts.

I cannot say what the approach would look like without having played around with xbuild a bit, but whatever leads us to use and maintain less code, and the ecosystem to build a shared standard, is a win :) I will try to spend a weekend sometime to create an issue on the xbuild side outlining what Emscripten support could look like.

Footnotes

1. That said, this sort of thing doesn't matter for software that hides all the murky details from the users, such as cibuildwheel in this case :D ↩