On parallel builds

[agriyakhetarpal]

The xbuildenv symlink is rewritten on each pyodide xbuildenv install call. If multiple builds for different Pyodide versions run concurrently (e.g. in cibuildwheel, when running the test suite in parallel), re-resolving through the symlink at a later point could yield the wrong versioned directory. Using self.pyodide_root, which was resolved at construction time, gives us a stable, race-free path.

Thanks for the investigation. Actually, I guess this is something that we need to think more carefully about, not just for emscripten but for other config variables as well. I personally never considered concurrent builds when designing xbuildenv installation. If cibuildwheel is using parallel builds a lot, we should be make pyodide-build more concurrent-robust.

Originally posted by @ryanking13 in #321 (comment)

[agriyakhetarpal]

One way to start out on this would be simply to run the test suite in parallel using pytest-xdist. We have been conservative and haven't really done that, but as we start to mature, why not!

The other thing should be an integration test that runs a Pyodide wheel build for a package across multiple scenarios in a concurrent manner, and two of them having different xbuildenvs and hence Emscripten versions.

What other cases could be there?