To further optimize wheel sizes I would like to introduce a pluggable post-processing step for wheels.
This would allow us to run a suite of optimizers that strip unnecessary data from the package, reducing the final binary size.
Pipeline
The high-level workflow would intercept the built wheel before final output:
Build Wheel ➡️ Unpack ➡️ Apply Optimizers ➡️ Repack
Note that we are already unpacking and repacking the wheel for tagging (emscripten to pyodide ABI), the only thing we need is to inject post processing steps in it.
Interface
The interface for these optimizers could be a simple, pluggable base class. Something like this:
from pathlib import Path
class Optimizer(ABC):
def process(self, file_path: Path, content: str | bytes) -> str | bytes:
"""
Process the file content and return the modified data.
"""
raise NotImplementedError
Optimizers
We could include several optimizers. Some would be safe to enable by default, while more aggressive ones could be entirely opt-in:
remove_docstring: Strips docstrings from .py files.
remove_types: Strips type hints to reduce file size.
compile_pyc: Compiles .py files to .pyc
uglify: Minifies/obfuscates the Python code.
remove_assertions: Strips assert statements (similar to running Python with -O).
remove_tests: Prunes common test directories and test files from the final wheel.
- ...more...
Configuration
Users can toggle these optimizers via their pyproject.toml file.
[tool.pyodide.optimizer]
disable_all = false
remove_docstring = true
remove_types = true
compile_pyc = false
uglify = false
remove_assertions = true
remove_tests = true
To further optimize wheel sizes I would like to introduce a pluggable post-processing step for wheels.
This would allow us to run a suite of optimizers that strip unnecessary data from the package, reducing the final binary size.
Pipeline
The high-level workflow would intercept the built wheel before final output:
Build Wheel➡️Unpack➡️Apply Optimizers➡️RepackNote that we are already unpacking and repacking the wheel for tagging (emscripten to pyodide ABI), the only thing we need is to inject post processing steps in it.
Interface
The interface for these optimizers could be a simple, pluggable base class. Something like this:
Optimizers
We could include several optimizers. Some would be safe to enable by default, while more aggressive ones could be entirely opt-in:
remove_docstring: Strips docstrings from.pyfiles.remove_types: Strips type hints to reduce file size.compile_pyc: Compiles.pyfiles to.pycuglify: Minifies/obfuscates the Python code.remove_assertions: Stripsassertstatements (similar to running Python with-O).remove_tests: Prunes common test directories and test files from the final wheel.Configuration
Users can toggle these optimizers via their
pyproject.tomlfile.