Skip to content

Add pluggable post-processors for wheel optimization #309

Description

@ryanking13

To further optimize wheel sizes I would like to introduce a pluggable post-processing step for wheels.

This would allow us to run a suite of optimizers that strip unnecessary data from the package, reducing the final binary size.

Pipeline

The high-level workflow would intercept the built wheel before final output:
Build Wheel ➡️ Unpack ➡️ Apply Optimizers ➡️ Repack

Note that we are already unpacking and repacking the wheel for tagging (emscripten to pyodide ABI), the only thing we need is to inject post processing steps in it.

Interface

The interface for these optimizers could be a simple, pluggable base class. Something like this:

from pathlib import Path

class Optimizer(ABC):
    def process(self, file_path: Path, content: str | bytes) -> str | bytes:
        """
        Process the file content and return the modified data.
        """
        raise NotImplementedError

Optimizers

We could include several optimizers. Some would be safe to enable by default, while more aggressive ones could be entirely opt-in:

  • remove_docstring: Strips docstrings from .py files.
  • remove_types: Strips type hints to reduce file size.
  • compile_pyc: Compiles .py files to .pyc
  • uglify: Minifies/obfuscates the Python code.
  • remove_assertions: Strips assert statements (similar to running Python with -O).
  • remove_tests: Prunes common test directories and test files from the final wheel.
  • ...more...

Configuration
Users can toggle these optimizers via their pyproject.toml file.

[tool.pyodide.optimizer]
disable_all = false
remove_docstring = true
remove_types = true
compile_pyc = false
uglify = false
remove_assertions = true
remove_tests = true

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions