Skip to content

Notebook test#20

Merged
dyylan merged 8 commits into
mainfrom
notebook_test
Jun 29, 2026
Merged

Notebook test#20
dyylan merged 8 commits into
mainfrom
notebook_test

Conversation

@therooler

Copy link
Copy Markdown
Collaborator

Closes #14 by converting notebooks to scripts and running them with pytest.

Proposed changes

Adds automated end-to-end execution tests for the example notebooks (docs/examples/*.ipynb) using nbconvert, so an API change that breaks a documented example is caught in CI rather than by users. The notebook tests are heavy, so they're kept out of the standard test suite and run in their own dedicated workflow — triggered on notebook changes and reused by the release pipeline to gate publishing.

Details

  • tests/test_notebooks.py auto-discovers every docs/examples/*.ipynb and re-executes each from a clean state via nbconvert's ExecutePreprocessor (900s timeout, run with the notebook's own dir as cwd). A test fails if any cell raises, surfacing the offending cell's traceback. Test IDs are the notebook filenames, so new examples are picked up automatically.

  • The module is marked pytest.mark.notebooks; pyproject.toml registers the notebooks marker and sets addopts = "-m 'not notebooks'", so a bare pytest (and the existing tests.yml job) deselects them.
    Added notebook-execution deps (nbconvert, nbformat, ipykernel, matplotlib) to the [dev] extra.
    Dedicated reusable workflow

  • .github/workflows/notebooks.yml runs pytest -m notebooks -v. Triggers: pull_request/push to main filtered to paths that affect notebooks (docs/examples/**.ipynb, tests/test_notebooks.py, pyproject.toml, the workflow itself).

  • updated .github/workflows/release.yml to call the reusable workflow via a test-notebooks job, and publish-testpypi now needs: [test-install, test-notebooks] so a broken notebook blocks the release.

  • pytest and tests.yml are unchanged in behavior (notebooks deselected).

  • pytest -m notebooks selects and runs both example notebooks; both pass locally (pulse_smoothing.ipynb ~2.5 min after the fix).

@therooler therooler requested a review from dyylan June 27, 2026 02:10
@dyylan dyylan merged commit 487bfca into main Jun 29, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Automatically test notebooks in the documentation

2 participants