Surface filter-promoted unraisable warnings directly (#14263)

[paulzuradzki]

Summary

Refs #14263.

At session end, unraisableexception forces a gc.collect() so leaked objects finalize and their warnings reach sys.unraisablehook. That collection used to run from a config.add_cleanup callback. Those callbacks pop in reverse order, so another plugin's cleanup could tear down a warning filter before the collection ran. The leaked warning then fired with no filter active, so pytest dropped it with no output.

This moves the collection into pytest_unconfigure, which runs before pytest closes the cleanup stack. The warning filters and sys.unraisablehook are both still in place there, so the leak surfaces no matter what order plugins clean up in. That's approach 2 from the issue. I left the collection in cleanup() too, as a final sweep for anything freed while the cleanup stack unwinds.

A surfaced unraisable is still wrapped in PytestUnraisableExceptionWarning. To fail a run on these, use a filter that matches the wrapper: bare error, -W error, or error::pytest.PytestUnraisableExceptionWarning. A class-only filter like error::ResourceWarning surfaces the warning but doesn't fail the run on its own (see Scope).

Testing

Automated

pytest testing/test_unraisableexception.py

New/changed tests:

• test_unraisable_decouples_from_cleanup_stack_order, the regression test. A conftest.py with @hookimpl(trylast=True) registers a warnings.resetwarnings cleanup that pops before unraisableexception's own cleanup, recreating the bad order that apply warnings filter as soon as possible, and remove it as late as possible #13057's plugin reorder doesn't cover. A __del__ raises ValueError under an error::pytest.PytestUnraisableExceptionWarning filter. Red on main (filter gone before the collection runs, exit 0); green here (collection runs in pytest_unconfigure, filter still active, exit 1).
• I dropped the tests that only covered the old direct-raise branch (test_refcycle_resource_warning_filter, test_refcycle_userwarning_filter, test_unraisable_warning_filter_add_note_dedups), since that branch is gone.

Manual. Save as test_del.py:

import gc; gc.disable()  # let the cycle survive to session-end gc

class BrokenDel:
    def __init__(self):
        self.self = self  # reference cycle

    def __del__(self):
        raise ValueError("del is broken")

def test_it():
    BrokenDel()

• pytest -W error test_del.py exits 1 (wrapper promoted, leak fails the run).
• pytest test_del.py exits 0 (wrapper logged, no failure).

To see the cleanup-order fix by hand, swap src/_pytest/unraisableexception.py between main and this branch and rerun test_unraisable_decouples_from_cleanup_stack_order: main exits 0, this branch exits 1.

If you run from inside the pytest source tree, the repo's pyproject.toml sets filterwarnings = ['error', ...], which promotes every warning. Comment out the bare 'error' entry or run the snippet from outside the repo.

Reproducer from the issue

My adapted version of the original gist. The original references an undefined stderr_lines in its run_scenario helper; mine applies a one-line fix.

Scope

collect_unraisable wraps every queued unraisable in PytestUnraisableExceptionWarning and emits it through warnings.warn. An earlier revision of this PR added an unwrap branch: when an active error::<class> filter matched the inner warning, it re-raised that warning directly so error::ResourceWarning failed the test. Per reviewer feedback I dropped that branch. Re-raising lost the traceback the hook captures as a string, and the filter check (action == "error" plus issubclass) only approximated Python's real filter matching. So pytest keeps wrapping and lets the warnings machinery decide.

The issue's exact error::ResourceWarning config now surfaces the leak (logged as PytestUnraisableExceptionWarning) instead of dropping it silently, but doesn't fail the run on its own. Suites using bare error, including this repo, do fail, since error matches the wrapper.

Branch structure

The branch builds the original direct-raise approach across red→green commits, then a final commit drops it and keeps the timing fix.

• Early commits added the unwrap branch and its tests, moved the GC into pytest_unconfigure, and guarded pytest_unconfigure against an unset stash when another plugin's pytest_configure raises UsageError.
• cd7e0432e moved gc.collect() and queue processing into pytest_unconfigure.
• The final feedback commit drops the unwrap branch and _warning_class_has_error_filter, keeps the GC in cleanup() as well as pytest_unconfigure, removes the tests that only covered the dropped behavior, and reworks the cleanup-order test to raise ValueError from __del__ under an error::pytest.PytestUnraisableExceptionWarning filter.

Relationship to #14273

#14273 attempted approach 2 (move GC to pytest_unconfigure). Maintainers closed it unmerged: #13057 (Dec 2024) had already shipped approach 1 by reordering default_plugins, so the move alone showed no behavior difference under the built-in plugin order. A maintainer asked for "a regression test which demonstrates that the unraisable hook is now subject to warnings."

test_unraisable_decouples_from_cleanup_stack_order answers that. The @hookimpl(trylast=True) conftest registers a warnings.resetwarnings cleanup that pops before unraisableexception's own, defeating the static default_plugins order. On main the run exits 0 (filter gone before the collection runs); here it exits 1 (collection runs in pytest_unconfigure, filter still active).

AI disclosure

• Yes. Used AI assist with code writing and learning artifacts.
• Knowledge gaps on my side: new to repo. Spent a good amount of time researching how Python reclaims memory, sys.unraisablehook, and manually running the regression test before and after changes.
• References used
  • https://adamj.eu/tech/2025/01/02/python-unraisiable-exceptions/
  • https://adamj.eu/tech/2025/01/07/python-test-unraisable-exceptions/
  • https://www.youtube.com/watch?v=G4Lya0KVG1E
  • https://vstinner.github.io/sys-unraisablehook-python38.html
  • https://docs.python.org/3.11/library/sys.html#sys.unraisablehook

Changelog

changelog/14263.bugfix.rst.

[FuzzysTodd]

pytest_unconfigure fires before _cleanup_stack.close(), so warning
filters managed via the cleanup stack (the warnings plugin's
catch_warnings context, in particular) are guaranteed active when GC

[paulzuradzki]

self-review

[paulzuradzki]

Explainer:

• Builds a PytestUnraisableExceptionWarning wrapper and emits that via warnings.warn. The user's filter is error::ResourceWarning.
• PytestUnraisableExceptionWarning is a pytest.PytestWarning subclass; no relationship to Python ResourceWarning in class hierarchy.
• So the user's filter doesn't match the wrapper. The wrapper gets emitted as a regular warning (just a stderr line in the warnings summary), not promoted to an error, and nothing fails the test.
• For the user's error::ResourceWarning filter to fail the suite on main, they'd have to also write error::PytestUnraisableExceptionWarning. This would be filtering on pytest's internal wrapping class. Leaky abstraction. They want to express "fail if a ResourceWarning leaks from a finalizer," not "fail if pytest's internal wrapper class fires."

[paulzuradzki]

Commit 4cf46 says, "if user said error::ResourceWarning, an unraisable ResourceWarning becomes a real raised ResourceWarning"

[paulzuradzki]

When collect_unraisable sees an unraisable whose exc_value is a Warning subclass instance, re-raise it directly instead of wrapping w/ pytest.PytestUnraisableExceptionWarning

[paulzuradzki]

Append meta.exc_value (the actual ResourceWarning instance) to errors (what eventually gets raise)

[paulzuradzki]

For Python 3.11+, attach meta.cause_msg as a PEP 678 note

[paulzuradzki]

Bug made possible by -- this PR -- moving GC and collect_unraisable from cleanup callback into pytest_unconfigure hook.

Guard checks: if our configure never ran, stash key is not there, and there's nothing to drain deque, so early return.

Else, this bug was possile:

• configure step crashes before def pytest_configure(config): ... config.stash[unraisable_exceptions] = deque
• stash never created
• unconfigure runs anyway
• -> tries to read missing stash
• -> KeyError

[paulzuradzki]

I noticed similar TODOs elsewhere.

AI explainer:

The exit code 1 comes from pytest's session-end unraisable handling, not from runpytest_subprocess() (that helper just reports the child's exit code). The leaked warning fires during session-end GC in pytest_unconfigure, after test_it has already been reported as passed, so it propagates to the top of the run instead of failing a specific test. Ideally we'd get failed=1 attributed to test_it; that needs allocation tracking and is out of scope here.

[paulzuradzki]

cc: @Zac-HD - this is the PR/GH Issue we chatted about at PyCon US sprints last Monday

[Zac-HD]

Adding the gc logic to pytest-unconfigure looks good to me, but I'd make two tweaks:

• keep the gc logic in cleanup() too, in case of resources which are (partly) freed between those two locations
• drop the magic warnings handling. While tempting, we can't fully handle e.g. the tracebacks which should be present, and so I think continuing to raise PytestUnraisableExceptionWarning is the right approach here.