Skip to content

test: real protocol assertions and an exercised enable_runtime_checks - #209

Merged
thomaspinder merged 1 commit into
mainfrom
test/174-181-runtime-checks
Jul 29, 2026
Merged

test: real protocol assertions and an exercised enable_runtime_checks#209
thomaspinder merged 1 commit into
mainfrom
test/174-181-runtime-checks

Conversation

@thomaspinder

Copy link
Copy Markdown
Owner

Summary

Two test-infrastructure issues, one PR — and the new test immediately earned its keep by surfacing two real defects.

  • test: tautological runtime_checkable assertions in test_protocols.py #174: five always-true assertions in tests/test_protocols.py (assert hasattr(P, "__protocol_attrs__") or runtime_checkable — the second operand is a truthy function object) replaced with real ones: typing_extensions.is_protocol + exact get_protocol_members sets per protocol (py3.11-verified — stdlib __protocol_attrs__ is 3.12+), plus conforming/non-conforming stub isinstance checks that genuinely pin runtime-checkability.
  • test: enable_runtime_checks is never exercised beyond importability #181: enable_runtime_checks() was only ever asserted callable. A new subprocess test enables it in a fresh interpreter, drives a full synthetic pipeline (VARData → VAR → FittedVAR → sigma/forecast → IdentifiedVAR → IRF), and asserts a BeartypeCallHintViolation actually fires for a mistyped call. It failed immediately, exposing two pre-existing defects that made the feature unusable (exactly as the feat(diagnostics): VAR-aware convergence and stability report #176 review predicted):
    1. Beartype resolves stringified annotations (-> "ForecastResult", "xr.Dataset") against the defining module at call time, and every such name was TYPE_CHECKING-only → BeartypeCallHintForwardRefException on first use. Fixed with _bind_deferred_imports() — AST-parses each wrapped module's if TYPE_CHECKING: block and binds the imports for real (no exec), including dotted aliases.
    2. The wrap loop decorated every class in dir(mod) — including typing.Protocol and classes owned by other modules (Constant reached through spec), mutating shared state. Fixed with a __module__ == mod.__name__ ownership guard.

Test teeth verified: with the src fix stashed, the subprocess test fails with the original forward-ref error.

Closes #174
Closes #181

Gates

ruff / ty / fast suite green (532 passed, 29 deselected); the rewritten protocol file passes standalone under Python 3.11.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Egjd7ToFeb9TQqFnfRQZxV

@codecov-commenter

codecov-commenter commented Jul 29, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 4.34783% with 22 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.3%. Comparing base (fa3375c) to head (4175dc5).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
src/impulso/__init__.py 4.3% 22 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##            main    #209     +/-   ##
=======================================
- Coverage   94.1%   93.3%   -0.8%     
=======================================
  Files         41      41             
  Lines       2479    2503     +24     
  Branches     290     298      +8     
=======================================
+ Hits        2333    2336      +3     
- Misses        98     119     +21     
  Partials      48      48             

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@thomaspinder
thomaspinder enabled auto-merge (squash) July 29, 2026 07:17
Closes #174: `assert hasattr(P, "__protocol_attrs__") or runtime_checkable`
is always true (`runtime_checkable` is a truthy function object). Each of the
five instances now pins the protocol's declared member set via
`typing_extensions.get_protocol_members` — version-stable, since stdlib
`__protocol_attrs__` only exists on 3.12+ — plus an isinstance pair over a
conforming and a non-conforming stub. Those isinstance calls are what pin
runtime-checkability: isinstance against a non-`@runtime_checkable` Protocol
raises TypeError.

Closes #181: adds a subprocess test that enables runtime checks in a fresh
interpreter (the beartype wrapping mutates classes in place, so it must not
leak into the rest of the suite) and drives a synthetic no-MCMC pipeline:
VARData.from_df -> VAR -> FittedVAR.sigma/forecast ->
set_identification_strategy -> impulse_response, then asserts a
BeartypeCallHintViolation fires for `forecast(steps="two")`.

That test surfaced two real defects in `enable_runtime_checks`:

* Every wrapped method annotated with a TYPE_CHECKING-only name (e.g.
  `FittedVAR.forecast -> "ForecastResult"`) raised
  BeartypeCallHintForwardRefException on call — beartype resolves stringified
  annotations against the defining module at call time. `_bind_deferred_imports`
  now materialises each wrapped module's `if TYPE_CHECKING:` imports first.
* The wrap loop wrapped every class in `dir(mod)`, including imported ones —
  `typing.Protocol`, `typing.Any`, `impulso.volatility.Constant`, the result
  types — mutating classes the module does not own. It now wraps only classes
  defined in the module.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Egjd7ToFeb9TQqFnfRQZxV
@thomaspinder
thomaspinder force-pushed the test/174-181-runtime-checks branch from 3f91446 to 4175dc5 Compare July 29, 2026 07:21
@thomaspinder
thomaspinder merged commit 4431d5c into main Jul 29, 2026
9 checks passed
@thomaspinder thomaspinder added the internal Internal improvements (refactor, test, chore) label Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

internal Internal improvements (refactor, test, chore)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test: enable_runtime_checks is never exercised beyond importability test: tautological runtime_checkable assertions in test_protocols.py

2 participants