Skip to content

test(engine): add Hypothesis laws for combine#98

Open
leemeo3 wants to merge 1 commit into
fu351:mainfrom
leemeo3:feat/engine/combine-hypothesis-laws
Open

test(engine): add Hypothesis laws for combine#98
leemeo3 wants to merge 1 commit into
fu351:mainfrom
leemeo3:feat/engine/combine-hypothesis-laws

Conversation

@leemeo3

@leemeo3 leemeo3 commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Slice

What this PR does

  • Replaces the seeded random.Random fuzz loop in tests/unit/test_verdict_ordering.py with Hypothesis property tests.
  • Keeps the existing exhaustive Verdict x Verdict and Risk x Risk matrices.
  • Adds property coverage for raise-only floors, commutativity up to reason-code ordering, associativity, and idempotence of decision strength/reasons.
  • Adds hypothesis to the dev extra.

Fixes #96

Tests added (run in CI)

  • tests/unit/test_verdict_ordering.py — Hypothesis strategy builds valid GuardrailResult values and exercises the raise-only, commutative, associative, and idempotent laws.

Local validation:

  • .venv/bin/python -m pip install -e ".[dev]"
  • .venv/bin/python -m pytest tests/unit/test_verdict_ordering.py -v — 36 passed
  • .venv/bin/ruff check .
  • .venv/bin/ruff format --check .
  • .venv/bin/lint-imports
  • .venv/bin/python -m pytest --cov=doberman --cov-report=term-missing — 1370 passed, 2 skipped, 91% coverage

Public-release safety (doberman-core only)

  • Contains nothing from the "not allowed" list: no enterprise/hosted code, no proprietary detection, no customer data, no secrets, no commercial-license code
  • Core still builds/tests/runs with NO enterprise package installed

Security checklist

  • Fails closed on error / uncertainty — no runtime path changed; this adds tests for the raise-only combine invariant
  • No secret, full file, or unredacted prompt logged or committed
  • Any guardrail/learning change is raise-only (no silent loosening) — no guardrail/learning behavior changed; tests assert raise-only combine behavior
  • Every BLOCK/AUTH carries reason codes + a human explanation — existing model invariant remains unchanged
  • doberman-core does not import doberman_enterprise — lint-imports kept both contracts

Edge cases covered / Deviations from plan / Risks introduced

  • PASS results may carry zero reason codes; non-PASS generated results always include at least one reason and explanation.
  • Commutativity compares reason-code sets because combine() intentionally preserves operand order in the merged reason list.
  • No README update: this is test coverage plus a dev-only dependency, with no public runtime behavior change.
  • Risks introduced: adds Hypothesis to the dev extra only.

AI assistance: Prepared with OpenAI Codex.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tests: property-based (Hypothesis) laws for the raise-only combine()

1 participant