Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .claude/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,11 @@
"Bash(git fetch *)",
"Bash(git merge *)",
"Bash(git add *)",
"Bash(git commit *)"
"Bash(git commit *)",
"Bash(\"C:\\\\Users\\\\Eric\\\\AppData\\\\Roaming\\\\Python\\\\Python313\\\\Scripts\\\\uv.exe\" run *)",
"Bash(git --no-pager diff .claude/settings.json)",
"Bash(echo \"exit=$?\")",
"Bash(git pull *)"
]
}
}
36 changes: 36 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,35 @@ follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

## [0.6.5] — 2026-06-12

Patch release. CLI hygiene and a store-aware `doctor` in the
*evidence-preservation* consumer surface, plus anti-entropy guards and
doc-freshness tripwires across the architecture and evidence-protocol docs.
No verdict, perturbation, invariant, or spec-language change.

### Changed

- **Read-only commands stay off the model stack.** `doctor`, `verify`,
`replay`, `inspect`, `diff`, `history`, `timeline`, `matrix`, and `export`
no longer import `litellm` (and its import-time warnings). `main.py`
dispatches each command module lazily, and `execution/__init__.py` defers
`LiteLLMAdapter` via PEP 562 `__getattr__` (the public import is unchanged).
A subprocess-per-module meta-guard
([`tests/meta/test_cli_import_hygiene.py`](tests/meta/test_cli_import_hygiene.py))
keeps it that way and also forbids read-only commands from importing
`falsifyai.verdict.resolver` — mechanizing the "consumers never re-resolve"
guarantee and closing a gap where `replay` previously had no such check.
(#84, #87)

- **`doctor` is store-backend aware.** It now resolves the store scheme the
same way `build_store` does: reports the selected scheme and registered
backends, **fails (exit 3)** when no backend is registered for a
`--store-path` scheme (a missing store plugin surfaces in diagnostics
instead of crashing later at `run`), and write-probes only the built-in
SQLite store. A registered plugin store (e.g. `postgres://…`) is reported
but never constructed, preserving doctor's diagnose-only contract. (#85)

### Tests

- **Consumer-side verdict-map coverage guard.** A new meta-test
Expand Down Expand Up @@ -57,6 +86,13 @@ follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
re-resolve" guarantee and closing a gap where `replay` had no such check. No
runtime behavior changes.

- **Stale phase/MVP prose refreshed** with doc-surface drift guards that fail
if the documented surface drifts from the code. (#93, #94)

- **Agent-context correction.** `.claude/CLAUDE.md` and `AGENTS.md` no longer
claim the subpackages are unimplemented stubs — the pipeline runs end-to-end;
both now point at the CHANGELOG instead. (#86)

## [0.6.4] — 2026-06-05

Patch release. Closes a self-falsification in the *evidence-generation* layer
Expand Down
9 changes: 7 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ falsifyai diff baseline candidate
[![Python](https://img.shields.io/badge/python-3.13%2B-blue)](https://www.python.org)
[![License](https://img.shields.io/badge/license-Apache%202.0-green)](LICENSE)

**Status:** 0.6.4Perturbation-validity integrity. The `paraphrase` validity gate now rejects *lossy* rewrites — an `llm_rewrite` that drops a task's grounding while keeping its vocabulary — via a bidirectional-NLI entailment check under `--nli` ([case study 06](docs/case-studies/06-perturbation-validity-omission.md)); previously such an invalid perturbation could manufacture a false `CONSISTENTLY_WRONG`. `--nli`-less runs are byte-identical; spec language and replay format stay backward-compatible across the 0.x line.
**Status:** 0.6.5CLI hygiene and a store-aware `doctor`. Read-only commands (`doctor`, `verify`, `replay`, `inspect`, `diff`, `history`, `timeline`, `matrix`, `export`) no longer load the model stack, and `doctor` now resolves the store backend your `--store-path` selects — failing fast when no backend is registered instead of crashing later at `run`. Atop perturbation-validity integrity (the `paraphrase` gate rejects *lossy* rewrites under `--nli`, [case study 06](docs/case-studies/06-perturbation-validity-omission.md)). Spec language and replay format stay backward-compatible across the 0.x line.

```bash
pip install falsifyai
Expand Down Expand Up @@ -319,7 +319,12 @@ Consumer surfaces (`replay`, `inspect`, `diff`, `history`, `verify`, `export`) e

## Status and roadmap

**0.6.4 (current release) — Perturbation-validity integrity.** Closes a generation-layer self-falsification ([case study 06](docs/case-studies/06-perturbation-validity-omission.md)): the `paraphrase` validity gate used embedding cosine, which preserves topic but not task completeness, so a rewrite that deleted a task's grounding passed and manufactured a false `CONSISTENTLY_WRONG`:
**0.6.5 (current release) — CLI hygiene + store-aware `doctor`.** Hardens the *evidence-preservation* consumer surface without touching verdict semantics:

- ✅ **Read-only commands stay off the model stack** — `doctor`, `verify`, `replay`, `inspect`, `diff`, `history`, `timeline`, `matrix`, and `export` no longer import `litellm` (lazy command dispatch + PEP 562 deferral of `LiteLLMAdapter`); a subprocess-per-module meta-guard keeps it that way and also forbids read-only commands from re-importing the verdict resolver.
- ✅ **Store-aware `doctor`** — reports the store scheme `--store-path` selects and the registered backends, fails (exit 3) when no backend is registered, and write-probes only the built-in SQLite store (plugin stores are reported, never constructed — preserving the diagnose-only contract).

**0.6.4 — Perturbation-validity integrity.** Closes a generation-layer self-falsification ([case study 06](docs/case-studies/06-perturbation-validity-omission.md)): the `paraphrase` validity gate used embedding cosine, which preserves topic but not task completeness, so a rewrite that deleted a task's grounding passed and manufactured a false `CONSISTENTLY_WRONG`:

- ✅ **Bidirectional-NLI paraphrase validity** — a paraphrase must entail the original *and* be entailed by it; an omission breaks the reverse direction and is rejected (under `--nli`, reusing the oracle NLI backend). Generation-layer only; resolver byte-identical; `--nli`-less runs unchanged.

Expand Down
2 changes: 1 addition & 1 deletion falsifyai/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
"""FalsifyAI — falsification-first reliability testing for AI systems."""

__version__ = "0.6.4"
__version__ = "0.6.5"
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "hatchling.build"

[project]
name = "falsifyai"
version = "0.6.4"
version = "0.6.5"
description = "Falsification-first reliability testing for AI systems: perturb inputs, preserve replayable evidence, diff reliability across model changes."
readme = "README.md"
license = "Apache-2.0"
Expand Down
4 changes: 2 additions & 2 deletions tests/unit/test_version.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@
import falsifyai


def test_dunder_version_is_0_6_4() -> None:
assert falsifyai.__version__ == "0.6.4"
def test_dunder_version_is_0_6_5() -> None:
assert falsifyai.__version__ == "0.6.5"


def test_dunder_version_matches_installed_metadata() -> None:
Expand Down
2 changes: 1 addition & 1 deletion uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading