chore: prepare Python 0.8.0 release

README.md

@@ -134,21 +134,16 @@ compiler owns deterministic state transitions.
 
 ## Does it Work?
 
-Yes, on the current scored demo set, compiler-mediated paths pass every scored
-scenario.
+Yes. The current demo suite in this repository contains 8 scored demos
+(`01`-`05`, `07`, `08`, `09`) plus 1 informational demo (`06`).
 
-- Scope: evaluated across **7 models** and **3 provider paths** (`ollama`, `openai`, `openai_compatible`).
-- Scored checks (**6 demos per model**; Demo 6 excluded): baseline **26 / 42**, compiler **42 / 42**, compiler+compact **42 / 42**.
-- Baseline behavior remains model-dependent.
+The current published verification matrix combines 7 current model runs across
+hosted/frontier providers and local Ollama models. In those current runs,
+baseline passed **24 / 56**, reinjected-state passed **40 / 56**, and both
+compiler paths passed **56 / 56**.
 
-Interpretation guide:
-- Demos `01`-`05` and `07` focus on persistence and policy-following behavior.
-- Demos `08`/`09` focus on rules for when state is allowed to change.
-- Demos `08`/`09` show what prompt text does not implement by itself.
-- Plain reinjection can produce plausible responses, but it does not check whether replacement is allowed or wait for confirmation before saving changes.
-
-→ [Full results and demo output](demos/README.md)
-Canonical matrix: [docs/demos-results.md](docs/demos-results.md)
+→ [Current demo set and output modes](demos/README.md)
+Current and historical published results: [docs/demos-results.md](docs/demos-results.md)
 
 ## Interactive Playground
 
@@ -205,8 +200,8 @@ pip install context-compiler
 
 Packaging notes:
 - Base install includes the core authority-layer engine and CLI.
-- Demo/evaluation workflows live outside the core package or behind optional
-  extras.
+- Install example files with `pip install "context-compiler[examples]"`.
+- Install demo files and demo dependencies with `pip install "context-compiler[demos]"`.
 
 ### Development
 

demos/02_llm_constraint_guardrail.py

@@ -34,6 +34,10 @@
 _TITLE_HINT_RE = re.compile(r"\b(recipe|curry)\b", flags=re.IGNORECASE)
 _PROHIBITED_RE = re.compile(r"\bpeanuts?\b", flags=re.IGNORECASE)
 _STYLE_REFERENCE_RE = re.compile(r"\bpeanut(?:[- ]style|[- ]like)\b", flags=re.IGNORECASE)
+_REPLACEMENT_REFERENCE_RE = re.compile(
+    r"\b(?:instead of|rather than|in place of)\s+peanuts?\b",
+    flags=re.IGNORECASE,
+)
 _NEGATION_RE = re.compile(
     r"\b(no|without|avoid(?:s|ed|ing)?|exclud(?:e|es|ed|ing)|free of|peanut-free)\b",
     flags=re.IGNORECASE,
@@ -92,6 +96,8 @@ def recipe_includes_prohibited_item(output: str) -> bool:
             continue
         if _STYLE_REFERENCE_RE.search(line):
             continue
+        if _REPLACEMENT_REFERENCE_RE.search(line):
+            continue
         if _NEGATION_RE.search(line):
             continue
         if _is_refusal_line(line):

demos/README.md

@@ -142,7 +142,13 @@ is unavailable.
 
 ## Results
 
-The canonical cross-model results matrix is maintained in [docs/demos-results.md](../docs/demos-results.md).
+This README describes the current demo suite in this repository.
+
+The published results page in [docs/demos-results.md](../docs/demos-results.md)
+includes:
+
+- a current 2026-06 verification matrix covering frontier-provider reruns and local Ollama runs
+- the older historical 0.6.15 matrix for the earlier 6-demo scored set
 
 Notes:
 - There are **8 scored demos** (`01`–`05`, `07`, `08`, `09`). `06_context_compaction` is informational and excluded from PASS/FAIL totals.

docs/DescriptionAndMilestones.md

@@ -155,6 +155,30 @@ Historical note:
 >
 > `context-compiler-directive-drafter`
 
+### Future Consideration — Engine Thread Safety
+
+Current contract:
+
+- Engine instances are deterministic.
+- Engine instances are not currently advertised as thread-safe.
+- Hosts should serialize access to a shared engine instance.
+
+Design notes:
+
+- Simple locking around `step()` is likely insufficient.
+- `preview()` is the primary complication because it performs a temporary mutate-and-restore sequence.
+- Any future thread-safety work should evaluate:
+  - atomic preview semantics
+  - export/import consistency
+  - checkpoint operations
+  - concurrency testing
+- Thread-safety should be designed holistically rather than added through ad hoc locking.
+
+Status:
+
+- Future infrastructure hardening.
+- Not planned for the 0.8.x series.
+
 ### 0.9 Candidate Direction — Canonical Export Integrity / Hashing
 
 This is future planning only. No 0.9 implementation is defined here.

docs/demos-results.md

@@ -1,19 +1,64 @@
 # Demo Results
 
-Canonical reference for the current LLM demo matrix and methodology.
+Published LLM demo evidence for this repository.
 
-Note: this published matrix predates the `reinjected-state` path added in the 0.7.1 demo/evaluation experiment. It currently reports baseline/compiler/compiler+compact only.
+This page answers a practical release question: does Context Compiler work, and
+what evidence supports that claim?
 
-## Scope
+## Current Verification Results (2026-06)
+
+Current release-facing verification covers the current 8-demo scored set:
+
+- scored demos: `01`-`05`, `07`, `08`, `09`
+- informational demo: `06_context_compaction`
+
+### Current Results Matrix
+
+| Provider Path | Model | Context | Baseline (P/F) | Reinjected-state (P/F) | Compiler (P/F) | Compiler+Compact (P/F) |
+| :-- | :-- | :-- | :--: | :--: | :--: | :--: |
+| `openai` | `gpt-4.1` | standard | 4 / 4 | 6 / 2 | 8 / 0 | 8 / 0 |
+| `openai` | `gpt-5` | standard | 3 / 5 | 6 / 2 | 8 / 0 | 8 / 0 |
+| `openai_compatible` | `anthropic/claude-sonnet-4-6` | standard | 3 / 5 | 5 / 3 | 8 / 0 | 8 / 0 |
+| `openai_compatible` | `anthropic/claude-opus-4-1` | standard | 4 / 4 | 6 / 2 | 8 / 0 | 8 / 0 |
+| `ollama` | `llama3.1:8b` | `131072 (default)` | 2 / 6 | 6 / 2 | 8 / 0 | 8 / 0 |
+| `ollama` | `qwen2.5:7b-instruct` | `32768 (default)` | 4 / 4 | 6 / 2 | 8 / 0 | 8 / 0 |
+| `ollama` | `qwen2.5:14b-instruct` | `32768 (default)` | 4 / 4 | 5 / 3 | 8 / 0 | 8 / 0 |
+
+### Current Totals
+
+- Model runs: `7`
+- Scored demos per run: `8`
+- Aggregate scored checks per path: `56`
+
+- Baseline: `24 / 56`
+- Reinjected-state: `40 / 56`
+- Compiler: `56 / 56`
+- Compiler+compact: `56 / 56`
+
+### Interpretation
+
+- Current verification shows `56 / 56` passes on both compiler paths across the hosted-provider and local Ollama rows.
+- Baseline and reinjected-state vary by model, but the compiler-mediated paths stay perfect across all listed current runs.
+- The current Ollama rows are current 8-demo reruns recorded at each model's discovered default context size.
+- `PASS` means the demo-specific expected behavior succeeded for that path.
+- `reinjected-state` is plain application-managed state text added to the prompt; `compiler` and `compiler+compact` add compiler-managed state and checks.
+
+### Methodology
+
+- Main run command: `uv run python -m demos.run_demo all`
+- Provider selection uses `PROVIDER`, `MODEL`, and provider-specific endpoint or key configuration.
+- The current matrix combines standard hosted/frontier runs and current local Ollama runs on the same 8-demo scored set.
+- Ollama context values shown in the current matrix are discovered defaults reported by the runner, not a fixed context-size sweep.
+- `06_context_compaction` is informational and excluded from PASS/FAIL totals.
+
+## Historical Results (0.6.15)
+
+Historical LLM demo evidence for an earlier published matrix.
 
 - Scored demos: `01`, `02`, `03`, `04`, `05`, `07` (6 total)
 - Informational demo: `06_context_compaction` (excluded from PASS/FAIL totals)
 
-Methodology note (2026-05): the demo suite now also includes scored demos `08`
-and `09` for rules about when state is allowed to change. The published matrix below predates those additions and
-has not yet been fully rerun with the expanded scored set.
-
-## Results Matrix
+## Historical Results Matrix
 
 | Provider Path | Model | Baseline (P/F) | Compiler (P/F) | Compiler+Compact (P/F) |
 | :-- | :-- | :--: | :--: | :--: |
@@ -25,7 +70,7 @@ has not yet been fully rerun with the expanded scored set.
 | `openai_compatible` | `anthropic/claude-sonnet-4-5-20250929` | 4 / 2 | 6 / 0 | 6 / 0 |
 | `openai_compatible` | `anthropic/claude-opus-4-1-20250805` | 4 / 2 | 6 / 0 | 6 / 0 |
 
-## Totals (Derived from Matrix)
+## Historical Totals
 
 - Model runs: `7`
 - Scored demos per run: `6`
@@ -37,60 +82,18 @@ Aggregate pass totals:
 - Compiler: `42 / 42`
 - Compiler+compact: `42 / 42`
 
-## Methodology
-
-Primary command:
-
-```bash
-uv run python -m demos.run_demo all
-```
-
-Provider/model selection is done via environment variables:
+## Historical Methodology
 
+- Main run command: `uv run python -m demos.run_demo all`
 - `PROVIDER` (`openai`, `ollama`, `openai_compatible`)
 - `MODEL`
 - `OPENAI_API_KEY` / `OPENAI_BASE_URL` as required by provider mode
 
-Historical Anthropic rows below were run through an OpenAI-compatible path using
-provider-prefixed model IDs. For new runs, model naming follows the configured
-endpoint or gateway contract.
-
-Scoring behavior uses post-audit oracle/checker logic in demos and shared helpers:
-
-- `demos/01_llm_contradiction_clarify.py`
-- `demos/02_llm_constraint_guardrail.py`
-- `demos/03_llm_premise_guardrail.py`
-- `demos/04_llm_tool_denylist_guardrail.py`
-- `demos/05_llm_prompt_drift_vs_state.py`
-- `demos/07_llm_prompt_vs_state.py`
-- `demos/08_llm_replacement_precondition.py`
-- `demos/09_llm_pending_clarification.py`
-- shared parsing/helpers in `demos/common.py`
-
-Scored checks focus on app-side state rules rather than preferred wording in
-model outputs. The `reinjected-state` path is plain state text injection and
-does not include compiler checks like replacement preconditions or pending
-confirmation handling.
-
-### Run metadata
-
 - Date: 2026-05-06
 - Context Compiler: 0.6.15
-- Command: `uv run python -m demos.run_demo all`
 - Demo 05 turn count in this matrix: default setting (`--turns` omitted)
 
-## Interpretation
-
-- Live demo runs are **evidence/smoke tests** across real model/provider behavior.
-- Deterministic test suites (unit/property tests) are the **regression authority** for oracle and engine contracts.
-- Persistence demos and state-change-rule demos should be interpreted differently.
-- Demos `01`-`05` and `07` mostly test persistence and policy-following under transcript pressure.
-- Demos `08`/`09` test rules for when state is allowed to change.
-- Demos `08` and `09` cover cases prompt text does not implement by itself, such as checking whether replacement is allowed and waiting for confirmation before saving changes.
-- Plain prompt reinjection can produce reasonable answers, but it does not run these checks by itself.
-- Demos `08`/`09` are not general LLM quality benchmarks. Baseline and reinjected-state can produce plausible text and still `FAIL` when those app-side checks are missing.
-
-## Demo 05 Long-Transcript Stress (Exploratory Frontier Runs)
+## Historical Demo 05 Long-Transcript Stress (Exploratory Frontier Runs)
 
 Additional exploratory runs extended Demo 05 to higher transcript lengths for selected
 frontier models. These runs are separate from the full cross-model matrix above, which
@@ -104,54 +107,3 @@ records the standard scored demo configuration.
 This is exploratory evidence, not benchmark authority. Reinjection can be
 enough in some persistence scenarios, while compiler-mediated paths still
 provide explicit state-change rules.
-
-## Local Ollama Context-Size Sweep (0.7.1 Experiment)
-
-This section reports the refreshed local-only matrix with the `reinjected-state`
-path and explicit context-size ladder runs. Historical hosted-provider matrix
-rows above remain as originally recorded.
-
-### Commands Run
-
-```bash
-PROVIDER=ollama MODEL=ollama/llama3.1:8b uv run python -m demos.run_demo all --context-size 8192
-PROVIDER=ollama MODEL=ollama/llama3.1:8b uv run python -m demos.run_demo all --context-size 4096
-PROVIDER=ollama MODEL=ollama/llama3.1:8b uv run python -m demos.run_demo all --context-size 2048
-
-PROVIDER=ollama MODEL=ollama/qwen2.5:7b-instruct uv run python -m demos.run_demo all --context-size 8192
-PROVIDER=ollama MODEL=ollama/qwen2.5:7b-instruct uv run python -m demos.run_demo all --context-size 4096
-PROVIDER=ollama MODEL=ollama/qwen2.5:7b-instruct uv run python -m demos.run_demo all --context-size 2048
-
-PROVIDER=ollama MODEL=ollama/qwen2.5:14b-instruct uv run python -m demos.run_demo all --context-size 8192
-PROVIDER=ollama MODEL=ollama/qwen2.5:14b-instruct uv run python -m demos.run_demo all --context-size 4096
-PROVIDER=ollama MODEL=ollama/qwen2.5:14b-instruct uv run python -m demos.run_demo all --context-size 2048
-```
-
-### Results Matrix (Scored Demos 01-05, 07)
-
-| Provider | Model | Context size | Baseline (P/F) | Reinjected-state (P/F) | Compiler (P/F) | Compiler+compact (P/F) |
-| :-- | :-- | :--: | :--: | :--: | :--: | :--: |
-| `ollama` | `llama3.1:8b` | `8192` | 2 / 4 | 5 / 1 | 6 / 0 | 6 / 0 |
-| `ollama` | `llama3.1:8b` | `4096` | 2 / 4 | 5 / 1 | 6 / 0 | 6 / 0 |
-| `ollama` | `llama3.1:8b` | `2048` | 2 / 4 | 5 / 1 | 6 / 0 | 6 / 0 |
-| `ollama` | `qwen2.5:7b-instruct` | `8192` | 4 / 2 | 6 / 0 | 6 / 0 | 6 / 0 |
-| `ollama` | `qwen2.5:7b-instruct` | `4096` | 4 / 2 | 6 / 0 | 6 / 0 | 6 / 0 |
-| `ollama` | `qwen2.5:7b-instruct` | `2048` | 4 / 2 | 6 / 0 | 6 / 0 | 6 / 0 |
-| `ollama` | `qwen2.5:14b-instruct` | `8192` | 4 / 2 | 5 / 1 | 6 / 0 | 6 / 0 |
-| `ollama` | `qwen2.5:14b-instruct` | `4096` | 4 / 2 | 5 / 1 | 6 / 0 | 6 / 0 |
-| `ollama` | `qwen2.5:14b-instruct` | `2048` | 4 / 2 | 5 / 1 | 6 / 0 | 6 / 0 |
-
-### Concise Observations
-
-- `compiler` and `compiler+compact` were stable at `6 / 0` across all models and all context sizes.
-- `reinjected-state` stayed competitive:
-  - `6 / 0` for `qwen2.5:7b-instruct`
-  - `5 / 1` for `llama3.1:8b` and `qwen2.5:14b-instruct`
-- `baseline` varied by model but not by context size in this sweep:
-  - `2 / 4` for `llama3.1:8b`
-  - `4 / 2` for both Qwen models
-- For monitored demos:
-  - Demo `02` was the most persistent failure point for baseline, and remained a reinjected failure on `llama3.1:8b` and `qwen2.5:14b-instruct`.
-  - Demo `05` only failed baseline on `llama3.1:8b`; other paths passed.
-  - Demo `01` baseline failed on `llama3.1:8b` but passed on Qwen models.
-  - Demo `07` passed on all paths for all model/context combinations in this run set.

examples/README.md

@@ -2,7 +2,12 @@
 
 This directory contains small authority-layer examples showing typical app-side usage of Context Compiler.
 
-These files are included in the base package install (`pip install context-compiler`).
+Install examples with:
+
+```bash
+pip install "context-compiler[examples]"
+```
+
 Non-integration example files in this directory are standalone scripts and can be run directly.
 
 ## 01_persistent_guardrails.py

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "context-compiler"
-version = "0.7.10"
+version = "0.8.0"
 description = "Deterministic conversational state engine for LLM applications."
 readme = "README.md"
 requires-python = ">=3.11"
@@ -13,8 +13,11 @@ authors = [{ name = "Robert Lippmann" }]
 dependencies = []
 keywords = [
   "llm",
+  "conversational-ai",
   "conversation-state",
+  "state-management",
   "state-machine",
+  "deterministic",
   "ai-infrastructure",
 ]
 classifiers = [
@@ -38,10 +41,23 @@ context-compiler = "context_compiler.repl:main"
 packages = ["src/context_compiler"]
 
 [tool.hatch.build.targets.wheel.force-include]
+"demos" = "demos"
 "examples" = "examples"
+"host_support" = "host_support"
 "src/context_compiler/py.typed" = "context_compiler/py.typed"
 
+[tool.hatch.build.targets.sdist]
+include = [
+  "/README.md",
+  "/pyproject.toml",
+  "/src/context_compiler",
+  "/demos",
+  "/examples",
+  "/host_support",
+]
+
 [project.optional-dependencies]
+examples = []
 demos = [
   "litellm>=1.0.0",
 ]

tests/test_demo_oracle_properties.py

@@ -44,6 +44,9 @@ def test_demo02_prohibited_detection_flags_plain_peanut_mentions(
             "Ingredients: no peanuts",
             "Ingredients: free of peanuts",
             "I cannot provide peanuts due to policy.",
+            "Cashew Curry uses cashews instead of peanuts.",
+            "Creamy Cashew Curry (using cashews rather than peanuts).",
+            "Sunflower Seed Curry in place of peanuts.",
             "Sunflower Seed Peanut-Style Curry",
             "Sunflower Seed Peanut Like Curry",
             "Sunflower Seed Peanut-like Curry",