diff --git a/README.md b/README.md index a54f1eb..8476f83 100644 --- a/README.md +++ b/README.md @@ -16,17 +16,15 @@ Context Compiler gives hosts fixed state rules: - clarification instead of silent overwrite for blocked/ambiguous changes - pending confirmation flows that must resolve before anything else changes - export and import checkpoints to restore saved state and pending confirmation flow -- produce structured saved state that the host can pass to the model +- produce structured authoritative state for downstream host decisions -The model generates responses. The compiler owns state transitions. +The model generates responses. The compiler owns state. ## How the compiler metaphor works -Context Compiler treats important instructions as structured state instead of -temporary prompt text. - Like a compiler, it parses input, validates it, applies fixed rules, and -produces a stable result the host can use. It is not source-code +produces a stable result the host can use. It treats important instructions as +structured state instead of temporary prompt text. It is not source-code compilation and not a reasoning model. ## 10-Second Example @@ -98,8 +96,8 @@ Host Application └─ update → authoritative state mutated; host may call LLM with compiled state ``` -The compiler updates state and never calls the LLM. -Your app decides whether to call the model based on the returned `Decision`. +The compiler never calls the LLM. Your app decides what to do with the returned +`Decision`. --- @@ -129,8 +127,15 @@ else: render(call_llm(user_input)) ``` -This is the main integration path: your app owns the model call, and the -compiler owns deterministic state transitions. +This is the main integration path: your app owns the model call and uses the +compiler as the authority layer for state transitions. + +For runnable application-layer examples, see +[`context-compiler-example-integrations`](https://github.com/rlippmann/context-compiler-example-integrations). +That companion repository shows enforcement points built on compiler state, +including retrieval filtering, schema selection, tool gating, execution +authorization, gateway middleware, checkpoint continuation, and prompt +construction. ## Does it Work? @@ -155,8 +160,6 @@ pip install context-compiler context-compiler ``` -`context-compiler` launches the interactive REPL. - Preload options keep saved rules separate from confirmation state in progress: - `--initial-state-json` / `--initial-state-file` load saved state (via exported state JSON). @@ -179,9 +182,6 @@ for non-interactive usage. context-compiler --json < input.txt ``` -`--json` writes machine-readable NDJSON for non-interactive usage -(one complete JSON object per processed input line). - Preload options keep saved rules separate from confirmation state in progress: - `--initial-state-json` / `--initial-state-file` load saved state (via exported state JSON). @@ -234,85 +234,32 @@ For normal app code, prefer the exported decision helpers (`is_clarify`, `is_update`, `is_passthrough`, `get_clarify_prompt`, `get_decision_state`) instead of direct key traversal. ---- - -### API Reference - -| API | Description | -|---|---| -| `create_engine(state=None)` | Create a new compiler engine; optional `state` provides initial authoritative state (validated/canonicalized). | -| `step(user_input)` | Parse one user turn and return a deterministic `Decision`. | -| `compile_transcript(messages: Transcript)` | Replay a transcript from a fresh engine and return either final state or a confirmation prompt. | -| `engine.apply_transcript(messages: Transcript)` | Replay a transcript onto the current engine state and return either final state or a confirmation prompt. | -| `engine.state` | Read the current opaque authoritative in-memory state snapshot; for normal host reads, prefer `get_premise_value(state)` and `get_policy_items(state, ...)`. | -| `engine.has_pending_clarification()` | Return whether a confirmation-required clarification is currently pending. | -| `get_premise_value(state)` | Read the current premise value from a state snapshot. | -| `get_policy_items(state, value=None)` | Read policy items from a state snapshot (all, `use`, or `prohibit`). | -| `engine.export_json()` | Export authoritative state as JSON (`str`) for state transport/persistence. | -| `engine.import_json(payload)` | Load/restore authoritative state from exported JSON (`str`). | -| `engine.export_checkpoint()` | Export resumable checkpoint object (`Checkpoint`). | -| `engine.import_checkpoint(payload)` | Restore full checkpoint (`Checkpoint`) and return `None`. | -| `engine.export_checkpoint_json()` | Export checkpoint as canonical JSON (`str`). | -| `engine.import_checkpoint_json(payload)` | Restore checkpoint from JSON (`str`) and return `None`. | - -### Controller API (Reusable Outside REPL) - -These controller APIs are public package exports. You can use them directly -in app code (not just inside the REPL). - -Controller quick example: - -```python -from context_compiler import ( - diff_has_changes, - get_step_decision, - get_step_state, - is_update, - get_preview_state_after, - create_engine, - preview, - preview_would_mutate, - state_diff, - step, -) - -engine = create_engine() +See [docs/api-reference.md](docs/api-reference.md) for the full public API +reference. -before = engine.state -dry_run = preview(engine, "prohibit peanuts") -print(preview_would_mutate(dry_run)) # True -planned_change = state_diff(before, get_preview_state_after(dry_run)) -print(diff_has_changes(planned_change)) # True - -after_preview = engine.state -print(diff_has_changes(state_diff(before, after_preview))) # False (preview does not mutate state) - -applied = step(engine, "prohibit peanuts") -print(is_update(get_step_decision(applied))) # True -print(get_step_state(applied) is not None) # True -``` +Common API entry points: -| API | Description | -|---|---| -| `step(engine, user_input)` | Run one turn through the engine and return `StepResult` (`output_version`, `mode`, `decision`, `state`). | -| `preview(engine, user_input)` | Run deterministic dry-run preview and return `PreviewResult` (`output_version`, `mode`, `decision`, `state_before`, `state_after`, `diff`, `would_mutate`). Live engine state is restored after preview. | -| `state_diff(state_before, state_after)` | Return a structural `StructuralDiff` (`changed`, premise before/after, policies added/removed/changed). | +- engine lifecycle: `create_engine(...)`, `engine.step(...)`, `engine.state`, + `engine.has_pending_clarification()` +- decision helpers: `is_clarify(...)`, `is_update(...)`, `is_passthrough(...)`, + `get_clarify_prompt(...)`, `get_decision_state(...)` +- state helpers: `get_premise_value(...)`, `get_policy_items(...)` +- transcript APIs: `compile_transcript(...)`, `engine.apply_transcript(...)` +- state and checkpoint transport: `export_json(...)`, `import_json(...)`, + `export_checkpoint(...)`, `import_checkpoint(...)` +- controller APIs: `preview(...)`, `step(...)`, `state_diff(...)` -The package also exports decision-kind constants for clearer host branching: -- `DECISION_PASSTHROUGH` -- `DECISION_UPDATE` -- `DECISION_CLARIFY` +### Controller API (Reusable Outside REPL) -The package also exports decision helpers for common host-side checks: -- `is_update(decision)` -- `is_clarify(decision)` -- `is_passthrough(decision)` -- `get_clarify_prompt(decision)` -- `get_decision_state(decision)` +- `preview(engine, user_input)` performs a deterministic dry run and restores + live engine state afterward +- `step(engine, user_input)` returns a reusable result envelope around one + engine turn +- `state_diff(state_before, state_after)` summarizes structural state changes -The package exports policy value constants for explicit policy comparisons: -- `POLICY_USE` -- `POLICY_PROHIBIT` +For examples and helper accessors such as `get_step_decision(...)`, +`get_preview_state_after(...)`, `preview_would_mutate(...)`, and +`diff_has_changes(...)`, see [docs/api-reference.md](docs/api-reference.md). --- @@ -325,8 +272,6 @@ authoritative in future turns. - `use` = affirmative selection or preference - `prohibit` = explicit exclusion -The compiler keeps this state in a form that your app can trust. - - Premise is a single value that can be set or replaced - Policies are per-item (`use` or `prohibit`) - State changes only through explicit directives @@ -389,50 +334,12 @@ still belongs to the host and model workflow. - authoritative state - pending confirmation flow state -Checkpoint object shape: - -```json -{ - "checkpoint_version": 1, - "authoritative_state": { - "premise": "concise replies", - "policies": { - "docker": "use" - }, - "version": 2 - }, - "pending": { - "kind": "replacement", - "replacement": { - "kind": "use_only", - "new_item": "kubectl", - "old_item": null - }, - "prompt_to_user": "..." - } -} -``` - -The checkpoint shape above is an explicit serialization contract. At this -boundary, direct key access is expected. - -Notes: - -- `pending` is `null` when no continuation is waiting for confirmation. -- `pending` captures confirmation-required operations (for example replacement flows). -- `old_item` may be `null` for `"use_only"` when confirming “use X instead?” without an existing exact policy to replace. -- imported policy keys are normalized during `import_json` / checkpoint authoritative-state restore. -- if a policy key normalizes to `""`, the payload is invalid and is rejected. -- this keeps import-time state integrity aligned with directive-time behavior, where empty policy items are not allowed. -- checkpoint restore is full and deterministic: it restores authoritative state and pending continuation together. -- checkpoint validation is all-or-nothing; invalid payloads raise and no partial restore occurs. -- `checkpoint_version` is independent of authoritative state `version` and must be bumped when checkpoint contract shape changes (especially `pending`). - -When to use checkpoint APIs: +Use state JSON when you only need authoritative state. Use checkpoint APIs when +you also need resumable continuation state across process or request +boundaries. -- stateless host or integration boundaries where engine instances are short-lived. -- resume after interruption without losing pending clarification flow. -- preserve pending confirmation flow state (`pending`) across process/request boundaries. +For the checkpoint object shape, API-level usage notes, and serialization +details, see [docs/api-reference.md](docs/api-reference.md#checkpoint-apis). --- @@ -476,33 +383,35 @@ For full directive grammar and edge-case behavior, see [DirectiveGrammarSpec.md] - [examples](examples/) — minimal usage patterns for the core authority layer - [demos](demos/) — concrete scenarios showing how behavior differs with and without the compiler +- [`context-compiler-example-integrations`](https://github.com/rlippmann/context-compiler-example-integrations) — runnable application-layer enforcement examples built around compiler state --- ## FAQ **Isn't this just prompt reinjection?** -No. Prompt reinjection is one way a host can use Context Compiler's -authoritative state, but Context Compiler is not a prompt-reinjection system. -It decides when state changes are allowed, when clarification is required, and -how state plus pending confirmation flow are restored. For runnable host -examples, see `context-compiler-example-integrations`. +No. Prompt construction is one downstream use of authoritative state. +Context Compiler is the authority layer that decides when state changes are +allowed, when clarification is required, and how continuation state is +restored. For runnable application-layer examples, see +[`context-compiler-example-integrations`](https://github.com/rlippmann/context-compiler-example-integrations). **Why not just use a plain dict?** -A plain dict is enough to drive prompt construction, schema selection, and -other host behavior. +A plain dict can hold state for prompt construction, schema selection, tool +gating, and other host behavior. -Context Compiler solves a different problem: who updates that state, under which -rules, and what happens when instructions conflict. +Context Compiler solves the authority problem: who updates that state, under +which rules, and what happens when instructions conflict. ```text User: use python_script User: prohibit python_script ``` -With a plain dict, the application must invent rules to resolve conflicts. -Context Compiler applies deterministic state-transition rules and can return -clarification instead of silently overwriting state. +Without an authority layer, the application must invent conflict-resolution and +continuation rules itself. Context Compiler applies deterministic +state-transition rules and can return clarification instead of silently +overwriting state. --- diff --git a/demos/README.md b/demos/README.md index cab1b2c..52b7d91 100644 --- a/demos/README.md +++ b/demos/README.md @@ -10,25 +10,32 @@ This demo set shows what users notice: saved authoritative state continues to affect later turns, and where your app needs deterministic state-transition rules. +These demos are proof-of-concept and evaluation surfaces. They isolate +authority semantics with dependency-light comparisons instead of requiring +deeper framework integration layers. + Scored demos now compare four paths: - baseline -- reinjected-state (application-managed state text injected into the prompt, without compiler semantics) +- reinjected-state (application-managed state text injected into the prompt, used here as a simple prompt-only comparison baseline without compiler semantics) - compiler-mediated (full transcript + saved compiler state added to the prompt) - compiler+compact (compacted transcript + saved compiler state added to the prompt) +Runnable application-layer enforcement-point integrations live in +[`context-compiler-example-integrations`](https://github.com/rlippmann/context-compiler-example-integrations). + ## Demo overview | Demo | Behavior | Concept | Most visible with | | :--: | --- | :--: | --- | -| [03](./03_llm_premise_guardrail.py) | Premise updates stay authoritative | fixed, repeatable premise updates | models that summarize conversation | | [01](./01_llm_contradiction_clarify.py) | Contradiction blocking | clarification gate | small instruct models | -| [08](./08_llm_replacement_precondition.py) | Replacement precondition | invalid replacement blocked without state mutation | any model | -| [09](./09_llm_pending_clarification.py) | Pending clarification continuation | confirmation-only resolution of suspended mutation | any model | -| [06](./06_llm_context_compaction.py) | Context compaction | saved compiler state replacing transcript context | small or local models | -| [07](./07_llm_prompt_vs_state.py) | Prompt engineering comparison | prompting vs saved compiler state | any model with long transcript sensitivity | | [02](./02_llm_constraint_guardrail.py) | Policy state stays active across turns | authoritative policy state | small or quantized models | +| [03](./03_llm_premise_guardrail.py) | Premise updates stay authoritative | fixed, repeatable premise updates | models that summarize conversation | | [04](./04_llm_tool_denylist_guardrail.py) | Tool governance | application-layer tool gating from saved state | general assistant models | | [05](./05_llm_prompt_drift_vs_state.py) | Prompt drift | long transcript failure | weaker long-context models ([see Demo 5 note](#demo-5-stress-ladder-turns)) | +| [06](./06_llm_context_compaction.py) | Context compaction | saved compiler state replacing transcript context | small or local models | +| [07](./07_llm_prompt_vs_state.py) | Prompt engineering comparison | prompting vs saved compiler state | any model with long transcript sensitivity | +| [08](./08_llm_replacement_precondition.py) | Replacement precondition | invalid replacement blocked without state mutation | any model | +| [09](./09_llm_pending_clarification.py) | Pending clarification continuation | confirmation-only resolution of suspended mutation | any model | Stronger frontier models may show these behaviors less often, but the same patterns still appear in real applications. @@ -158,13 +165,13 @@ Notes: - There are **8 scored demos** (`01`–`05`, `07`, `08`, `09`). `06_context_compaction` is informational and excluded from PASS/FAIL totals. - Anthropic runs in this repo are executed through the `openai_compatible` provider path. - `PASS` means the demo-specific expected-behavior check for that path succeeded; `FAIL` means it did not. -- `reinjected-state` can be enough for some persistence cases; this comparison shows where app-side state rules add value. -- Scored checks focus on app-side state rules (for example blocked mutation and confirmation-only resolution), not model prose quality. `reinjected-state` remains plain text injection only. +- `reinjected-state` can be enough for some persistence cases; in this demo set it is intentionally used as a prompt-only comparison baseline. +- Scored checks focus on app-side authority rules (for example blocked mutation and confirmation-only resolution), not model prose quality. `reinjected-state` remains plain text injection only. - Interpretation: - Demos `01`-`05` and `07` mostly test persistence and policy-following behavior across turns. - Demos `08`/`09` test rules for when state is allowed to change. -- Demos `08` and `09` cover cases prompt text does not implement by itself, such as checking whether replacement is allowed and waiting for confirmation before saving changes. -- Plain prompt reinjection can produce reasonable answers, but it does not run these checks by itself. +- Demos `08` and `09` cover authority semantics prompt text does not implement by itself, such as replacement preconditions, blocked mutations, and waiting for confirmation before saving changes. +- Plain prompt reinjection can produce reasonable answers, but it does not run these authority checks by itself and is not the only or preferred production integration pattern. - Similar outcomes across models in `08`/`09` reflect app behavior limits, not model leaderboard ranking. ### Demo 05 example (prompt drift under longer context) diff --git a/docs/README.md b/docs/README.md index 0019fc0..f7e5e50 100644 --- a/docs/README.md +++ b/docs/README.md @@ -6,6 +6,7 @@ ## Core Concepts - [Architecture boundaries](architecture.md) - [Directive Grammar (exact command forms the engine accepts)](DirectiveGrammarSpec.md) +- [Public API reference](api-reference.md) ## Evaluation & Evidence - [Demo results matrix](demos-results.md) diff --git a/docs/api-reference.md b/docs/api-reference.md new file mode 100644 index 0000000..56b8163 --- /dev/null +++ b/docs/api-reference.md @@ -0,0 +1,341 @@ +# API Reference + +Public API reference for `context_compiler`. + +This page documents the exported package surface and typical usage patterns. It +does not redefine behavioral semantics. + +Authoritative behavior documents: + +- [Directive Grammar Specification](DirectiveGrammarSpec.md) +- [Architecture boundaries](architecture.md) +- [Project README](../README.md) + +For behavioral semantics, use the authoritative documents above. This page +documents the public checkpoint APIs and their contract surface without +redefining directive or continuation behavior. + +## Engine Lifecycle + +### `create_engine(state=None)` + +Create a new engine instance. + +- `state=None`: start from empty authoritative state +- `state=`: initialize from a validated authoritative state snapshot + +Typical use: + +```python +from context_compiler import create_engine + +engine = create_engine() +``` + +### `engine.step(user_input)` + +Parse one user turn and return a deterministic `Decision`. + +Typical use: + +```python +decision = engine.step("set premise current project uses uv") +``` + +Behavior for directive handling, clarification, and confirmation flows is +defined by the [Directive Grammar Specification](DirectiveGrammarSpec.md). + +### `engine.state` + +Read the current authoritative in-memory state snapshot. + +The internal structure is intentionally opaque for normal host use. Prefer +`get_premise_value(state)` and `get_policy_items(state, ...)` over direct key +traversal unless you are working at an explicit serialization boundary. + +### `engine.has_pending_clarification()` + +Return whether a confirmation-required clarification is currently pending. + +Typical use: + +```python +if engine.has_pending_clarification(): + show_pending_ui() +``` + +## Decision API + +Each user message produces a `Decision`. + +```python +class Decision(TypedDict): + kind: Literal["passthrough", "update", "clarify"] + state: dict | None + prompt_to_user: str | None +``` + +Decision kinds: + +| kind | Intended host use | +|---|---| +| `passthrough` | forward the user input to the model/runtime | +| `update` | authoritative state changed; host may apply downstream behavior using updated state | +| `clarify` | show `prompt_to_user`; do not continue normal downstream processing yet | + +Helper functions: + +- `is_passthrough(decision)` +- `is_update(decision)` +- `is_clarify(decision)` +- `get_clarify_prompt(decision)` +- `get_decision_state(decision)` + +Typical use: + +```python +from context_compiler import get_clarify_prompt, is_clarify, is_update + +decision = engine.step(user_input) + +if is_clarify(decision): + show_to_user(get_clarify_prompt(decision)) +elif is_update(decision): + apply_runtime_rules() +``` + +## State Access + +Use the exported helpers for normal reads from a `State` snapshot. + +### Premise helpers + +- `get_premise_value(state)` returns the current premise value or `None` + +### Policy helpers + +- `get_policy_items(state)` returns all policy items +- `get_policy_items(state, "use")` returns `use` items +- `get_policy_items(state, "prohibit")` returns `prohibit` items + +Typical use: + +```python +from context_compiler import POLICY_PROHIBIT, get_policy_items + +blocked_tools = get_policy_items(state, POLICY_PROHIBIT) +``` + +See the README’s [State Model](../README.md#state-model) section for conceptual +guidance on premise vs policy usage. + +## Transcript APIs + +### `compile_transcript(messages: Transcript)` + +Replay a transcript from a fresh engine. + +Use this when you want transcript-derived state or clarification outcome +without starting from an existing in-memory engine instance. + +### `engine.apply_transcript(messages: Transcript)` + +Replay a transcript onto the current engine state. + +Use this when you want transcript replay relative to an existing authoritative +state snapshot. + +Transcript replay behavior is governed by the engine semantics and transcript +fixture contracts, not by this page. + +## State Import/Export + +### `engine.export_json()` + +Export authoritative state as canonical JSON text. + +### `engine.import_json(payload)` + +Validate and restore authoritative state from exported JSON text. + +Use these APIs for authoritative-state transport or persistence only. + +Conceptual boundary: + +- `export_json()` / `import_json()` transport authoritative state only +- checkpoint APIs transport authoritative state plus resumable continuation state + +## Checkpoint APIs + +### `engine.export_checkpoint()` + +Export a resumable checkpoint object. + +### `engine.import_checkpoint(payload)` + +Validate and restore a checkpoint object. + +### `engine.export_checkpoint_json()` + +Export a checkpoint as canonical JSON text. + +### `engine.import_checkpoint_json(payload)` + +Validate and restore a checkpoint from JSON text. + +Use checkpoint APIs when you need both: + +- authoritative state +- pending confirmation/continuation state + +Checkpoint object shape: + +```json +{ + "checkpoint_version": 1, + "authoritative_state": { + "premise": "concise replies", + "policies": { + "docker": "use" + }, + "version": 2 + }, + "pending": { + "kind": "replacement", + "replacement": { + "kind": "use_only", + "new_item": "kubectl", + "old_item": null + }, + "prompt_to_user": "..." + } +} +``` + +At this boundary, direct key access is expected. + +API-level contract notes: + +- `pending` is `null` when no continuation is waiting for confirmation +- `pending` captures confirmation-required operations such as replacement flows +- `old_item` may be `null` for `"use_only"` when confirming “use X instead?” + without an existing exact policy to replace +- imported policy keys are normalized during `import_json(...)` and checkpoint + authoritative-state restore +- if a policy key normalizes to `""`, the payload is invalid and is rejected +- checkpoint restore is full and deterministic: authoritative state and pending + continuation are restored together +- checkpoint validation is all-or-nothing; invalid payloads raise and no + partial restore occurs +- `checkpoint_version` is independent of authoritative state `version` and must + be bumped when checkpoint contract shape changes, especially `pending` + +Typical use cases: + +- stateless host or integration boundaries where engine instances are short-lived +- resume after interruption without losing pending clarification flow +- preserve pending confirmation flow state across process or request boundaries + +## Controller APIs + +These controller APIs are public package exports and can be used directly in +host code, not only through the REPL. + +### `step(engine, user_input)` + +Run one turn through an engine and return a `StepResult`. + +`StepResult` contains: + +- `output_version` +- `mode` +- `decision` +- `state` + +### `preview(engine, user_input)` + +Run a deterministic dry-run preview and return a `PreviewResult`. + +`PreviewResult` contains: + +- `output_version` +- `mode` +- `decision` +- `state_before` +- `state_after` +- `diff` +- `would_mutate` + +`preview(...)` restores live engine state after the dry run. + +### `state_diff(state_before, state_after)` + +Return a `StructuralDiff` describing premise and policy changes between two +state snapshots. + +Typical use: + +```python +from context_compiler import ( + create_engine, + diff_has_changes, + get_preview_state_after, + preview, + state_diff, +) + +engine = create_engine() +before = engine.state +dry_run = preview(engine, "prohibit peanuts") +diff = state_diff(before, get_preview_state_after(dry_run)) + +if diff_has_changes(diff): + show_preview(diff) +``` + +Controller helper functions: + +- `get_step_decision(step_result)` +- `get_step_state(step_result)` +- `get_preview_decision(preview_result)` +- `get_preview_state_after(preview_result)` +- `preview_would_mutate(preview_result)` +- `diff_has_changes(diff)` + +For controller result-envelope details, see the controller conformance fixture +documentation in [tests/fixtures/README.md](../tests/fixtures/README.md). + +## Public Constants + +Decision-kind constants: + +- `DECISION_PASSTHROUGH` +- `DECISION_UPDATE` +- `DECISION_CLARIFY` + +Policy-value constants: + +- `POLICY_USE` +- `POLICY_PROHIBIT` + +Use these when you want explicit string comparisons without hard-coding +literals in host code. + +## Result Object Summaries + +Public result and data object names exported at package root include: + +- `Decision` +- `State` +- `Checkpoint` +- `Transcript` +- `TranscriptMessage` +- `StepResult` +- `PreviewResult` +- `StructuralDiff` +- `ApplyResult` +- `Engine` + +These names are part of the public package surface. For the exact portable API +export contract used by tests and ports, see +[tests/fixtures/conformance/api/public-api-v1.json](../tests/fixtures/conformance/api/public-api-v1.json). diff --git a/docs/demos-results.md b/docs/demos-results.md index a1835e7..d08ea4c 100644 --- a/docs/demos-results.md +++ b/docs/demos-results.md @@ -5,6 +5,9 @@ Published LLM demo evidence for this repository. This page answers a practical release question: does Context Compiler work, and what evidence supports that claim? +For runnable application-layer enforcement-point integrations, see +[`context-compiler-example-integrations`](https://github.com/rlippmann/context-compiler-example-integrations). + ## Current Verification Results (2026-06) Current release-facing verification covers the current 8-demo scored set: @@ -41,7 +44,9 @@ Current release-facing verification covers the current 8-demo scored set: - Baseline and reinjected-state vary by model, but the compiler-mediated paths stay perfect across all listed current runs. - The current Ollama rows are current 8-demo reruns recorded at each model's discovered default context size. - `PASS` means the demo-specific expected behavior succeeded for that path. -- `reinjected-state` is plain application-managed state text added to the prompt; `compiler` and `compiler+compact` add compiler-managed state and checks. +- `baseline` reflects model behavior without added saved-state authority. +- `reinjected-state` is a prompt-only baseline: plain application-managed state text added to the prompt, without authority semantics. +- `compiler` and `compiler+compact` reflect compiler-mediated authority behavior rather than prompt-only persistence. ### Methodology @@ -106,4 +111,5 @@ records the standard scored demo configuration. This is exploratory evidence, not benchmark authority. Reinjection can be enough in some persistence scenarios, while compiler-mediated paths still -provide explicit state-change rules. +provide authority semantics such as replacement preconditions, blocked +mutations, pending confirmation handling, and checkpoint continuation. diff --git a/examples/README.md b/examples/README.md index 816666e..82dd8a8 100644 --- a/examples/README.md +++ b/examples/README.md @@ -1,6 +1,18 @@ # Examples -This directory contains small authority-layer examples showing typical app-side usage of Context Compiler. +This directory contains small authority-layer examples showing core engine usage +patterns in Context Compiler. + +These examples are intended to teach: + +- directive grammar +- `Decision` handling +- engine lifecycle +- state access +- checkpoints +- transcript replay +- controller APIs +- authority-layer usage patterns Install the core package with: @@ -10,10 +22,17 @@ pip install context-compiler Example files in this directory are included in the repository and source distribution, and can be run directly. +They are not intended to be the primary source of framework, provider, or +production runtime integration guidance. + +For runnable application-layer enforcement-point and host integration examples, +see +[`context-compiler-example-integrations`](https://github.com/rlippmann/context-compiler-example-integrations). + ## 01_persistent_guardrails.py Shows how explicit policy state stays authoritative across later turns. -Shows the app sending saved state so later answers are interpreted in that context. +Shows core authority-layer state being used in later turns. ## 02_configuration_and_correction.py @@ -25,21 +44,21 @@ Shows `set premise ...` followed by `change premise to ...`. Shows `clarify` behavior before state changes. Shows how the app handles `clarify` and skips the LLM call. -## 08_controller_preview_diff.py +## 04_tool_governance_denylist.py -Shows controller-layer dry-run behavior with `preview(engine, user_input)`. -Shows structural state inspection with `state_diff(state_before, state_after)`. -Shows `step(engine, user_input)` after preview to apply the same input. +Shows an application-layer use of authoritative policy state for tool selection. +Shows how apps can prevent denied tools from being selected without changing compiler identity. ## 05_llm_integration_pattern.py Shows end-to-end app control flow around compiler outcomes. -Shows what to do on `clarify`, when to call the model, and how to include saved state in prompts. +Shows what to do on `clarify`, when to call the model, and how host code can +use saved state downstream. Includes a single-item policy removal step via `remove policy `. ## 06_transcript_replay.py -Shows transcript replay helpers for app integration. +Shows transcript replay helpers for authority-layer integration. Shows `compile_transcript(messages)` from a fresh engine and `engine.apply_transcript(messages)` on current engine state. ## 07_single_policy_correction.py @@ -47,7 +66,8 @@ Shows `compile_transcript(messages)` from a fresh engine and `engine.apply_trans Demonstrates explicit single-policy correction without `reset policies`. Shows `prohibit peanuts` -> `remove policy peanuts` -> `use peanuts`. -## 04_tool_governance_denylist.py +## 08_controller_preview_diff.py -Shows an application-layer use of authoritative policy state for tool selection. -Shows how apps can prevent denied tools from being selected without changing compiler identity. +Shows controller-layer dry-run behavior with `preview(engine, user_input)`. +Shows structural state inspection with `state_diff(state_before, state_after)`. +Shows `step(engine, user_input)` after preview to apply the same input. diff --git a/pyproject.toml b/pyproject.toml index d15756b..dde2da7 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "hatchling.build" [project] name = "context-compiler" -version = "0.8.1" +version = "0.8.2" description = "Deterministic conversational state engine for LLM applications." readme = "README.md" requires-python = ">=3.11" diff --git a/uv.lock b/uv.lock index 7044a0d..1a3844b 100644 --- a/uv.lock +++ b/uv.lock @@ -296,7 +296,7 @@ wheels = [ [[package]] name = "context-compiler" -version = "0.8.1" +version = "0.8.2" source = { editable = "." } [package.optional-dependencies]