From f1a3bf6a662d212184f1e1099c582eaaf946142c Mon Sep 17 00:00:00 2001 From: Robert Lippmann Date: Tue, 26 May 2026 01:09:17 -0400 Subject: [PATCH 1/6] docs: reframe 0.7.1 docs around host-side state boundaries --- README.md | 81 ++++++++++++++++++----------- demos/README.md | 15 ++++-- docs/demos-results.md | 8 +-- docs/llm-preprocessor.md | 44 +++++++++++++--- experimental/preprocessor/README.md | 9 ++++ 5 files changed, 112 insertions(+), 45 deletions(-) diff --git a/README.md b/README.md index 6c8c113..9cdfc08 100644 --- a/README.md +++ b/README.md @@ -1,27 +1,56 @@ - # Context Compiler [![PyPI version](https://img.shields.io/pypi/v/context-compiler)](https://pypi.org/project/context-compiler/) [![Python versions](https://img.shields.io/pypi/pyversions/context-compiler)](https://pypi.org/project/context-compiler/) [![License](https://img.shields.io/pypi/l/context-compiler)](https://pypi.org/project/context-compiler/) -Context Compiler lets users set rules and corrections that actually stick. -It helps applications keep explicit user instructions consistent across turns. -It stores premise and policy rules outside the model, so corrections do not drift or conflict over time. +Some behaviors require explicit host-side state machinery. + +Context Compiler is a deterministic host-side state layer for LLM applications. +It handles explicit state transitions for premise and policies so that mutation +rules are fixed, inspectable, and repeatable. + +## What prompting and reinjection can do + +Prompting and reinjection are useful. In many real systems, reinjecting saved +state text is enough to keep instructions and policies persistent across turns. + +Context Compiler adds host-owned transition rules for behaviors that plain text +reinjection does not implement by itself: replacement preconditions, blocked +mutation with clarification, pending confirmation state, and checkpointed +continuation. + +## What prompting cannot do by itself + +Prompt text (including reinjected state text) does not, by itself, define a +host-authoritative mutation boundary. By itself, it does not provide: + +- authoritative state transition rules +- replacement precondition checks (`use X instead of Y` when `Y` may be absent) +- pending clarification continuation state +- deterministic decisions about when mutation is blocked +- reliable persistence/checkpoint boundaries for restoring both state and pending flow -LLMs are good at conversation, but bad at consistently following long-term rules and corrections. Constraints drift, corrections conflict, and long chats can become inconsistent. +## What Context Compiler provides -The model writes responses. The compiler stores premise and policy rules. +Context Compiler provides fixed host-side state machinery: -Context Compiler is a deterministic control layer for LLM applications. It processes explicit user instructions before model calls so applications can reliably enforce premise and policy constraints. +- deterministic directive handling for explicit user state mutations +- clarification instead of silent overwrite for blocked/ambiguous changes +- pending confirmation flows that must resolve before continuation +- checkpoint export/import for restoring authoritative state and pending continuation +- structured authoritative state that the host can pass to the model -Context Compiler handles the cases that become operationally complicated quickly: +The model generates responses. The compiler owns state transitions. -- replacing a policy that may or may not exist yet -- detecting contradictory directives before applying them -- enforcing valid state transitions (you can't change what isn't set) -- persisting authoritative state across stateless API boundaries -- resuming after an interrupted clarification flow +## How the compiler metaphor works + +Context Compiler treats important instructions as structured state instead of +temporary prompt text. + +Like a compiler, it parses input, validates it, applies fixed rules, and +produces a stable representation the host can use. It is not source-code +compilation and not a reasoning model. ## Does it work? @@ -32,12 +61,12 @@ Yes, on the current scored demo set. - Across tested models, compiler-mediated paths pass all scored scenarios; baseline behavior is model-dependent. Interpretation guide: -- Persistence/policy-following demos test whether instructions keep applying across turns. -- State-transition demos (`08`/`09`) test whether the host enforces state changes in a fixed, repeatable way. -- Demos `08`/`09` are capability checks, not general model-quality rankings. -- In those demos, baseline or reinjected-state can sound reasonable and still `FAIL` because required host-side transition checks are not present. +- Demos `01`-`05` and `07` focus on persistence and policy-following behavior. +- Demos `08`/`09` focus on fixed host-side state-transition rules. +- Demos `08`/`09` are capability-boundary checks, not prompt-following leaderboard comparisons. +- Plain reinjection can produce plausible responses, but it does not implement replacement precondition checks or pending confirmation flows by itself. -→ [Full results and demo output](demos/README.md) +→ [Full results and demo output](demos/README.md) Canonical matrix: [docs/demos-results.md](docs/demos-results.md) ## Quickstart @@ -52,7 +81,7 @@ context-compiler --json < input.txt `context-compiler` launches the interactive REPL. `--with-preprocessor` enables the experimental preprocessor before each REPL turn -(heuristic + validation only). Near-miss inputs are not rewritten and are +(simple rule-based handling plus conservative validation). Near-miss inputs are not rewritten and are passed through to the engine, which continues to return clarify behavior for those forms. @@ -117,16 +146,6 @@ uv run pytest --- -## Why “Compiler”? - -Context Compiler treats explicit user directives as inputs to a fixed, repeatable process. - -Instead of relying on the LLM to remember constraints across a conversation, user instructions are compiled into structured state before the model runs. - -The idea is similar to a traditional compiler: user directives are translated into a structured representation that the rest of the system can rely on. - ---- - ## FAQ **Is this just prompt reinjection?** @@ -166,7 +185,9 @@ Later in the conversation: User: how should I make this curry? ``` -Your app sends the saved state to the model so the rule still applies on later turns. +Your host sends the saved policy state with this later request, so the model is +constrained by explicit state (`peanuts: prohibit`) instead of relying on memory +of earlier conversation text. --- diff --git a/demos/README.md b/demos/README.md index 14e63c5..0cb5eb4 100644 --- a/demos/README.md +++ b/demos/README.md @@ -1,13 +1,14 @@ # LLM Demos -These scripts show common ways LLM conversations can go wrong. +These scripts show common reliability boundaries in LLM application behavior. They compare normal prompting with an approach where the application tracks important instructions explicitly instead of relying only on the conversation history. The scripts are designed to produce consistent results so the behavior is easy to see. -This demo set shows what users notice: rules and corrections keep applying -later in the conversation instead of fading over time. +This demo set shows what users notice: rules and corrections can keep applying +later in the conversation, and where explicit host-side state machinery is +required. Scored demos now compare four paths: - baseline @@ -149,8 +150,12 @@ Notes: - `PASS` means the demo-specific expected-behavior check for that path succeeded; `FAIL` means it did not. - `reinjected-state` can be enough for some persistence cases; this comparison is intended to show where deterministic state semantics add value. - Anti-overfitting guardrail: demos score host/state-transition invariants (for example blocked mutation, pending continuation, confirmation-only resolution), not model prose quality. Reinjected-state remains plain text injection only and does not include hidden compiler semantics. -- Interpretation: demos `01`-`05` and `07` focus on persistence and policy-following across turns, while demos `08`/`09` focus on fixed host-side state-transition rules. -- For demos `08`/`09`, similar outcomes across models are about architecture limits, not provider/model leaderboard ranking. +- Interpretation: +- Demos `01`-`05` and `07` mostly test persistence and policy-following behavior across turns. +- Demos `08`/`09` test fixed host-side state-transition rules. +- Demos `08` and `09` cover scenarios that require fixed host-side state rules, such as replacement precondition checks and confirmation flows. +- Plain prompt reinjection does not implement these behaviors by itself. These demos show a capability boundary rather than a prompt-following performance difference. +- Similar outcomes across models in `08`/`09` are about architecture limits, not provider/model leaderboard ranking. ### Demo 05 example (prompt drift under longer context) diff --git a/docs/demos-results.md b/docs/demos-results.md index e0478ac..cb4f915 100644 --- a/docs/demos-results.md +++ b/docs/demos-results.md @@ -83,9 +83,11 @@ compiler precondition or pending-state semantics. - Live demo runs are **evidence/smoke tests** across real model/provider behavior. - Deterministic test suites (unit/property tests) are the **regression authority** for oracle and engine contracts. -- Persistence demos and transition-semantics demos should be interpreted differently: -- `01`-`05`, `07`: persistence and policy-following under transcript pressure. -- `08`/`09`: host-side transition checks (for example, replacement preconditions and pending-confirmation handling). +- Persistence demos and transition-semantics demos should be interpreted differently. +- Demos `01`-`05` and `07` mostly test persistence and policy-following under transcript pressure. +- Demos `08`/`09` test fixed host-side state-transition rules. +- Demos `08` and `09` cover scenarios that require fixed host-side state rules, such as replacement precondition checks and confirmation flows. +- Plain prompt reinjection does not implement these behaviors by itself. These demos show a capability boundary rather than a prompt-following performance difference. - Demos `08`/`09` are not general LLM quality benchmarks. Baseline and reinjected-state can produce plausible text and still `FAIL` when those host-side transition checks are missing. ## Demo 05 Long-Transcript Stress (Exploratory Frontier Runs) diff --git a/docs/llm-preprocessor.md b/docs/llm-preprocessor.md index a09fb23..ca965d3 100644 --- a/docs/llm-preprocessor.md +++ b/docs/llm-preprocessor.md @@ -17,15 +17,15 @@ layer. Do not rely on repo-relative preprocessor paths. The preprocessor is a host adaptation layer, not an authoritative state engine. -The preprocessor is a host adaptation layer for environments where no capable -LLM is available to perform directive translation. +Model/tool-description translation can help with simple direct cases, but +integrations should not rely on model intent translation alone as the mutation +boundary. -When a capable LLM is present, it can translate user intent into canonical -directives directly (for example via MCP tool descriptions, an embedded model, -or another integration path). +In simpler hosts without an embedded model, this preprocessor provides a +conservative translation path. -In simpler hosts without an embedded model, this preprocessor fills that -translation role conservatively. +In model-assisted hosts, the same conservative boundary still applies: candidate +directives are suggestions until validated. Both paths feed canonical directives into the same deterministic engine. The compiler remains authoritative regardless of directive source. @@ -45,6 +45,7 @@ All preprocessor outputs, including heuristic outputs, must be validated with `parse_preprocessor_output(...)` before being applied. Raw heuristic/LLM outputs must not be passed directly to the compiler. +Raw model output must never directly mutate state. Pending clarification rule: @@ -53,6 +54,15 @@ Pending clarification rule: - This preserves deterministic continuation behavior because pending resolution accepts only confirmation tokens until resolved. +Host handling notes: + +- `passthrough` means no directive was applied; the host may call the model with + unchanged user input. +- `clarify` means mutation is blocked; the host should surface + `prompt_to_user` and wait for confirmation-style input. +- `update` means a validated canonical directive was applied to authoritative + state. + ## Limits The preprocessor is best-effort and intentionally conservative. Ambiguous, @@ -74,6 +84,26 @@ Boundary policy (explicit): Natural-language state proposal workflows should be handled by explicit host assist/proposal flows, not implicit preprocessing. +## Future direction (planning note) + +This section is architectural direction, not committed implementation. + +Future preprocessing may evolve beyond direct natural-language to directive +canonicalization. + +- Policy preprocessing and premise-like facts have different risk profiles. +- Premise-like facts (for example, `I am vegetarian`) may be useful persistent + context, but are high risk to auto-persist. +- Likely direction: + - keep directive preprocessing conservative and non-expansive + - add a separate inspectable, non-mutating suggestion layer for possible + persistent context + - require explicit host/user confirmation before any mutation + +This aligns with the post-0.7 / 0.8 direction: inspectable, previewable, +non-mutating suggestions with host-mediated confirmation, while the +authoritative engine remains deterministic and explicit. + ## Status This preprocessor surface is experimental and may evolve independently of the diff --git a/experimental/preprocessor/README.md b/experimental/preprocessor/README.md index 28aa78d..0b27e6b 100644 --- a/experimental/preprocessor/README.md +++ b/experimental/preprocessor/README.md @@ -5,6 +5,10 @@ Compiler integrations. It is experimental and separate from the deterministic core engine in `src/`. +Model/tool-description translation can help with simple direct cases, but raw +model output is not a safe mutation boundary by itself. Candidate directives +must be validated before they can mutate authoritative state. + Recommended install for integrations using this package: `pip install "context-compiler[experimental]"`. @@ -88,6 +92,11 @@ host-side assist workflow with preview/diff/confirmation, not this preprocessor. - use `engine.apply_transcript(...)` to update an existing engine Otherwise pass the original user input unchanged. +Decision handling reminder: +- `passthrough`: no directive applied; host may call the model normally. +- `clarify`: mutation blocked; host should surface `prompt_to_user`. +- `update`: validated canonical directive applied to authoritative state. + ## Prompt guidance - Use `prompts/default.txt` as the recommended default prompt. From ef704073ea7022a71533b7987a1e83f08e93150b Mon Sep 17 00:00:00 2001 From: Robert Lippmann Date: Tue, 26 May 2026 01:12:13 -0400 Subject: [PATCH 2/6] docs: tighten preprocessor mutation-boundary guidance --- docs/llm-preprocessor.md | 23 +++++++++++++---------- experimental/preprocessor/README.md | 28 +++++++++++++++++++++++++--- 2 files changed, 38 insertions(+), 13 deletions(-) diff --git a/docs/llm-preprocessor.md b/docs/llm-preprocessor.md index ca965d3..ad54626 100644 --- a/docs/llm-preprocessor.md +++ b/docs/llm-preprocessor.md @@ -30,6 +30,10 @@ directives are suggestions until validated. Both paths feed canonical directives into the same deterministic engine. The compiler remains authoritative regardless of directive source. +In MCP/tool-calling environments, over-eager tool calling on conversational or +ambiguous input is a known failure mode. Conservative preprocessing and +validation help reduce unintended mutation. + ## Required flow Recommended conceptual flow: @@ -56,12 +60,11 @@ Pending clarification rule: Host handling notes: -- `passthrough` means no directive was applied; the host may call the model with - unchanged user input. -- `clarify` means mutation is blocked; the host should surface - `prompt_to_user` and wait for confirmation-style input. -- `update` means a validated canonical directive was applied to authoritative - state. +- `passthrough`: no directive was applied; handle as ordinary user input. +- `clarify`: mutation is blocked; surface `prompt_to_user` and do not treat + state as updated. +- `update`: a validated directive was applied; use updated state as + authoritative. ## Limits @@ -93,11 +96,11 @@ canonicalization. - Policy preprocessing and premise-like facts have different risk profiles. - Premise-like facts (for example, `I am vegetarian`) may be useful persistent - context, but are high risk to auto-persist. + context, but should not be auto-persisted without confirmation. - Likely direction: - - keep directive preprocessing conservative and non-expansive - - add a separate inspectable, non-mutating suggestion layer for possible - persistent context + - keep conservative directive preprocessing separate + - add a possible suggestion layer that is inspectable, non-mutating, and + previewable - require explicit host/user confirmation before any mutation This aligns with the post-0.7 / 0.8 direction: inspectable, previewable, diff --git a/experimental/preprocessor/README.md b/experimental/preprocessor/README.md index 0b27e6b..caa1afb 100644 --- a/experimental/preprocessor/README.md +++ b/experimental/preprocessor/README.md @@ -9,6 +9,10 @@ Model/tool-description translation can help with simple direct cases, but raw model output is not a safe mutation boundary by itself. Candidate directives must be validated before they can mutate authoritative state. +In MCP/tool-calling environments, over-eager tool calling on conversational or +ambiguous input is a known failure mode. Conservative preprocessing and +validation help reduce unintended mutation. + Recommended install for integrations using this package: `pip install "context-compiler[experimental]"`. @@ -58,6 +62,7 @@ Engine-owned near-misses are reject cases (for example `set premise to X`, `change premise X`) and must remain `unknown` (not rewritten). Raw preprocessor/LLM outputs must not be passed directly to the compiler. +Raw model output must never directly mutate state. The preprocessor does not expand directive grammar. It may emit only validated canonical directives accepted by the compiler. @@ -93,9 +98,26 @@ host-side assist workflow with preview/diff/confirmation, not this preprocessor. Otherwise pass the original user input unchanged. Decision handling reminder: -- `passthrough`: no directive applied; host may call the model normally. -- `clarify`: mutation blocked; host should surface `prompt_to_user`. -- `update`: validated canonical directive applied to authoritative state. +- `passthrough`: no directive was applied; handle as ordinary user input. +- `clarify`: mutation is blocked; surface `prompt_to_user` and do not treat + state as updated. +- `update`: validated canonical directive applied; use updated state as + authoritative. + +## Future direction (planning note) + +This section is architectural direction, not committed implementation. + +Future preprocessing may evolve beyond direct natural-language to directive +canonicalization. + +- Policy directives and premise-like facts have different risk profiles. +- Premise-like facts (for example, `I am vegetarian`) may be useful persistent + context, but should not be auto-persisted without confirmation. +- Likely direction: + - conservative directive preprocessing remains separate + - a possible suggestion layer is inspectable, non-mutating, and previewable + - host/user confirmation is required before mutation ## Prompt guidance From ce20d79b8361ee98c91ab9da6d405b252b7cf59c Mon Sep 17 00:00:00 2001 From: Robert Lippmann Date: Tue, 26 May 2026 01:20:07 -0400 Subject: [PATCH 3/6] docs: simplify 0.7.1 framing language across docs --- README.md | 39 ++++++++++++++--------------- demos/README.md | 17 ++++++------- docs/demos-results.md | 28 ++++++++++----------- docs/llm-preprocessor.md | 34 ++++++++++++------------- experimental/preprocessor/README.md | 16 ++++++------ 5 files changed, 66 insertions(+), 68 deletions(-) diff --git a/README.md b/README.md index 9cdfc08..7dc37cf 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ Some behaviors require explicit host-side state machinery. Context Compiler is a deterministic host-side state layer for LLM applications. It handles explicit state transitions for premise and policies so that mutation -rules are fixed, inspectable, and repeatable. +rules are fixed and repeatable. ## What prompting and reinjection can do @@ -22,14 +22,14 @@ continuation. ## What prompting cannot do by itself -Prompt text (including reinjected state text) does not, by itself, define a -host-authoritative mutation boundary. By itself, it does not provide: +Prompt text (including reinjected state text) helps, but it does not give your +app controlled rules for when state can change. By itself, it does not provide: -- authoritative state transition rules +- rules your app controls for state changes - replacement precondition checks (`use X instead of Y` when `Y` may be absent) -- pending clarification continuation state -- deterministic decisions about when mutation is blocked -- reliable persistence/checkpoint boundaries for restoring both state and pending flow +- confirmation flows that must complete before anything else changes +- clear decisions about when a change is blocked +- reliable checkpoint restore for both saved state and pending confirmation flow ## What Context Compiler provides @@ -38,8 +38,8 @@ Context Compiler provides fixed host-side state machinery: - deterministic directive handling for explicit user state mutations - clarification instead of silent overwrite for blocked/ambiguous changes - pending confirmation flows that must resolve before continuation -- checkpoint export/import for restoring authoritative state and pending continuation -- structured authoritative state that the host can pass to the model +- checkpoint export/import for restoring saved state and pending confirmation flow +- structured saved state that the host can pass to the model The model generates responses. The compiler owns state transitions. @@ -62,9 +62,9 @@ Yes, on the current scored demo set. Interpretation guide: - Demos `01`-`05` and `07` focus on persistence and policy-following behavior. -- Demos `08`/`09` focus on fixed host-side state-transition rules. -- Demos `08`/`09` are capability-boundary checks, not prompt-following leaderboard comparisons. -- Plain reinjection can produce plausible responses, but it does not implement replacement precondition checks or pending confirmation flows by itself. +- Demos `08`/`09` focus on rules for when state is allowed to change. +- Demos `08`/`09` show what prompt text does not implement by itself. +- Plain reinjection can produce plausible responses, but it does not check whether replacement is allowed or wait for confirmation before saving changes. → [Full results and demo output](demos/README.md) Canonical matrix: [docs/demos-results.md](docs/demos-results.md) @@ -150,8 +150,7 @@ uv run pytest **Is this just prompt reinjection?** Reinjection helps with persistence, and it remains useful. Context Compiler -addresses a different boundary: authoritative state transitions. It defines how -state mutates before generation, with deterministic externalized semantics. +handles a different problem: rules for when state is allowed to change. Examples: - replacement semantics (`use X instead of Y`) when `Y` may not exist @@ -159,8 +158,8 @@ Examples: - lifecycle enforcement (for example, you cannot change an unset premise) - pending clarification flows that must be resolved before other mutations -In short: reinjection carries state forward; Context Compiler governs how that -state is created and changed. +In short: reinjection carries state forward; Context Compiler decides when your +app should change state. **Isn’t this just prompt engineering?** It complements prompt engineering, but solves a different problem. Prompting @@ -193,7 +192,7 @@ of earlier conversation text. ## Deterministic behavior (examples) -Context Compiler externalizes mutation semantics so transitions are deterministic. +Context Compiler makes mutation rules explicit so behavior stays repeatable. **Explicit directive** ```text @@ -216,7 +215,7 @@ clear state change premise to formal tone ``` - Without explicit transition checks: behavior depends on host/model handling -- Context Compiler: clarifies and preserves authoritative state +- Context Compiler: asks for clarification and keeps saved state unchanged --- @@ -323,7 +322,7 @@ The internal structure of the state is intentionally opaque to host applications - `export_json()` / `import_json()` transport **authoritative state only** - checkpoint APIs transport **serialized continuation**: - authoritative state - - pending confirmation-required continuation state + - pending confirmation flow state Checkpoint object shape: @@ -365,7 +364,7 @@ When to use checkpoint APIs: - stateless host/integration boundaries where engine instances are short-lived. - resume after interruption without losing pending clarification flow. -- preserve confirmation-required continuation state (`pending`) across process/request boundaries. +- preserve pending confirmation flow state (`pending`) across process/request boundaries. --- diff --git a/demos/README.md b/demos/README.md index 0cb5eb4..29ddb5f 100644 --- a/demos/README.md +++ b/demos/README.md @@ -1,14 +1,13 @@ # LLM Demos -These scripts show common reliability boundaries in LLM application behavior. +These scripts show common reliability limits in LLM app behavior. They compare normal prompting with an approach where the application tracks important instructions explicitly instead of relying only on the conversation history. The scripts are designed to produce consistent results so the behavior is easy to see. This demo set shows what users notice: rules and corrections can keep applying -later in the conversation, and where explicit host-side state machinery is -required. +later in the conversation, and where app-side state rules are required. Scored demos now compare four paths: - baseline @@ -148,14 +147,14 @@ Notes: - There are **8 scored demos** (`01`–`05`, `07`, `08`, `09`). `06_context_compaction` is informational and excluded from PASS/FAIL totals. - Anthropic runs in this repo are executed through the `openai_compatible` provider path. - `PASS` means the demo-specific expected-behavior check for that path succeeded; `FAIL` means it did not. -- `reinjected-state` can be enough for some persistence cases; this comparison is intended to show where deterministic state semantics add value. -- Anti-overfitting guardrail: demos score host/state-transition invariants (for example blocked mutation, pending continuation, confirmation-only resolution), not model prose quality. Reinjected-state remains plain text injection only and does not include hidden compiler semantics. +- `reinjected-state` can be enough for some persistence cases; this comparison shows where app-side state rules add value. +- Scored checks focus on app-side state rules (for example blocked mutation and confirmation-only resolution), not model prose quality. `reinjected-state` remains plain text injection only. - Interpretation: - Demos `01`-`05` and `07` mostly test persistence and policy-following behavior across turns. -- Demos `08`/`09` test fixed host-side state-transition rules. -- Demos `08` and `09` cover scenarios that require fixed host-side state rules, such as replacement precondition checks and confirmation flows. -- Plain prompt reinjection does not implement these behaviors by itself. These demos show a capability boundary rather than a prompt-following performance difference. -- Similar outcomes across models in `08`/`09` are about architecture limits, not provider/model leaderboard ranking. +- Demos `08`/`09` test rules for when state is allowed to change. +- Demos `08` and `09` cover cases prompt text does not implement by itself, such as checking whether replacement is allowed and waiting for confirmation before saving changes. +- Plain prompt reinjection can produce reasonable answers, but it does not run these checks by itself. +- Similar outcomes across models in `08`/`09` are about app behavior limits, not model leaderboard ranking. ### Demo 05 example (prompt drift under longer context) diff --git a/docs/demos-results.md b/docs/demos-results.md index cb4f915..00777c6 100644 --- a/docs/demos-results.md +++ b/docs/demos-results.md @@ -9,8 +9,8 @@ Note: this published matrix predates the `reinjected-state` path added in the 0. - Scored demos: `01`, `02`, `03`, `04`, `05`, `07` (6 total) - Informational demo: `06_context_compaction` (excluded from PASS/FAIL totals) -Methodology note (2026-05): the demo suite now also includes scored state-transition -semantics demos `08` and `09`. The published matrix below predates those additions and +Methodology note (2026-05): the demo suite now also includes scored demos `08` +and `09` for rules about when state is allowed to change. The published matrix below predates those additions and has not yet been fully rerun with the expanded scored set. ## Results Matrix @@ -67,10 +67,10 @@ Scoring behavior uses post-audit oracle/checker logic in demos and shared helper - `demos/09_llm_pending_clarification.py` - shared parsing/helpers in `demos/common.py` -Anti-overfitting note: scored checks are designed around deterministic host/state -invariants rather than preferred wording in model outputs. The `reinjected-state` -path is intentionally limited to plain state text injection and does not include -compiler precondition or pending-state semantics. +Scored checks focus on app-side state rules rather than preferred wording in +model outputs. The `reinjected-state` path is plain state text injection and +does not include compiler checks like replacement preconditions or pending +confirmation handling. ### Run metadata @@ -83,12 +83,12 @@ compiler precondition or pending-state semantics. - Live demo runs are **evidence/smoke tests** across real model/provider behavior. - Deterministic test suites (unit/property tests) are the **regression authority** for oracle and engine contracts. -- Persistence demos and transition-semantics demos should be interpreted differently. +- Persistence demos and state-change-rule demos should be interpreted differently. - Demos `01`-`05` and `07` mostly test persistence and policy-following under transcript pressure. -- Demos `08`/`09` test fixed host-side state-transition rules. -- Demos `08` and `09` cover scenarios that require fixed host-side state rules, such as replacement precondition checks and confirmation flows. -- Plain prompt reinjection does not implement these behaviors by itself. These demos show a capability boundary rather than a prompt-following performance difference. -- Demos `08`/`09` are not general LLM quality benchmarks. Baseline and reinjected-state can produce plausible text and still `FAIL` when those host-side transition checks are missing. +- Demos `08`/`09` test rules for when state is allowed to change. +- Demos `08` and `09` cover cases prompt text does not implement by itself, such as checking whether replacement is allowed and waiting for confirmation before saving changes. +- Plain prompt reinjection can produce reasonable answers, but it does not run these checks by itself. +- Demos `08`/`09` are not general LLM quality benchmarks. Baseline and reinjected-state can produce plausible text and still `FAIL` when those app-side checks are missing. ## Demo 05 Long-Transcript Stress (Exploratory Frontier Runs) @@ -101,9 +101,9 @@ records the standard scored demo configuration. - In those exploratory runs, `reinjected-state`, `compiler`, and `compiler+compact` continued to preserve premise-consistent behavior. -This is exploratory evidence rather than deterministic benchmark authority. Reinjection can -be sufficient in some persistence scenarios, while compiler-mediated paths still provide -deterministic state-transition semantics. +This is exploratory evidence, not benchmark authority. Reinjection can be +enough in some persistence scenarios, while compiler-mediated paths still +provide explicit state-change rules. ## Local Ollama Context-Size Sweep (0.7.1 Experiment) diff --git a/docs/llm-preprocessor.md b/docs/llm-preprocessor.md index ad54626..3d0a6fe 100644 --- a/docs/llm-preprocessor.md +++ b/docs/llm-preprocessor.md @@ -4,8 +4,8 @@ The experimental preprocessor is an optional host-side layer that can convert natural-language messages into canonical Context Compiler directives before compilation. -The compiler remains deterministic and authoritative. The preprocessor does not -replace core parsing or state semantics. +The compiler keeps state rules fixed. The preprocessor does not replace core +parsing or state rules. Install path for integrations using this layer: `pip install "context-compiler[experimental]"`. @@ -15,20 +15,20 @@ layer. Do not rely on repo-relative preprocessor paths. ## Architectural framing -The preprocessor is a host adaptation layer, not an authoritative state engine. +The preprocessor is a helper layer in your app, not the source of truth for +state changes. Model/tool-description translation can help with simple direct cases, but -integrations should not rely on model intent translation alone as the mutation -boundary. +integrations should not rely on model intent translation alone to decide when +state changes. In simpler hosts without an embedded model, this preprocessor provides a conservative translation path. -In model-assisted hosts, the same conservative boundary still applies: candidate -directives are suggestions until validated. +In model-assisted hosts, outputs are still checked before being applied. Both paths feed canonical directives into the same deterministic engine. The -compiler remains authoritative regardless of directive source. +engine still controls state updates regardless of where the directive came from. In MCP/tool-calling environments, over-eager tool calling on conversational or ambiguous input is a known failure mode. Conservative preprocessing and @@ -49,14 +49,14 @@ All preprocessor outputs, including heuristic outputs, must be validated with `parse_preprocessor_output(...)` before being applied. Raw heuristic/LLM outputs must not be passed directly to the compiler. -Raw model output must never directly mutate state. +Raw model output must never directly change state. Pending clarification rule: - If the engine has pending clarification state, bypass preprocessing and pass raw user input directly to `engine.step(...)`. -- This preserves deterministic continuation behavior because pending resolution - accepts only confirmation tokens until resolved. +- This keeps confirmation flows correct: while confirmation is pending, only + confirmation-style input should resolve it. Host handling notes: @@ -74,7 +74,7 @@ clarification behavior. Boundary policy (explicit): -- Whole-message canonicalization only. +- Only process the full message, not pieces of it. - At most one canonical directive may be emitted; otherwise abstain. - Do not extract directives from surrounding prose, questions, or reporting. - Do not split sentences or mine multi-line batches for commands. @@ -84,15 +84,15 @@ Boundary policy (explicit): payload quotes (for example `use "docker"` remains quoted). - Prefer false negatives over false positive state mutation. -Natural-language state proposal workflows should be handled by explicit host -assist/proposal flows, not implicit preprocessing. +If you want natural-language proposals for state, handle that in an explicit +host flow. Do not treat implicit preprocessing as state mutation. ## Future direction (planning note) This section is architectural direction, not committed implementation. Future preprocessing may evolve beyond direct natural-language to directive -canonicalization. +conversion. - Policy preprocessing and premise-like facts have different risk profiles. - Premise-like facts (for example, `I am vegetarian`) may be useful persistent @@ -104,8 +104,8 @@ canonicalization. - require explicit host/user confirmation before any mutation This aligns with the post-0.7 / 0.8 direction: inspectable, previewable, -non-mutating suggestions with host-mediated confirmation, while the -authoritative engine remains deterministic and explicit. +non-mutating suggestions that users review first and confirm before saving, +while the engine keeps state rules explicit and repeatable. ## Status diff --git a/experimental/preprocessor/README.md b/experimental/preprocessor/README.md index caa1afb..ad7455b 100644 --- a/experimental/preprocessor/README.md +++ b/experimental/preprocessor/README.md @@ -6,8 +6,8 @@ Compiler integrations. It is experimental and separate from the deterministic core engine in `src/`. Model/tool-description translation can help with simple direct cases, but raw -model output is not a safe mutation boundary by itself. Candidate directives -must be validated before they can mutate authoritative state. +model output is not safe by itself for state changes. Outputs must be validated +before anything is applied. In MCP/tool-calling environments, over-eager tool calling on conversational or ambiguous input is a known failure mode. Conservative preprocessing and @@ -62,14 +62,14 @@ Engine-owned near-misses are reject cases (for example `set premise to X`, `change premise X`) and must remain `unknown` (not rewritten). Raw preprocessor/LLM outputs must not be passed directly to the compiler. -Raw model output must never directly mutate state. +Raw model output must never directly change state. The preprocessor does not expand directive grammar. It may emit only validated canonical directives accepted by the compiler. Conservative boundary policy: -- Whole-message canonicalization only (no directive mining from prose). +- Only process the full message, not pieces of it (no directive mining from prose). - Emit at most one canonical directive; otherwise abstain. - No sentence splitting or hidden multi-line extraction. - No mixed directive + task extraction. @@ -79,8 +79,8 @@ Conservative boundary policy: - Quoted payload tokens inside an otherwise canonical directive (for example `use "docker"`) are preserved as-is; they are not silently unquoted. -If you need natural-language proposal/orchestration behavior, use an explicit -host-side assist workflow with preview/diff/confirmation, not this preprocessor. +If you need natural-language proposal behavior, use an explicit host workflow +that shows suggestions first and asks for confirmation. ## Safe usage pattern @@ -109,7 +109,7 @@ Decision handling reminder: This section is architectural direction, not committed implementation. Future preprocessing may evolve beyond direct natural-language to directive -canonicalization. +conversion. - Policy directives and premise-like facts have different risk profiles. - Premise-like facts (for example, `I am vegetarian`) may be useful persistent @@ -117,7 +117,7 @@ canonicalization. - Likely direction: - conservative directive preprocessing remains separate - a possible suggestion layer is inspectable, non-mutating, and previewable - - host/user confirmation is required before mutation + - user confirms before anything is saved ## Prompt guidance From 1bad3ed666f34d9a92cb6a60d85dafa4288b3062 Mon Sep 17 00:00:00 2001 From: Robert Lippmann Date: Tue, 26 May 2026 01:24:21 -0400 Subject: [PATCH 4/6] docs: polish final prose across 0.7.1 docs --- README.md | 26 +++++++++++++------------- demos/README.md | 4 ++-- docs/demos-results.md | 4 ++-- docs/llm-preprocessor.md | 12 ++++++------ experimental/preprocessor/README.md | 10 +++++----- 5 files changed, 28 insertions(+), 28 deletions(-) diff --git a/README.md b/README.md index 7dc37cf..09ebbec 100644 --- a/README.md +++ b/README.md @@ -16,9 +16,9 @@ Prompting and reinjection are useful. In many real systems, reinjecting saved state text is enough to keep instructions and policies persistent across turns. Context Compiler adds host-owned transition rules for behaviors that plain text -reinjection does not implement by itself: replacement preconditions, blocked -mutation with clarification, pending confirmation state, and checkpointed -continuation. +reinjection does not implement by itself: replace `X` only if `X` exists, block +conflicting changes and ask for confirmation, and restore saved state plus +pending confirmations from checkpoints. ## What prompting cannot do by itself @@ -35,9 +35,9 @@ app controlled rules for when state can change. By itself, it does not provide: Context Compiler provides fixed host-side state machinery: -- deterministic directive handling for explicit user state mutations +- deterministic directive handling for explicit user state changes - clarification instead of silent overwrite for blocked/ambiguous changes -- pending confirmation flows that must resolve before continuation +- pending confirmation flows that must resolve before anything else changes - checkpoint export/import for restoring saved state and pending confirmation flow - structured saved state that the host can pass to the model @@ -81,9 +81,9 @@ context-compiler --json < input.txt `context-compiler` launches the interactive REPL. `--with-preprocessor` enables the experimental preprocessor before each REPL turn -(simple rule-based handling plus conservative validation). Near-miss inputs are not rewritten and are -passed through to the engine, which continues to return clarify behavior for -those forms. +(simple rule-based checks plus conservative validation). For near-miss inputs, +the preprocessor does not rewrite the text. It passes the input to the engine, +and the engine can return `clarify`. `--json` enables machine-readable NDJSON output for non-interactive usage (one complete JSON object per processed input line). @@ -95,7 +95,7 @@ Preload options keep saved rules separate from in-progress confirmation state: continuation checkpoint (saved state + pending confirmation state). REPL commands (controller layer, not engine directives): -- `state` shows current authoritative state. +- `state` shows current saved state. - `preview ` runs deterministic dry-run without mutating live state. - `step ` is an explicit alias of normal bare-input step behavior. @@ -163,8 +163,8 @@ app should change state. **Isn’t this just prompt engineering?** It complements prompt engineering, but solves a different problem. Prompting -shapes model behavior; Context Compiler provides a deterministic state layer -that updates only through explicit directives. +shapes model behavior. Context Compiler enforces state rules and updates state +only through explicit directives. --- @@ -199,7 +199,7 @@ Context Compiler makes mutation rules explicit so behavior stays repeatable. set premise concise replies ``` - Base model: silently accepts / rewrites -- Context Compiler: applies a deterministic state update +- Context Compiler: applies a repeatable state update **State-dependent operation** ```text @@ -207,7 +207,7 @@ clear state use podman instead of docker ``` - Without explicit state transition rules: behavior depends on host/model handling -- Context Compiler: returns deterministic clarify behavior before mutation +- Context Compiler: returns `clarify` before changing state **Lifecycle enforcement** ```text diff --git a/demos/README.md b/demos/README.md index 29ddb5f..9deb519 100644 --- a/demos/README.md +++ b/demos/README.md @@ -7,7 +7,7 @@ important instructions explicitly instead of relying only on the conversation history. The scripts are designed to produce consistent results so the behavior is easy to see. This demo set shows what users notice: rules and corrections can keep applying -later in the conversation, and where app-side state rules are required. +later in the conversation, and where your app needs explicit state rules. Scored demos now compare four paths: - baseline @@ -154,7 +154,7 @@ Notes: - Demos `08`/`09` test rules for when state is allowed to change. - Demos `08` and `09` cover cases prompt text does not implement by itself, such as checking whether replacement is allowed and waiting for confirmation before saving changes. - Plain prompt reinjection can produce reasonable answers, but it does not run these checks by itself. -- Similar outcomes across models in `08`/`09` are about app behavior limits, not model leaderboard ranking. +- Similar outcomes across models in `08`/`09` reflect app behavior limits, not model leaderboard ranking. ### Demo 05 example (prompt drift under longer context) diff --git a/docs/demos-results.md b/docs/demos-results.md index 00777c6..c2cdfc4 100644 --- a/docs/demos-results.md +++ b/docs/demos-results.md @@ -108,8 +108,8 @@ provide explicit state-change rules. ## Local Ollama Context-Size Sweep (0.7.1 Experiment) This section reports the refreshed local-only matrix with the `reinjected-state` -path and explicit context-size ladder runs. Historical hosted-provider matrix rows -above are preserved as originally recorded. +path and explicit context-size ladder runs. Historical hosted-provider matrix +rows above remain as originally recorded. ### Commands Run diff --git a/docs/llm-preprocessor.md b/docs/llm-preprocessor.md index 3d0a6fe..b4b6f7a 100644 --- a/docs/llm-preprocessor.md +++ b/docs/llm-preprocessor.md @@ -1,6 +1,6 @@ # LLM Preprocessor (Optional, Experimental) -The experimental preprocessor is an optional host-side layer that can convert +The experimental preprocessor is an optional app-side layer that can convert natural-language messages into canonical Context Compiler directives before compilation. @@ -25,10 +25,10 @@ state changes. In simpler hosts without an embedded model, this preprocessor provides a conservative translation path. -In model-assisted hosts, outputs are still checked before being applied. +In model-assisted hosts, the app still checks outputs before applying them. -Both paths feed canonical directives into the same deterministic engine. The -engine still controls state updates regardless of where the directive came from. +Both paths send canonical directives to the same deterministic engine. The +engine still controls state updates. In MCP/tool-calling environments, over-eager tool calling on conversational or ambiguous input is a known failure mode. Conservative preprocessing and @@ -63,8 +63,8 @@ Host handling notes: - `passthrough`: no directive was applied; handle as ordinary user input. - `clarify`: mutation is blocked; surface `prompt_to_user` and do not treat state as updated. -- `update`: a validated directive was applied; use updated state as - authoritative. +- `update`: the app applied a validated directive; use updated state as the + source of truth. ## Limits diff --git a/experimental/preprocessor/README.md b/experimental/preprocessor/README.md index ad7455b..b94b7ff 100644 --- a/experimental/preprocessor/README.md +++ b/experimental/preprocessor/README.md @@ -1,13 +1,13 @@ # Experimental Preprocessor Package -This package provides optional host-layer preprocessing utilities for Context +This package provides optional app-layer preprocessing utilities for Context Compiler integrations. It is experimental and separate from the deterministic core engine in `src/`. Model/tool-description translation can help with simple direct cases, but raw -model output is not safe by itself for state changes. Outputs must be validated -before anything is applied. +model output is not safe by itself for state changes. The app must validate +outputs before applying them. In MCP/tool-calling environments, over-eager tool calling on conversational or ambiguous input is a known failure mode. Conservative preprocessing and @@ -25,7 +25,7 @@ Compatibility note: ## Modules - `heuristic_preprocessor.py`: conservative structural preprocessing pass. -- `output_validation.py`: shared normalization/validation boundary. +- `output_validation.py`: shared normalization and validation checks. - `prompt_utils.py`: state-aware prompt rendering helper. - `constants.py`: shared protocol literals and directive validation patterns. - `prompts/default.txt`: default runtime prompt. @@ -102,7 +102,7 @@ Decision handling reminder: - `clarify`: mutation is blocked; surface `prompt_to_user` and do not treat state as updated. - `update`: validated canonical directive applied; use updated state as - authoritative. + the source of truth. ## Future direction (planning note) From b9e5546dd7b6237a81df1713e49ad6553815f9b9 Mon Sep 17 00:00:00 2001 From: Robert Lippmann Date: Tue, 26 May 2026 01:34:18 -0400 Subject: [PATCH 5/6] docs: simplify integration docs for practical app flows --- examples/README.md | 20 +++++++------- examples/integrations/README.md | 11 ++++---- examples/integrations/litellm/README.md | 23 ++++++++++------ examples/integrations/litellm_proxy/README.md | 7 ++--- examples/integrations/openwebui/README.md | 27 ++++++++++++------- 5 files changed, 53 insertions(+), 35 deletions(-) diff --git a/examples/README.md b/examples/README.md index 6c0976f..19424c9 100644 --- a/examples/README.md +++ b/examples/README.md @@ -1,14 +1,14 @@ # Examples -This directory contains small integration examples showing typical host-side usage of the Context Compiler. +This directory contains small integration examples showing typical app-side usage of Context Compiler. These files are included in the base package install (`pip install context-compiler`). Non-integration example files in this directory are standalone scripts and can be run directly. ## 01_persistent_guardrails.py -Demonstrates how a prohibition persists as authoritative state across later turns. -Shows the host using compiled state to keep constraints active. +Shows how a prohibition stays in saved state across later turns. +Shows the app sending saved state so the rule keeps applying. ## 02_configuration_and_correction.py @@ -17,23 +17,23 @@ Shows `set premise ...` followed by `change premise to ...`. ## 03_ambiguity_with_clarification.py -Demonstrates contradiction clarify behavior before state mutation. -Shows how the host handles clarify results and blocks LLM calls. +Shows `clarify` behavior before state changes. +Shows how the app handles `clarify` and skips the LLM call. ## 04_tool_governance_denylist.py -Demonstrates tool-governance policy handling via prohibition directives. -Shows how hosts can prevent denied tools from being selected. +Shows tool-governance policy handling via `prohibit ...` directives. +Shows how apps can prevent denied tools from being selected. ## 05_llm_integration_pattern.py -Demonstrates end-to-end host control flow around compiler outcomes. -Shows when to clarify, when to call the model, and how to include compiled state in prompts. +Shows end-to-end app control flow around compiler outcomes. +Shows what to do on `clarify`, when to call the model, and how to include saved state in prompts. Includes a single-item policy removal step via `remove policy `. ## 06_transcript_replay.py -Demonstrates transcript replay helpers for host integration. +Shows transcript replay helpers for app integration. Shows `compile_transcript(messages)` from a fresh engine and `engine.apply_transcript(messages)` on current engine state. ## 07_single_policy_correction.py diff --git a/examples/integrations/README.md b/examples/integrations/README.md index f5947df..5d3a3fc 100644 --- a/examples/integrations/README.md +++ b/examples/integrations/README.md @@ -1,6 +1,6 @@ # Integrations -These examples show how to integrate Context Compiler with external systems. +These examples show how to use Context Compiler inside external app runtimes. ## LiteLLM (SDK) @@ -26,9 +26,10 @@ See the LiteLLM examples README for setup and usage: ### Behavior -- Context Compiler runs before any LLM call. -- If clarification is required, no LLM call is made. -- Otherwise, saved compiler state is added to the prompt before calling the model. +- Context Compiler runs before each LLM call. +- If result is `clarify`, show the question and do not call the LLM. +- If result is `passthrough`, send normal user input. +- If result is `update`, use updated state and call the model with saved state added to the prompt. ## LiteLLM Proxy @@ -40,7 +41,7 @@ See: [LiteLLM Proxy README](litellm_proxy/README.md) Tested target: Open WebUI `v0.8.12`. -Open WebUI is host-provided runtime infrastructure and must be installed/configured separately. +Open WebUI is a separate runtime and must be installed/configured separately. Files: - Basic example: [open_webui_pipe.py](openwebui/open_webui_pipe.py) diff --git a/examples/integrations/litellm/README.md b/examples/integrations/litellm/README.md index 87eb921..d4cc4ff 100644 --- a/examples/integrations/litellm/README.md +++ b/examples/integrations/litellm/README.md @@ -48,7 +48,7 @@ print(handle_turn("set premise to concise replies", engine)) PY ``` -This near-miss input is expected to clarify rather than be canonicalized. +This near-miss input should return `clarify` instead of being rewritten. ## Environment configuration @@ -110,15 +110,17 @@ These files are importable integration references for host applications. - Import `handle_turn(...)` from either `basic.py` or `with_preprocessor.py`. - Create and retain an engine instance in host/session state. - Pass each user input through `handle_turn(user_input, engine)`. -- Optional serialized continuation checkpointing: pass `session_key=...` and - let the example integration restore before first `engine.step(...)` and - persist after `update`/`clarify` decisions. +- Optional checkpointing: pass `session_key=...`. + The example restores checkpoint data before the first `engine.step(...)` and + saves checkpoint data after `update`/`clarify`. - In this example, checkpoint/session storage is in-memory only. - Continuation state is limited to the current process lifetime; real restart - continuity requires external persistence (DB/Redis/etc.). + State lasts only for the current process. To survive restarts, store + checkpoints in external storage (DB/Redis/etc.). - Display the returned assistant text. -Note: In these LiteLLM example integrations, update decisions are rendered locally in a fixed, repeatable way and do not call the downstream LLM. This makes state transitions explicit. Production hosts may choose different rendering behavior. +Note: In these LiteLLM examples, `update` is rendered locally and does not call +the downstream LLM. This makes state changes explicit. Production apps may +choose different rendering behavior. ## Troubleshooting @@ -137,6 +139,11 @@ Note: In these LiteLLM example integrations, update decisions are rendered local - If fallback yields nothing usable or errors, behavior safely remains equivalent to basic. - Behavior is reject-first and does not broaden the directive grammar. +Decision flow in both examples: +- `passthrough`: call the model with normal input. +- `clarify`: show `prompt_to_user`; do not treat state as changed. +- `update`: state changed; use updated state for the next model call. + ## Example checks - Near-miss passthrough (`with_preprocessor.py`): @@ -144,7 +151,7 @@ Note: In these LiteLLM example integrations, update decisions are rendered local - Engine returns clarify (`Did you mean 'set premise concise replies'?`). - Lifecycle enforcement (both): - `change premise to formal tone` with no premise -> clarify (`set premise ...` first). -- Conflict semantics (both): +- Conflict behavior (both): - `use docker` then `prohibit docker` -> conflict clarify. - Replacement precondition (both): - `use podman instead of docker` without prior `use docker` -> replacement clarify. diff --git a/examples/integrations/litellm_proxy/README.md b/examples/integrations/litellm_proxy/README.md index c87dd94..ea4f65f 100644 --- a/examples/integrations/litellm_proxy/README.md +++ b/examples/integrations/litellm_proxy/README.md @@ -1,7 +1,7 @@ # LiteLLM Proxy (pre-call hook) This example shows how to run Context Compiler inside a LiteLLM proxy pre-call hook. -The hook applies fixed, repeatable state handling before any upstream model call. +The hook applies fixed state handling before any upstream model call. Available hook files: @@ -83,8 +83,9 @@ curl http://localhost:4000/v1/chat/completions \ ### Behavior - User messages are replayed through Context Compiler before the model call. -- If clarification is required, the proxy returns the clarification text as the response instead of calling the model. -- Otherwise, compiler state is added as a system message. +- If result is `clarify`, the proxy returns clarification text and does not call the model. +- If result is `passthrough`, the proxy forwards the request normally. +- If result is `update`, the proxy injects compiler state as a system message and then calls the model. Preprocessor-enabled variant behavior: diff --git a/examples/integrations/openwebui/README.md b/examples/integrations/openwebui/README.md index 88c881a..08cd7f1 100644 --- a/examples/integrations/openwebui/README.md +++ b/examples/integrations/openwebui/README.md @@ -23,7 +23,7 @@ The minimal pipe path below is the easiest first-run flow and was runtime-valida 4. Set `BASE_MODEL_ID` to a valid Open WebUI model id (required). 5. Select the pipe model in chat. -Open WebUI is host-provided runtime infrastructure and must already be installed/configured separately. +Open WebUI is a separate runtime and must already be installed/configured separately. Open WebUI also needs at least one real backend model/provider configured (for example Ollama or OpenAI) so `BASE_MODEL_ID` resolves to an actual model. Note: The `PROVIDER` environment contract used in LiteLLM examples/demos does not apply to OpenWebUI. OpenWebUI manages providers via its own connection settings and model IDs. @@ -84,12 +84,16 @@ Use the Open WebUI model picker/list to copy exact model ids for `BASE_MODEL_ID` ## Manual Validation -Validate clarify short-circuit, passthrough forwarding without state injection, -update forwarding with compiler state (`[[cc_state]]`) added to the request, chat isolation -with real chat ids, restart state loss, and non-text bypass behavior. +Validate these behaviors: +- `clarify` short-circuits the LLM call. +- `passthrough` forwards input without state injection. +- `update` forwards with compiler state (`[[cc_state]]`) added to the request. +- chat isolation works with real chat ids. +- state is lost after restart (no external persistence). +- non-text input is bypassed. Note: In the OpenWebUI example pipes, recognized directive-only `update` -decisions return deterministic local acknowledgments and do not call the +decisions return fixed local acknowledgments and do not call the downstream LLM. Both pipes support an exact local inspection command: `show state`. When the latest user message is exactly `show state` (case-insensitive after trim), @@ -97,6 +101,11 @@ the pipe returns current compiler state locally and does not call the downstream When trace is enabled, responses include concise evidence of decision kind, active state, downstream LLM call/no-call, and whether state was injected. +Decision flow in both pipes: +- `passthrough`: call the downstream model with normal input. +- `clarify`: show `prompt_to_user`; do not change saved state. +- `update`: state changed; render local acknowledgment for directive-only input, or call downstream model with updated state injected. + ## Behavioral comparisons **Case 1** @@ -113,7 +122,7 @@ active state, downstream LLM call/no-call, and whether state was injected. - base model: generic Docker/prohibition guidance text - basic pipe: `'docker' is already in use. Only one policy per item is allowed. Use 'reset policies' to change it.` - preprocessor pipe: same conflict clarify -- why this matters: explicit conflict semantics are preserved instead of conversational interpretation. +- why this matters: the app asks before applying a conflicting change. **Case 3** @@ -121,7 +130,7 @@ active state, downstream LLM call/no-call, and whether state was injected. - base model: generic “how to switch to Podman” tutorial - basic pipe: `No exact policy found for "docker". Replacement requires an exact policy match...` - preprocessor pipe: same replacement clarify -- why this matters: replacement precondition (old item must exist) is enforced. +- why this matters: the app only replaces a policy when the old item already exists. **Case 4** @@ -129,7 +138,7 @@ active state, downstream LLM call/no-call, and whether state was injected. - base model: accepts conversational style phrasing - basic pipe: `Did you mean 'set premise concise replies'?` - preprocessor pipe: same clarify (near-miss is not rewritten) -- why this matters: preprocessor stays reject-first and preserves engine-owned clarify behavior. +- why this matters: near-miss text is not silently rewritten. **Case 5** @@ -137,4 +146,4 @@ active state, downstream LLM call/no-call, and whether state was injected. - base model: generic “please clarify changes” response - basic pipe: `Did you mean 'change premise to concise replies'?` - preprocessor pipe: same clarify (near-miss is passed through unchanged) -- why this matters: near-miss inputs are not canonicalized, so directive semantics stay engine-owned. +- why this matters: the app waits for explicit, valid directive text before changing state. From c10313d63396c4e7c58acc02b90e9159529b915e Mon Sep 17 00:00:00 2001 From: Robert Lippmann Date: Tue, 26 May 2026 01:35:45 -0400 Subject: [PATCH 6/6] chore: bump version to 0.7.1 --- pyproject.toml | 2 +- uv.lock | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/pyproject.toml b/pyproject.toml index bad4433..22f9363 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "hatchling.build" [project] name = "context-compiler" -version = "0.7.0" +version = "0.7.1" description = "Deterministic conversational state engine for LLM applications." readme = "README.md" requires-python = ">=3.11" diff --git a/uv.lock b/uv.lock index 4cc5631..154f740 100644 --- a/uv.lock +++ b/uv.lock @@ -468,7 +468,7 @@ wheels = [ [[package]] name = "context-compiler" -version = "0.7.0" +version = "0.7.1" source = { editable = "." } [package.optional-dependencies]