diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..190533d --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,36 @@ +# Contributing + +Thanks for your interest in improving this project. Contributions are welcome. + +## Workflow + +Contributions are typically submitted via fork and pull request: + +- fork the repository +- create a feature branch +- run tests and pre-commit checks +- open a pull request + +## Development Setup + +```bash +uv sync --group dev +``` + +## Running Tests + +```bash +uv run pytest +``` + +## Code Quality + +```bash +pre-commit run --all-files +``` + +## Scope of Changes + +- keep pull requests focused +- include tests if behavior changes +- open an issue first for large design changes diff --git a/README.md b/README.md index 9d946db..a044c09 100644 --- a/README.md +++ b/README.md @@ -1,16 +1,30 @@ # Context Compiler -Deterministic conversational state for LLM systems. +[![PyPI version](https://img.shields.io/pypi/v/context-compiler)](https://pypi.org/project/context-compiler/) +[![Python versions](https://img.shields.io/pypi/pyversions/context-compiler)](https://pypi.org/project/context-compiler/) +[![License](https://img.shields.io/pypi/l/context-compiler)](https://pypi.org/project/context-compiler/) -Modern language models are strong at reasoning but unreliable at maintaining consistent state across interactions. Corrections compete with earlier statements, constraints disappear, and long conversations accumulate contradictions. +A deterministic directive engine that converts explicit user instructions +into structured conversational state for LLM applications. + +Modern language models reason well but are unreliable at maintaining +consistent state across interactions. + +Corrections compete with earlier statements, constraints disappear, +and long conversations accumulate contradictions. The **Context Compiler** introduces a deterministic state layer that governs authoritative conversational state independently of the model. -The model performs reasoning and generation. -The compiler manages facts and constraints. +The model performs reasoning and generation while the compiler manages facts and constraints. Once accepted, directives remain authoritative until explicitly corrected or reset. + +## Installation -Once a directive is accepted, it becomes authoritative for the remainder of the session. +- Python 3.11+ +- `pip install context-compiler` +- Dev/test: `uv sync --group dev` and `uv run pytest` +- Examples: see [examples/README.md](examples/README.md) +- Demonstrations: see [demos/README.md](demos/README.md) --- @@ -46,22 +60,6 @@ The host supplies the authoritative state to the model so the constraint persist --- -## Why Not Just Prompt Engineering? - -Prompt instructions are soft and easy to lose across long interactions. - -The Context Compiler gives the host an **authoritative state representation** that is independent of transcript drift. - -Only **explicit user directives** can modify state. - -In other words: - -- the model reasons -- the compiler decides whether state changes -- the host controls when the LLM runs - ---- - ## Architecture ```text @@ -76,30 +74,8 @@ Host Application LLM ``` -The compiler governs conversational state. It never calls the LLM. - -The LLM performs reasoning and generation but cannot modify authoritative state. - -The host application decides whether the model runs based on the compiler’s `Decision`. - -### Compiler responsibilities - -The compiler: - -1. Parses user input -2. Detects explicit directives -3. Ensures mutations are unambiguous -4. Returns a deterministic `Decision` - -The compiler **never calls the LLM**. - -### Host responsibilities - -The host: - -- displays clarification prompts -- calls the LLM when allowed -- formats prompts using compiled state +The compiler governs authoritative conversational state and never calls the LLM. +The host decides whether to call the model based on the returned `Decision`. --- @@ -122,6 +98,30 @@ Meaning: | update | forward input with updated state | | clarify | show `prompt_to_user` and do not call the LLM | +### Host Integration Example + +```python +engine = create_engine() +decision = engine.step(user_input) + +if decision["kind"] == "clarify": + show_to_user(decision["prompt_to_user"]) +else: + state = decision["state"] or engine.state + messages = build_messages(state, user_input) + render(call_llm(messages)) +``` + +### API Reference + +| API | Description | +|---|---| +| `create_engine(...)` | Create a new compiler engine, optionally with replacement initial state. | +| `step(user_input)` | Parse one user turn and return a deterministic `Decision`. | +| `engine.state` | Read or replace full in-memory authoritative state. | +| `export_json()` | Export current state as JSON for persistence/transport. | +| `import_json(payload)` | Load state from exported JSON payload. | + --- ## State Model @@ -142,57 +142,22 @@ The compiler maintains an authoritative state: ## State Access and Persistence -The compiler exposes authoritative state through in-memory replacement APIs -and JSON persistence/transport APIs. - -Supported host interaction model: - -```python -engine = create_engine() -engine = create_engine(state=initial_state_obj) -decision = engine.step(user_input) -state = engine.state -engine.state = replacement_state_obj -payload = engine.export_json() -engine.import_json(payload) -``` - -API boundaries: - -- `step()` performs directive-driven mutation. -- `engine.state` / `engine.state = ...` are in-memory inspection/replacement APIs. -- `export_json()` / `import_json()` are persistence/transport APIs. -- No imperative convenience mutation methods are provided; operations such as - `reset policies` and `clear state` are handled through directive input to `step()`. - -Example persistence lifecycle: - -```python -payload = engine.export_json() -save_to_storage(payload) - -restored_payload = load_from_storage() -engine = create_engine() -engine.import_json(restored_payload) -``` +Hosts may inspect or replace in-memory state (`engine.state`) or persist it using `export_json()` and `import_json()`. State changes occur only through directives processed by `step()`. Storage is managed by the host application. -The compiler does not manage storage or snapshots. Persistence policies -belong to the host application. +### Fact Schema -**Note** -The fact schema currently contains a single exclusive slot: `facts["focus.primary"]`. - -This slot exists to demonstrate deterministic fact replacement and correction semantics. -Richer fact schemas may be introduced in future releases. +The current schema contains a single exclusive slot: `facts["focus.primary"]`. +This demonstrates deterministic fact replacement and correction behavior. +Richer schemas may be introduced in future releases. ### State Properties -- **Facts are exclusive** (last write wins) -- **Policies are additive** -- **No inference or semantic reasoning** -- **State is deterministic** +- Facts are exclusive (last write wins) +- Policies are additive +- No inference or semantic reasoning -The same input sequence always produces the same state. +Identical input sequences always produce identical compiler state. +LLM responses may still vary unless deterministic decoding is used by the host. --- @@ -295,22 +260,9 @@ After `clear state`: ## Examples -The `examples/` directory contains small integration demonstrations. - -- [Persistent guardrails](examples/01_persistent_guardrails.py) - Demonstrates constraints persisting across turns. - -- [Configuration with correction](examples/02_configuration_and_correction.py) - Shows deterministic fact replacement. +Integration examples are available in the [examples/](examples/) directory. -- [Ambiguity detection with clarification](examples/03_ambiguity_with_clarification.py) - Demonstrates clarification before state mutation. - -- [Tool governance for agents](examples/04_tool_governance_denylist.py) - Shows how host applications can block tools using compiler policies. - -- [LLM integration pattern](examples/05_llm_integration_pattern.py) - Demonstrates the host control flow around the `Decision` API. +See [examples/README.md](examples/README.md) for walkthroughs. --- @@ -331,66 +283,28 @@ python examples/01_persistent_guardrails.py Run tests: ```bash -pytest +uv run pytest ``` --- ## Guarantees -The compiler enforces several invariants: - -- State never changes without explicit directive or confirmation -- The same input sequence always produces the same state -- LLM output never affects state -- No mutation occurs during clarification -- Administrative state replacement clears pending clarification state -- Facts are exclusive -- Policies are additive -- Pending clarification blocks mutation - ---- - -## Implementation Status +- State changes only through explicit user directives or confirmation. +- Identical input sequences produce identical compiler state. +- Model responses never modify compiler state. +- Ambiguous directives trigger clarification instead of changing state. -The current implementation provides the deterministic directive compiler -defined in [this document](/docs/M1Design.md). +These invariants are verified through behavioral tests and Hypothesis-based property tests. --- -## Future Work - -This project intentionally focuses on the deterministic directive compiler itself. - -Higher-level systems such as session scope management, memory layers, agent tooling, -or context routing belong in host applications or separate projects built on top of -the compiler. - ---- - -## Design Philosophy - -The Context Compiler deliberately avoids: +## Design Notes -- semantic reasoning -- ontology inference -- machine-learning parsing -- transcript reconstruction -- passive memory - -Authoritative state must originate only from **explicit user directives**. - ---- - -## Specification - -The authoritative behavioral specification is: - -```text -M1Design.md -``` +More detailed design and milestone documents are available in: -Future milestone documents describe potential extensions but are not normative for current behavior. +- [Project overview](docs/DescriptionAndMilestones.md) +- [M1 design document](docs/M1Design.md) --- diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..dcb6c6b --- /dev/null +++ b/docs/README.md @@ -0,0 +1,6 @@ +# Documentation + +This directory contains the project design and milestone specifications. + +- `DescriptionAndMilestones.md` — project overview and milestone roadmap +- `M1Design.md` — authoritative behavioral specification for the deterministic directive engine diff --git a/examples/README.md b/examples/README.md new file mode 100644 index 0000000..42c7934 --- /dev/null +++ b/examples/README.md @@ -0,0 +1,28 @@ +# Examples + +This directory contains small integration examples showing typical host-side usage of the Context Compiler. + +## 01_persistent_guardrails.py + +Demonstrates how a prohibition persists as authoritative state across later turns. +Shows the host using compiled state to keep constraints active. + +## 02_configuration_and_correction.py + +Demonstrates deterministic fact replacement with explicit correction. +Shows last-write-wins behavior for `facts.focus.primary`. + +## 03_ambiguity_with_clarification.py + +Demonstrates ambiguity detection before state mutation. +Shows how the host handles `Decision.kind == "clarify"` and resumes after confirmation. + +## 04_tool_governance_denylist.py + +Demonstrates tool-governance policy handling via prohibition directives. +Shows how hosts can prevent denied tools from being selected. + +## 05_llm_integration_pattern.py + +Demonstrates the end-to-end host control flow around `Decision`. +Shows when to clarify, when to call the model, and how to include compiled state in prompts. diff --git a/pyproject.toml b/pyproject.toml index d20ae2c..333cb2b 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,13 +4,32 @@ build-backend = "hatchling.build" [project] name = "context-compiler" -version = "0.1.0" +version = "0.2.0" description = "Deterministic conversational state engine for LLM applications." readme = "README.md" requires-python = ">=3.11" license = { text = "Apache-2.0" } authors = [{ name = "Robert Lippmann" }] dependencies = [] +keywords = [ + "llm", + "conversation-state", + "state-machine", + "ai-infrastructure", +] +classifiers = [ + "Development Status :: 4 - Beta", + "Intended Audience :: Developers", + "License :: OSI Approved :: Apache Software License", + "Programming Language :: Python :: 3", + "Topic :: Software Development :: Libraries", + "Topic :: Scientific/Engineering :: Artificial Intelligence", +] + +[project.urls] +Homepage = "https://github.com/rlippmann/context-compiler" +Repository = "https://github.com/rlippmann/context-compiler" +Issues = "https://github.com/rlippmann/context-compiler/issues" [project.scripts] context-compiler = "context_compiler.repl:main" diff --git a/src/context_compiler/__init__.py b/src/context_compiler/__init__.py index d4268bf..0946943 100644 --- a/src/context_compiler/__init__.py +++ b/src/context_compiler/__init__.py @@ -1,3 +1,7 @@ +from importlib.metadata import version + from .engine import Decision, Engine, State, create_engine +__version__ = version("context-compiler") + __all__ = ["Decision", "Engine", "State", "create_engine"] diff --git a/uv.lock b/uv.lock index 51ca4e7..c0d361f 100644 --- a/uv.lock +++ b/uv.lock @@ -53,7 +53,7 @@ wheels = [ [[package]] name = "context-compiler" -version = "0.1.0" +version = "0.2.0" source = { editable = "." } [package.optional-dependencies]