Finalize engine and state apis

.github/workflows/ci.yml

@@ -35,7 +35,7 @@ jobs:
         run: uv run ruff format --check .
 
       - name: Mypy
-        run: uv run mypy .
+        run: uv run mypy src
 
       - name: Install Hypothesis for tests
         run: uv pip install --python .venv/bin/python hypothesis

.pre-commit-config.yaml

@@ -10,6 +10,7 @@ repos:
     rev: v1.19.1
     hooks:
       - id: mypy
+        exclude: ^tests/
         args: [--pretty]
         additional_dependencies:
           - hypothesis

README.md

@@ -42,7 +42,7 @@ Later in the conversation:
 User: how should I deploy my service?
 ```
 
-The host supplies the compiled state to the model so the constraint persists across turns.
+The host supplies the authoritative state to the model so the constraint persists across turns.
 
 ---
 
@@ -140,19 +140,40 @@ The compiler maintains an authoritative state:
 }
 ```
 
-## State Export / Import
+## State Access and Persistence
 
-The compiler exposes its authoritative state so host applications can
-persist or restore it across sessions.
+The compiler exposes authoritative state through in-memory replacement APIs
+and JSON persistence/transport APIs.
 
-Example lifecycle:
+Supported host interaction model:
 
 ```python
-state = engine.export_state()
-save_to_storage(state)
+engine = create_engine()
+engine = create_engine(state=initial_state_obj)
+decision = engine.step(user_input)
+state = engine.state
+engine.state = replacement_state_obj
+payload = engine.export_json()
+engine.import_json(payload)
+```
+
+API boundaries:
+
+- `step()` performs directive-driven mutation.
+- `engine.state` / `engine.state = ...` are in-memory inspection/replacement APIs.
+- `export_json()` / `import_json()` are persistence/transport APIs.
+- No imperative convenience mutation methods are provided; operations such as
+  `reset policies` and `clear state` are handled through directive input to `step()`.
+
+Example persistence lifecycle:
 
-restored = load_from_storage()
-engine = create_engine(state=restored)
+```python
+payload = engine.export_json()
+save_to_storage(payload)
+
+restored_payload = load_from_storage()
+engine = create_engine()
+engine.import_json(restored_payload)
 ```
 
 The compiler does not manage storage or snapshots. Persistence policies
@@ -323,6 +344,7 @@ The compiler enforces several invariants:
 - The same input sequence always produces the same state
 - LLM output never affects state
 - No mutation occurs during clarification
+- Administrative state replacement clears pending clarification state
 - Facts are exclusive
 - Policies are additive
 - Pending clarification blocks mutation
@@ -344,8 +366,6 @@ Higher-level systems such as session scope management, memory layers, agent tool
 or context routing belong in host applications or separate projects built on top of
 the compiler.
 
-One possible future extension is cross-session persistence using exported state.
-
 ---
 
 ## Design Philosophy

docs/DescriptionAndMilestones.md

@@ -56,7 +56,7 @@ Richer fact schemas may be introduced in future milestones.
 - State data model (facts + policies)
 - Deterministic update rules (exclusive vs additive slots)
 - Clarification mechanism for ambiguous mutations
-- Context serialization interface (state → host application)
+- Context serialization interface (`export_json` / `import_json`, state → host application)
 - Reference integration harness (example host)
 - Tests: persistence and non-regression of corrections
 
@@ -67,7 +67,7 @@ After correcting or constraining the assistant once, the behavior remains consis
 ### M3 — Cross-Session Recall
 
 **Goal**
-Allow host applications to restore previously exported state safely and intentionally.
+Extend host-level workflows around persisted exported state safely and intentionally.
 
 **Core capability:**
 
@@ -77,7 +77,7 @@ Allow host applications to restore previously exported state safely and intentio
 
 **Deliverables:**
 
-- Import/Export API
+- Host-side storage/recovery patterns built on the existing import/export API
 
 **User-visible outcome:**
 

docs/M1Design.md

@@ -33,6 +33,8 @@ The compiler:
 4. Returns a deterministic `Decision`
 
 The compiler never calls the LLM.
+Mutations are expressed through directive input to `step()`, including reset/clear operations.
+Imperative convenience methods such as `reset_policies()` and `clear_state()` are not part of the public API.
 
 ### 3. Host Responsibilities
 
@@ -241,6 +243,12 @@ While pending exists, no other mutation may occur.
 Adding duplicate policy is a no-op.
 Policies stored in sorted lexical order.
 
+Administrative state replacement is also supported through public host APIs:
+- `engine.state = ...` (object replacement)
+- `engine.import_json(payload)` (JSON replacement)
+
+Both replacement paths clear pending clarification state and must behave like live state for subsequent `step()` calls.
+
 ### 10. Context Serialization Contract
 
 The compiler outputs structured state only. The host formats prompts.
@@ -254,6 +262,10 @@ Example:
     }
 `
 
+JSON persistence boundary:
+- `engine.export_json()` serializes authoritative state for transport/storage.
+- `engine.import_json(payload)` validates/canonicalizes payload and fully replaces active state.
+
 ### 11. Reset Commands
 
 Explicit only:

pyproject.toml

@@ -46,6 +46,7 @@ select = ["E", "F", "I", "UP", "B", "SIM"]
 [tool.mypy]
 python_version = "3.11"
 strict = true
+exclude = ["^tests/"]
 warn_unused_configs = true
 disallow_untyped_defs = true
 disallow_incomplete_defs = true

src/context_compiler/const.py

@@ -12,3 +12,7 @@
 
 # Policy keys
 POLICY_PROHIBIT: Final = "prohibit"
+
+# Event kinds
+EVENT_RESET_POLICIES: Final = "reset_policies"
+EVENT_CLEAR_STATE: Final = "clear_state"

src/context_compiler/engine.py

@@ -14,6 +14,8 @@
 from unicodedata import normalize as unicode_normalize
 
 from .const import (
+    EVENT_CLEAR_STATE,
+    EVENT_RESET_POLICIES,
     FOCUS_PRIMARY,
     POLICY_PROHIBIT,
     STATE_FACTS,
@@ -90,38 +92,64 @@ class NegativeDirectiveRule:
 )
 
 
-def create_engine() -> "Engine":
-    """Create a new deterministic Engine instance with initial M1 state."""
-    return Engine()
+def create_engine(state: State | None = None) -> "Engine":
+    """Create an Engine with initial state or validated replacement state.
 
+    When ``state`` is provided, it is validated and canonicalized before use.
+    """
+    return Engine(state=state)
 
-class Engine:
-    """Deterministic state engine implementing M1 directive semantics."""
 
-    def __init__(self) -> None:
-        self._state: State = _initial_state()
+class Engine:
+    """Deterministic state engine implementing M1 directive semantics.
+
+    Design note:
+    - ``step()`` is the mutation interface for directive-driven updates.
+    - Operations like "reset policies" and "clear state" are expressed as
+      directive input to ``step()``.
+    - Host code should not rely on imperative helpers such as
+      ``reset_policies()`` or ``clear_state()``.
+    - State may be administratively replaced via ``engine.state = ...`` and
+      ``engine.import_json(...)``.
+    """
+
+    def __init__(self, state: State | None = None) -> None:
+        """Initialize engine state from defaults or a supplied state object.
+
+        Supplied state is validated and canonicalized. Initialization performs
+        full state replacement and clears pending clarification state.
+        """
+        self._state: State
         self._pending: PendingEvent | None = None
         self._pending_prompt: str | None = None
         self._last_exclusive_fact_key: str | None = None
+        self._replace_state(_initial_state() if state is None else _load_state_obj(state))
 
     @property
     def state(self) -> State:
-        """Return a defensive copy of the current authoritative state snapshot."""
+        """Return a defensive copy of the current authoritative in-memory state."""
         return deepcopy(self._state)
 
+    @state.setter
+    def state(self, value: State) -> None:
+        """Replace authoritative in-memory state from a supplied object.
+
+        The supplied value is validated and canonicalized. Replacement is full,
+        and pending clarification state is cleared.
+        """
+        self._replace_state(_load_state_obj(value))
+
     def export_json(self) -> str:
-        """Serialize authoritative state to canonical JSON."""
+        """Serialize authoritative state for persistence or transport."""
         return json.dumps(self._state, sort_keys=True, separators=(",", ":"))
 
     def import_json(self, payload: str) -> None:
-        """Replace authoritative state from a JSON payload."""
-        state = _load_state_json(payload)
-        self._state = state
-        self._pending = None
-        self._pending_prompt = None
-        self._last_exclusive_fact_key = (
-            FOCUS_PRIMARY if state[STATE_FACTS][FOCUS_PRIMARY] is not None else None
-        )
+        """Load and replace authoritative state from a JSON payload.
+
+        Payload state is validated and canonicalized. Replacement is full, and
+        pending clarification state is cleared.
+        """
+        self._replace_state(_load_state_json(payload))
 
     def step(self, user_input: str) -> Decision:
         """Process one user input and return a deterministic Decision."""
@@ -144,10 +172,10 @@ def _classify(self, user_input: str) -> PendingEvent | Decision:
         normalized_message = _normalize_message(user_input)
 
         if normalized_message in _RESET_POLICY:
-            return PendingEvent(kind="reset_policies")
+            return PendingEvent(kind=EVENT_RESET_POLICIES)
 
         if normalized_message in _CLEAR_STATE:
-            return PendingEvent(kind="clear_state")
+            return PendingEvent(kind=EVENT_CLEAR_STATE)
 
         correction = _parse_correction(user_input)
         if correction is not None:
@@ -226,14 +254,22 @@ def _apply_event(self, event: PendingEvent) -> Decision:
         if event.kind == "fact_set":
             assert event.fact_value is not None
             return self._set_focus_primary(event.fact_value)
-        if event.kind == "reset_policies":
+        if event.kind == EVENT_RESET_POLICIES:
             self._state[STATE_POLICIES][POLICY_PROHIBIT] = []
             return _update_decision(self._state)
 
         self._state = _initial_state()
         self._last_exclusive_fact_key = None
         return _update_decision(self._state)
 
+    def _replace_state(self, state: State) -> None:
+        self._state = state
+        self._pending = None
+        self._pending_prompt = None
+        self._last_exclusive_fact_key = (
+            FOCUS_PRIMARY if state[STATE_FACTS][FOCUS_PRIMARY] is not None else None
+        )
+
     def _set_focus_primary(self, value: str) -> Decision:
         self._state[STATE_FACTS][FOCUS_PRIMARY] = _clean_fact_value(value)
         self._last_exclusive_fact_key = FOCUS_PRIMARY
@@ -268,8 +304,13 @@ def _load_state_json(payload: str) -> State:
     except json.JSONDecodeError as exc:
         raise ValueError("Invalid JSON payload.") from exc
 
+    return _load_state_obj(raw)
+
+
+def _load_state_obj(raw: object) -> State:
     if not isinstance(raw, dict):
         raise ValueError("Invalid state payload.")
+
     if set(raw.keys()) != {STATE_FACTS, STATE_POLICIES, STATE_VERSION}:
         raise ValueError("Invalid state payload.")