Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
690640c
test(config): enforce reference wiring contract
Q00 Aug 9, 2026
8045749
fix(config): preserve runtime read provenance
Q00 Aug 10, 2026
6a4ff8f
fix(config): track structured runtime provenance
Q00 Aug 10, 2026
689d68d
fix(config): model standard mapping consumers
Q00 Aug 10, 2026
cb8122e
fix(config): propagate mapping mutations across aliases
Q00 Aug 10, 2026
ac1e568
fix(config): preserve dynamic mapping key collisions
Q00 Aug 10, 2026
9895bb2
fix(config): strongly update conditional receivers
Q00 Aug 10, 2026
2eb483d
fix(config): respect runtime reachability in contract scan
Q00 Aug 10, 2026
eb6a9f5
fix(config): harden local call provenance
Q00 Aug 10, 2026
51f57db
fix(config): join callable control-flow bindings
Q00 Aug 10, 2026
e5e33ae
fix(config): preserve abrupt control-flow bindings
Q00 Aug 10, 2026
0e8f018
fix(config): distinguish abrupt exception paths
Q00 Aug 10, 2026
92714e8
fix(config): consume ordered exception handlers
Q00 Aug 10, 2026
a113994
fix(config): subtract prior handler coverage
Q00 Aug 10, 2026
45c1016
fix(config): preserve caught exception domains
Q00 Aug 10, 2026
9a4bc03
fix(config): trace invoked callable boundaries
Q00 Aug 10, 2026
c856f60
test(config): cover callable resolution adversaries
Q00 Aug 10, 2026
36f3aaf
fix(config): preserve callable object provenance
Q00 Aug 10, 2026
7205152
fix(config): close reflective scanner gaps
Q00 Aug 10, 2026
3476dfd
fix(config): preserve exact runtime read provenance
Q00 Aug 15, 2026
74d187d
fix(config): track builtin accessor aliases
Q00 Aug 15, 2026
32f9ed6
fix(config): evaluate safe constant expressions
Q00 Aug 15, 2026
06797ab
fix(config): ignore erased reader definitions
Q00 Aug 15, 2026
5441c1a
Merge remote-tracking branch 'origin/main' into codex/issue1998-confi…
Q00 Aug 15, 2026
f54c0bc
fix(contract): model deferred and scope-bound reads
Q00 Aug 15, 2026
abad3fc
fix(contract): track generators and closure bindings
Q00 Aug 15, 2026
879ff9c
fix(ci): resolve generator consumers by identity
Q00 Aug 15, 2026
aa500ed
fix(ci): model lazy and async generator consumers
Q00 Aug 15, 2026
428fbe7
fix(ci): model deferred generator consumption
Q00 Aug 15, 2026
5ec1066
fix(ci): model short-circuit generator advancement
Q00 Aug 15, 2026
b0c466f
fix(ci): persist generator continuation state
Q00 Aug 15, 2026
dac18d8
fix(config): model deferred coroutine and iterator consumers
Q00 Aug 15, 2026
164c6cc
fix(config): model callback and coroutine consumers
Q00 Aug 15, 2026
96a8ad3
fix(config): preserve partial and serializer provenance
Q00 Aug 15, 2026
79c782c
fix(config): model mapping and context boundaries
Q00 Aug 15, 2026
237a980
fix(config-audit): model stdlib execution protocols
Q00 Aug 16, 2026
d34b21e
fix(config-audit): cover mapping and callable consumers
Q00 Aug 16, 2026
3e1dd69
fix(config): track ordinary context entry provenance
Q00 Aug 16, 2026
7934bf1
fix(config): track alternate executable access paths
Q00 Aug 16, 2026
182cf6d
fix(config): preserve transformed section provenance
Q00 Aug 16, 2026
82b6cd7
fix(config): close serialization and reachability gaps
Q00 Aug 16, 2026
c68baee
fix(config): preserve accessor provenance and reachability
Q00 Aug 16, 2026
490c0b0
fix(config): model filtered dumps and callbacks
Q00 Aug 16, 2026
31a9c95
fix(config): preserve decorator and partial provenance
Q00 Aug 16, 2026
9bfc1cc
fix(config): close remaining scanner provenance gaps
Q00 Aug 16, 2026
cb97a5d
fix(config): resolve builtin scanner identities
Q00 Aug 16, 2026
73e180d
fix(config): preserve descriptor and callback identity
Q00 Aug 16, 2026
fdf1595
fix(config): model iterator and exit callback reachability
Q00 Aug 16, 2026
1c02cd8
fix(ci): preserve accessor callback provenance
Q00 Aug 16, 2026
9c2d260
fix(ci): model iterator consumption and local raises
Q00 Aug 16, 2026
7d1d2ae
fix(ci): follow local iterable and conditional exits
Q00 Aug 16, 2026
8087c89
fix(ci): model iterator and context cleanup protocols
Q00 Aug 16, 2026
1e5d2cb
fix(ci): resolve imported data provenance
Q00 Aug 16, 2026
f0bc1e2
fix(config-audit): resolve qualified imported data
Q00 Aug 16, 2026
82fb571
fix(config-audit): resolve serialized mapping key aliases
Q00 Aug 16, 2026
56c8b5c
fix(config): resolve class and starred accessor reads
Q00 Aug 16, 2026
18d9924
test(config): cover provenance overreach
Q00 Aug 16, 2026
125b546
fix(config): evaluate final imported bindings
Q00 Aug 16, 2026
7637315
fix: close config scanner soundness gaps
Q00 Aug 16, 2026
fb17d2f
fix: model implicit config protocols
Q00 Aug 16, 2026
c80c44f
fix: close implicit scanner dispatch gaps
Q00 Aug 16, 2026
8328af7
fix(audit): model implicit Python dispatch precisely
Q00 Aug 16, 2026
cc6dbe0
fix(audit): cover truth and protocol dispatch
Q00 Aug 16, 2026
7722a0e
fix(config): model subscription mutation and sys exit
Q00 Aug 16, 2026
b1d710d
fix(config): complete protocol fallback dispatch
Q00 Aug 16, 2026
dde0ffd
fix(contract): model protocol and context control flow
Q00 Aug 16, 2026
df6cd98
fix(contract): execute initializer and class hooks
Q00 Aug 16, 2026
4788286
fix(contract): execute invoked private config readers
Q00 Aug 16, 2026
4e9aa19
fix(contract): execute public config call paths
Q00 Aug 16, 2026
8743bf4
fix(contract): model annotation execution scopes
Q00 Aug 16, 2026
904c68f
fix(contract): enforce decorator semantics
Q00 Aug 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,14 @@ jobs:

- name: Run ruff
run: |
uv run ruff check src/ tests/
uv run ruff check src/ tests/ scripts/check-config-reference-contract.py

- name: Run ruff format check
run: |
uv run ruff format --check src/ tests/
uv run ruff format --check src/ tests/ scripts/check-config-reference-contract.py

- name: Verify config reference contract
run: uv run python scripts/check-config-reference-contract.py

mypy:
name: MyPy Type Check
Expand Down
26 changes: 18 additions & 8 deletions docs/config-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -442,14 +442,20 @@ evaluation:

| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `stage1_enabled` | `bool` | `true` | **Currently inert in `config.yaml`.** Runtime builders do not copy this field into `PipelineConfig`. |
| `stage2_enabled` | `bool` | `true` | **Currently inert in `config.yaml`.** Runtime builders do not copy this field into `PipelineConfig`. |
| `stage3_enabled` | `bool` | `true` | **Currently inert in `config.yaml`.** Runtime builders do not copy this field into `PipelineConfig`. |
| `satisfaction_threshold` | `float [0.0, 1.0]` | `0.8` | **Currently inert.** The field is validated but the pipeline compares Stage 2 scores against a hardcoded `0.8`; changing this value does not change the gate. See [Evaluation Pipeline Guide](./guides/evaluation-pipeline.md#stage-2-semantic-evaluation). |
| `uncertainty_threshold` | `float [0.0, 1.0]` | `0.3` | **Currently inert in `config.yaml`.** Runtime builders do not copy it into `TriggerConfig`. |
| `stage1_enabled` | `bool` | `true` | **Currently inert in `config.yaml`.** Runtime builders do not copy this field into `PipelineConfig`. **Effective control:** direct-Python `PipelineConfig.stage1_enabled`. |
| `stage2_enabled` | `bool` | `true` | **Currently inert in `config.yaml`.** Runtime builders do not copy this field into `PipelineConfig`. **Effective control:** direct-Python `PipelineConfig.stage2_enabled`. |
| `stage3_enabled` | `bool` | `true` | **Currently inert in `config.yaml`.** Runtime builders do not copy this field into `PipelineConfig`. **Effective control:** direct-Python `PipelineConfig.stage3_enabled`. |
| `satisfaction_threshold` | `float [0.0, 1.0]` | `0.8` | **Currently inert.** The field is validated but the pipeline compares Stage 2 scores against a hardcoded `0.8`; changing this value does not change the gate. **Effective control:** the hardcoded `0.8` comparison. See [Evaluation Pipeline Guide](./guides/evaluation-pipeline.md#stage-2-semantic-evaluation). |
| `uncertainty_threshold` | `float [0.0, 1.0]` | `0.3` | **Currently inert in `config.yaml`.** Runtime builders do not copy it into `TriggerConfig`. **Effective control:** direct-Python `TriggerConfig.uncertainty_threshold`. |
| `semantic_model` | `string` | `"claude-opus-4-8"` | Model used for Stage 2 semantic evaluation. Overridable via `OUROBOROS_SEMANTIC_MODEL`. |
| `assertion_extraction_model` | `string` | `"claude-sonnet-4-6"` | Model used for extracting verification assertions from seed criteria. Overridable via `OUROBOROS_ASSERTION_EXTRACTION_MODEL`. |

<!-- config-field-contract: {"section":"evaluation","field":"stage1_enabled","status":"inert","effective_control":"PipelineConfig.stage1_enabled"} -->
<!-- config-field-contract: {"section":"evaluation","field":"stage2_enabled","status":"inert","effective_control":"PipelineConfig.stage2_enabled"} -->
<!-- config-field-contract: {"section":"evaluation","field":"stage3_enabled","status":"inert","effective_control":"PipelineConfig.stage3_enabled"} -->
<!-- config-field-contract: {"section":"evaluation","field":"satisfaction_threshold","status":"inert","effective_control":"hardcoded `0.8` comparison"} -->
<!-- config-field-contract: {"section":"evaluation","field":"uncertainty_threshold","status":"inert","effective_control":"TriggerConfig.uncertainty_threshold"} -->

> **Configuration boundary:** the top-level `evaluation.stage1_enabled`, `stage2_enabled`, `stage3_enabled`, and `uncertainty_threshold` keys are schema-validated placeholders, not runtime controls. The similarly named direct-Python `PipelineConfig.stage*_enabled` fields and `TriggerConfig.uncertainty_threshold` are separate and active when explicitly supplied to `EvaluationPipeline`; see [Disabling Stages](./guides/evaluation-pipeline.md#disabling-stages) and [Trigger Configuration](./guides/evaluation-pipeline.md#trigger-configuration).

---
Expand Down Expand Up @@ -482,14 +488,18 @@ consensus:

| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `min_models` | `int >= 2` | `3` | **Currently inert.** After reviewer-independence filtering, simple consensus separately requires at least two successfully collected votes; this top-level field is not wired to that rule. |
| `threshold` | `float [0.0, 1.0]` | `0.67` | **Currently inert.** Runtime simple consensus compares approvals divided by successful post-filter votes with direct-Python `ConsensusConfig.majority_threshold` (default `0.66`); this top-level field is not copied into it. |
| `diversity_required` | `bool` | `true` | **Currently inert.** The field exists on `ConsensusConfig` and in the schema, but nothing reads it. Provider diversity depends on actual adapter routing; neither this flag nor differently named roster entries attest it. See [Evaluation Pipeline Guide](./guides/evaluation-pipeline.md#stage-3-consensus-multi-model-or-single-model-fallback). |
| `min_models` | `int >= 2` | `3` | **Currently inert.** After reviewer-independence filtering, simple consensus separately requires at least two successfully collected votes; this top-level field is not wired to that rule. **Effective control:** the hardcoded minimum of two successfully collected post-filter votes. |
| `threshold` | `float [0.0, 1.0]` | `0.67` | **Currently inert.** Runtime simple consensus compares approvals divided by successful post-filter votes with direct-Python `ConsensusConfig.majority_threshold` (default `0.66`); this top-level field is not copied into it. **Effective control:** `ConsensusConfig.majority_threshold`. |
| `diversity_required` | `bool` | `true` | **Currently inert.** The field exists on `ConsensusConfig` and in the schema, but nothing reads it. Provider diversity depends on actual adapter routing; neither this flag nor differently named roster entries attest it. **Effective control:** actual adapter routing and reviewer-independence filtering. See [Evaluation Pipeline Guide](./guides/evaluation-pipeline.md#stage-3-consensus-multi-model-or-single-model-fallback). |
| `models` | `list[string]` | (see above) | Model roster for Stage 3 simple voting. With `llm.backend: litellm`, use `provider/model` or `openrouter/provider/model`. With `llm.backend: codex`, use Codex/OpenAI model IDs such as `gpt-5.4`. Overridable via `OUROBOROS_CONSENSUS_MODELS` (comma-separated). |
| `advocate_model` | `string` | `"openrouter/anthropic/claude-opus-4.8"` | Model that argues in favor of the proposed solution in deliberative consensus. With `llm.backend: codex`, this can be a Codex/OpenAI model ID such as `gpt-5.4`. Overridable via `OUROBOROS_CONSENSUS_ADVOCATE_MODEL`. |
| `devil_model` | `string` | `"openrouter/openai/gpt-4o"` | Model that argues against (devil's advocate) in deliberative consensus. With `llm.backend: codex`, this can be a Codex/OpenAI model ID such as `gpt-5.4`. Overridable via `OUROBOROS_CONSENSUS_DEVIL_MODEL`. |
| `judge_model` | `string` | `"openrouter/google/gemini-2.5-pro"` | Model that renders a final verdict after deliberation. With `llm.backend: codex`, this can be a Codex/OpenAI model ID such as `gpt-5.4`. Overridable via `OUROBOROS_CONSENSUS_JUDGE_MODEL`. |

<!-- config-field-contract: {"section":"consensus","field":"min_models","status":"inert","effective_control":"hardcoded minimum of two"} -->
<!-- config-field-contract: {"section":"consensus","field":"threshold","status":"inert","effective_control":"ConsensusConfig.majority_threshold"} -->
<!-- config-field-contract: {"section":"consensus","field":"diversity_required","status":"inert","effective_control":"actual adapter routing and reviewer-independence filtering"} -->

> **Configuration boundary:** `consensus.min_models` and `consensus.threshold` are schema-validated placeholders. Runtime simple consensus hardcodes a minimum of two successful post-filter votes and reads the separate direct-Python `ConsensusConfig.majority_threshold`. Changing these YAML keys does not change either rule.
>
> **Backend note:** With `llm.backend: litellm`, consensus models typically go through OpenRouter/LiteLLM and require the corresponding provider credentials (commonly `OPENROUTER_API_KEY`). With `llm.backend: codex`, the configured model strings are sent through Codex CLI instead.
Expand Down
Loading