TASK-033: support deterministic CEG outcome replay - #179
Conversation
L9 Audit Harness Report
Step Results
Architecture Audit Findings
See Spec Coverage
See Next StepsAll checks passed. Safe to merge. |
|
📋 Best Practices for Large Changes
✅ This PR passes the blocking limit but is larger than recommended. |
|
There was a problem hiding this comment.
🟡 Not ready to approve
Input normalization currently has a determinism edge case and the new unit test file likely fails formatting/lint checks due to overlong lines.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.
Pull request overview
Adds an offline, deterministic “outcome replay” facility to the CEG engine so Odoo-exported fixtures can be reprocessed into stable per-event and aggregate hashes without invoking Gate/Neo4j/network paths.
Changes:
- Introduces
engine.replaypure functions to validate Odoo replay inputs and deterministically computeoutcome_hash/outcome_set_hash. - Adds a CLI tool (
tools/outcome_replay.py) to run the replay and emit a JSON artifact. - Adds unit tests plus an ADR and runbook documenting the workflow and guarantees.
File summaries
| File | Description |
|---|---|
| tools/outcome_replay.py | New CLI wrapper for running offline replay and writing deterministic JSON output. |
| engine/replay/outcome.py | Core deterministic normalization + hashing logic for replay inputs/outputs. |
| engine/replay/init.py | Exposes replay API surface (replay_outcomes, schemas, error type). |
| tests/unit/test_outcome_replay.py | Unit tests for determinism, schema rejection, and CLI behavior. |
| docs/runbooks/CEG_OUTCOME_REPLAY.md | Operator runbook for executing replay and expected guarantees. |
| docs/adr/ADR-111-ceg-outcome-replay.md | ADR capturing schema constraints and offline-only design decision. |
Review details
Suppressed comments (1)
tests/unit/test_outcome_replay.py:79
- This
spec_from_file_locationcall is very long and is likely to fail formatting checks; wrap it across lines for consistency with other tests (e.g.test_payload_contract_compiler.py).
spec = importlib.util.spec_from_file_location("ceg_outcome_replay_cli", ROOT / "tools" / "outcome_replay.py")
- Files reviewed: 8/8 changed files
- Comments generated: 3
- Review effort level: Lite
We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.
| doc = _fixture() | ||
| doc["events"] = list(reversed(doc["events"])) | ||
| assert replay_outcomes(doc)["outcome_set_hash"] == replay_outcomes(_fixture())["outcome_set_hash"] |
| if isinstance(raw.get("payload"), dict): | ||
| event["payload"] = raw["payload"] | ||
| normalized.append(event) | ||
| normalized.sort(key=lambda e: (e["packet_id"], e["action"], e["tenant"])) |
| if document.get("gate_mutation") is True: | ||
| raise ReplayError("gate_mutation=true is forbidden in replay mode") | ||
| events = document.get("events") | ||
| if not isinstance(events, list): | ||
| raise ReplayError("events must be a list") |
There was a problem hiding this comment.
🟡 Not ready to approve
Determinism and repo metadata conventions have gaps (notably outcome ordering under duplicate event keys and missing L9_META in new entrypoints) that should be fixed before approval.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.
Review details
Suppressed comments (5)
tools/outcome_replay.py:16
sys.path.insert(0, ...)runs at import time, which means importing this module in unit tests mutates global interpreter state and can affect import resolution order. Prefer making the insertion idempotent (at minimum) so repeated imports don’t keep reorderingsys.path.
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT))
engine/replay/outcome.py:103
- Event order independence is not fully guaranteed when there are multiple events with the same
(packet_id, action, tenant)(the current normalization sort key), because stable sorting will preserve input order among ties and the returnedoutcomeslist is hashed as-is. Sorting the computedoutcomesby a tie-free key (includingoutcome_hash) makesoutcome_set_hashinvariant to input ordering even under duplicates.
"packet_id": event["packet_id"],
"tenant": event["tenant"],
}
)
result: dict[str, Any] = {
"event_count": len(outcomes),
tests/unit/test_outcome_replay.py:60
- This assertion line is very long and is likely to be reformatted (or rejected) by the repo’s Ruff formatting/line-length conventions, which can cause avoidable CI churn. Splitting it improves readability and keeps formatting stable.
assert replay_outcomes(doc)["outcome_set_hash"] == replay_outcomes(_fixture())["outcome_set_hash"]
engine/replay/init.py:1
engine/subpackage__init__.pyfiles typically include an L9_META header (e.g.,engine/intake/__init__.py:1andengine/gates/__init__.py:1). This newengine/replay/__init__.pyis missing it, which can break contract/metadata tooling expectations.
"""Deterministic CEG outcome replay from Odoo/fixture inputs (TASK-033)."""
tools/outcome_replay.py:5
- Most
tools/*.pyentrypoints include an L9_META header (for exampletools/payload_contract_compiler.py:2andtools/contract_scanner.py:2). This new CLI tool doesn’t include it, which can cause it to be missed by metadata/contract scanners.
This issue also appears on line 14 of the same file.
"""CLI: deterministic CEG outcome replay from Odoo fixtures (TASK-033).
No Gate / Neo4j / network calls.
"""
- Files reviewed: 8/8 changed files
- Comments generated: 0 new
- Review effort level: Lite
We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.
Generated under L9 controlled autonomy.
Task: TASK-033
Program: sha256:9cd1a79f948dac419913c134396e58359e4df82862bb3901bdd327684a37cb52
Contract: sha256:f4d768907ad10db24c9398dd6eceaa073896eaf96210c24bdc6853458df78cfc
Verification: sha256:6539f1c0469c852d11c4ff83a5d978840660bb008d93adbdca2f72a363cc990e
This PR is draft only. The controller cannot mark ready, approve, merge, tag, release, or deploy.