Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
518 changes: 518 additions & 0 deletions contract_tests/test_audit.py

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions contract_tests/test_claiming.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@
"github.pull-request.comments.read",
"github.pull-request.reviews.read",
"github.pull-request.checks.read",
"github.repository.required-checks.read",
"github.pull-request.threads.read",
)

Expand Down
7 changes: 7 additions & 0 deletions contract_tests/test_host_boundary.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
"github.pull-request.comments.read",
"github.pull-request.reviews.read",
"github.pull-request.checks.read",
"github.repository.required-checks.read",
"github.pull-request.threads.read",
]
HEAD_SHA = "a" * 40
Expand Down Expand Up @@ -149,12 +150,17 @@ def complete_observation() -> dict[str, object]:
"pull_request_number": 785,
"kind": "CHECK_RUN",
"name": "test",
"integration_id": 42,
"status": "COMPLETED",
"conclusion": "SUCCESS",
"candidate_sha": HEAD_SHA,
"details_url": "https://github.com/shaug/atelier/actions/runs/1",
}
],
"required_checks": {
"configuration_read": True,
"contexts": [{"name": "test", "integration_id": 42}],
},
"threads": [
{
"id": "thread-1",
Expand All @@ -175,6 +181,7 @@ def complete_observation() -> dict[str, object]:
"pull_request_comments": True,
"reviews": True,
"checks": True,
"required_checks": True,
"threads": True,
},
}
Expand Down
12 changes: 12 additions & 0 deletions contract_tests/test_mailbox.py
Original file line number Diff line number Diff line change
Expand Up @@ -413,6 +413,18 @@ def add_work(
"policy_commit": SHA_A,
"candidate_revision": SHA_B,
"evidence": {name: "satisfied" for name in EVIDENCE},
"audit_evidence": {
"schema": "atelier.audit-evidence/v1",
"review": {
"mechanism": "review-code-change",
"verdict": "clean",
"candidate_revision": SHA_B,
"comparison_base_revision": SHA_A,
"observed_at": TIMESTAMP,
"findings": [],
},
"feedback_dispositions": [],
},
}
self.works[work_id] = work
self.write_work(work_id)
Expand Down
2 changes: 2 additions & 0 deletions contract_tests/test_planning.py
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,7 @@ def observation(*, blocked: bool = False, with_pull_request: bool = False) -> di
"pull_request_comments": [],
"reviews": [],
"checks": [],
"required_checks": {"configuration_read": True, "contexts": []},
"threads": [],
"completeness": {
"issue": True,
Expand All @@ -135,6 +136,7 @@ def observation(*, blocked: bool = False, with_pull_request: bool = False) -> di
"pull_request_comments": True,
"reviews": True,
"checks": True,
"required_checks": True,
"threads": True,
},
}
Expand Down
19 changes: 15 additions & 4 deletions docs/git-mailbox-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -696,12 +696,23 @@ acceptance:
required-validation-reported: satisfied
independent-review-current: satisfied
unresolved-feedback-zero: satisfied
audit_evidence:
schema: atelier.audit-evidence/v1
review:
mechanism: review-code-change
verdict: clean
candidate_revision: 2222222222222222222222222222222222222222
comparison_base_revision: 1111111111111111111111111111111111111111
observed_at: 2026-07-25T13:14:30Z
findings: []
feedback_dispositions: []
```

Acceptance records what was verified at that commit. If evidence later becomes
unavailable, stale, or contradictory, history is not rewritten. Audit reports
the current promise as `unknown`, `stale`, or `violated` and cites the earlier
acceptance.
Acceptance records what was verified at that commit, including the exact structured
review findings and explicit dispositions for top-level review and comment bodies.
If evidence later becomes unavailable, stale, or contradictory, history is not
rewritten. Audit reports the current promise as `unknown`, `stale`, or `violated`
and cites the earlier acceptance.

## Minimal planner-worker interactions

Expand Down
62 changes: 39 additions & 23 deletions docs/project-policy-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -244,26 +244,36 @@ Each predicate has one authoritative source and exact evaluation.
### `pull-request-open`

- **Source:** live GitHub pull-request state.
- **Satisfied:** the PR is open.
- **Violated:** the PR is closed or merged.
- **Satisfied:** the PR is open and not a draft.
- **Violated:** the PR is closed, merged, or still a draft.
- **Unknown:** GitHub cannot be read.
- **Stale:** not applicable.

### `pull-request-mergeable`

- **Source:** live GitHub mergeability.
- **Satisfied:** GitHub reports the PR mergeable.
- **Violated:** GitHub reports a conflict.
- **Unknown:** mergeability is unavailable or still being calculated.
- **Stale:** the PR head changed after the observation.
- **Source:** live GitHub conflict and merge-state status.
- **Satisfied:** GitHub reports the exact head mergeable and its current merge state
does not block readiness. `UNSTABLE` does not fail this predicate by itself;
configured required checks are evaluated separately.
- **Violated:** GitHub reports a conflict or a policy-blocked merge state.
- **Unknown:** conflict or merge-state status is unavailable or still being
calculated.
- **Stale:** the PR head or exact comparison base changed after the observation.

### `required-checks-pass`

- **Source:** live required GitHub checks for the exact PR head.
- **Satisfied:** every required check completed successfully.
- **Violated:** a required check completed unsuccessfully.
- **Unknown:** required-check configuration or a required result cannot be read.
- **Stale:** the PR head changed after the check result.
- **Source:** live required-check configuration plus check results for the exact PR
head. Required identities are the exact context name plus optional GitHub App or
ruleset integration identity obtained from effective branch protection and
repository rulesets; observed results retain their check/status kind and
integration identity.
- **Satisfied:** the required configuration was read and every named required
check from its configured provider completed successfully; an empty configured
set is satisfied.
- **Violated:** a named required check completed unsuccessfully.
- **Unknown:** required-check configuration or a named required result from its
configured provider cannot be read, is missing, or is ambiguous.
- **Stale:** a named required result belongs to another PR head.

### `required-validation-reported`

Expand All @@ -276,20 +286,26 @@ Each predicate has one authoritative source and exact evaluation.

### `independent-review-current`

- **Source:** a structured `review-code-change` result recorded in the receipt.
- **Satisfied:** the result is clean and bound to the delivered SHA and
comparison base.
- **Violated:** the result requires changes.
- **Unknown:** the result is missing, malformed, or blocked.
- **Stale:** the delivered head or effective comparison base changed.
- **Source:** the delivered receipt plus normalized aggregate
`review-code-change` audit evidence with structured finding dispositions.
- **Satisfied:** both records are clean, every finding has a non-unresolved
disposition, and both are bound to the delivered SHA and comparison base.
- **Violated:** the result requires changes or retains an unresolved finding.
- **Unknown:** either record is missing, malformed, blocked, or contradictory.
- **Stale:** the delivered head, live PR base, or effective comparison base changed.

### `unresolved-feedback-zero`

- **Source:** live GitHub review, comment, and thread state.
- **Satisfied:** no unresolved material review, comment, or thread remains.
- **Violated:** an unresolved material item remains.
- **Unknown:** thread-aware feedback cannot be read.
- **Stale:** the PR head changed after a previously clean observation.
- **Source:** live GitHub review, comment, and thread state plus the normalized
audit-evidence dispositions bound to provider IDs and exact body digests.
- **Satisfied:** every nonempty top-level review or comment body has an explicit
non-unresolved disposition and no unresolved material thread or receipt
obligation remains. This predicate is an unconditional v0 acceptance guard,
even when an older approval omitted it from `required_evidence`.
- **Violated:** an item is explicitly unresolved, the live review decision requires
changes, or a material thread or receipt obligation remains unresolved.
- **Unknown:** thread-aware feedback cannot be read or a live body lacks a disposition.
- **Stale:** a disposition identifies missing feedback or an earlier body revision.

## Policy drift

Expand Down
14 changes: 10 additions & 4 deletions scripts/validate_repository.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,11 +82,17 @@ def validate_skill(errors: list[str]) -> None:
errors.append("Atelier skill frontmatter must include a description")
if "[TODO:" in text:
errors.append("Atelier skill contains a TODO placeholder")
if "Production `audit` behavior is not implemented yet" not in text:
errors.append("skill must state that audit is unavailable")
for relative in ("references/delegation.md", "scripts/delegation.py"):
if "Production `audit` behavior is implemented" not in text:
errors.append("skill must publish the audit boundary")
required = (
("delegated work", "references/delegation.md"),
("delegated work", "scripts/delegation.py"),
("audit", "references/audit.md"),
("audit", "scripts/audit.py"),
)
for boundary, relative in required:
if not (path.parent / relative).is_file():
errors.append(f"delegated work boundary missing: {relative}")
errors.append(f"{boundary} boundary missing: {relative}")


def validate_reset_boundary(errors: list[str]) -> None:
Expand Down
24 changes: 14 additions & 10 deletions skills/atelier/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ The preflight must prove:
the host capability descriptor;
- the installed and authorized `github@openai-curated` connector;
- every required read-only issue, relationship, pull-request, comment, review,
check, and thread operation; and
check, effective required-check configuration, and thread operation; and
- one complete observation conforming to
`references/github-observation.schema.json` when live state is requested.

Expand Down Expand Up @@ -61,15 +61,19 @@ result. Launch one fresh worker through the host with the installed
substitute CLI process, cache provider observations, or widen Atelier's v0
authority ceiling.

Production `audit` behavior is not implemented yet. When it is requested:

1. Report that audit is unavailable in the reset scaffold.
1. Link the owning issue: shaug/atelier#780.
1. Make no mailbox, repository, ticket, pull-request, or acceptance mutation.

An explicit host-readiness request may complete the read-only host preflight and
return its exact compatibility or failure result. That does not make audit
available.
Production `audit` behavior is implemented for one delivered or accepted
assignment. Before audit, read `references/audit.md`, repeat the fail-closed host
preflight, obtain one fresh complete GitHub observation, normalize the exact
aggregate review and live feedback dispositions as `atelier.audit-evidence/v1`,
and use `scripts/audit.py audit`. Preserve every predicate verdict, structured
finding, and visible feedback disposition in the report; do not infer success
from unavailable, undispositioned, or stale state.

Audit itself is read-only. Only after the operator explicitly confirms the
report's exact acceptance fence may a strictly newer complete provider snapshot
be supplied to `scripts/audit.py accept` for the one acceptance commit. Reusing
report evidence is rejected. Acceptance never implies merge, deployment,
native-ticket mutation, or issue closure.

## Mailbox boundary

Expand Down
70 changes: 70 additions & 0 deletions skills/atelier/references/audit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Audit and acceptance

Use this boundary only for one validated mailbox assignment whose current state
is `delivered` or `accepted`.

## Live audit

1. Complete the host preflight in `host-boundary.md`.
2. Read the canonical mailbox through `GitMailboxWriter.observe`; do not reuse a
prior clone or write an audit projection.
3. Generate one complete `atelier.github-observation/v1` snapshot at the live
read boundary. It must include the exact issue relationships, pull request,
comments, reviews, checks, effective required-check configuration, and review
threads. A failed ruleset or branch-protection read is represented by
`required_checks.configuration_read: false`; it is never inferred from the
visible check list.
4. Normalize the exact aggregate `review-code-change` result and every deliberate
disposition for a nonempty top-level review or pull-request comment into an
`atelier.audit-evidence/v1` JSON object. Each review finding records an ID,
summary, disposition, rationale, and optional follow-up. Each live review or
comment disposition records its kind, provider ID, exact `sha256:` body digest,
disposition, rationale, and optional follow-up. Omitted, duplicate, foreign,
or body-stale identities fail closed.
5. Invoke `scripts/audit.py audit` with the mailbox remote and branch, work ID,
policy target, host target, observation path and lower bound, and
`--audit-evidence` path.
6. Present the complete report. Every registry predicate is classified as
`satisfied`, `violated`, `unknown`, or `stale`; the report also preserves the
structured review findings, receipt obligations, and explicit disposition and
body of every review, pull-request comment, and thread.

The report reconstructs the exact delivered receipt, remote candidate, pull
request, head and comparison base, approved and current policy, native ticket
observation, required validation, independent `review-code-change` result, and
current feedback. An accepted assignment additionally derives the Git commit
that introduced acceptance and the preceding delivery observation. Later drift
changes the current audit verdict; it never rewrites historical acceptance.

Audit is read-only. Missing host capability or project-policy identity reports
`authority-unreconstructable`. Missing live evidence reports `unknown`.
Contradictory state reports `violated`; changed candidate-bound state reports
`stale`.

## Explicit acceptance

Acceptance is a second operation. Do not infer it from a clean audit.

1. Show the operator the report and its `acceptance_fence`.
2. Require explicit confirmation of that exact fence.
3. Start a new provider read only after the confirmed report's `observed_at`.
Produce a second complete observation whose live-read lower bound and
`observed_at` are both strictly later. Reusing the report snapshot is rejected.
4. Invoke `scripts/audit.py accept` with that second observation and lower bound,
the same normalized audit-evidence path, the complete fence, `--accepted-at`,
and `--confirm`.
5. The transition rereads the canonical mailbox, host boundary, policy, ticket,
candidate, pull request base and head, required-check configuration and
results, reviews, comments, and threads. It requires the fence's mailbox,
receipt, candidate, and semantic evidence digest to remain exact while the
observation timestamp advances. Changed facts are rejected as stale or
violated; only the observation timestamp may differ.
6. Only when every currently required predicate and the unconditional
no-unresolved-feedback guard are `satisfied` does one verified mailbox commit
move `delivered` to `accepted`, clear the claim, and bind the operator record
to the exact receipt, candidate, audit evidence, and satisfied verdicts. The
accepted timestamp is always checked against the current UTC clock, even when
no test clock is supplied.

Acceptance does not merge a pull request, deploy, mutate or close a native
ticket, delete a branch, or accept a parent initiative.
Loading
Loading