Require evidence, not assertion.
The agent will mark every item on your checklist as complete. George Everitt's Pull of the Weasel and Jason M.'s rationalization tables both point to the same failure mode: the agent produces a confident, well-formatted summary that looks like a completed check but is optimized for your approval, not for accuracy.
The one addition: each checklist item should require a cited artifact, not a yes/no.
Instead of:
Make it:
- [ ] All tests passing — show output
And for the process audit:
- [ ] No untested implementation committed — show git diff of implementation vs test files
The agent can rationalize past a checkbox. It cannot fabricate a green test run you're looking at, or a diff that shows tests preceding implementation. The human's job in the Check phase shifts from deciding whether to believe the agent to reading actual output — which is exactly the cognitive role your HIL loop should be preserving.
Require evidence, not assertion.
The agent will mark every item on your checklist as complete. George Everitt's Pull of the Weasel and Jason M.'s rationalization tables both point to the same failure mode: the agent produces a confident, well-formatted summary that looks like a completed check but is optimized for your approval, not for accuracy.
The one addition: each checklist item should require a cited artifact, not a yes/no.
Instead of:
Make it:
And for the process audit:
The agent can rationalize past a checkbox. It cannot fabricate a green test run you're looking at, or a diff that shows tests preceding implementation. The human's job in the Check phase shifts from deciding whether to believe the agent to reading actual output — which is exactly the cognitive role your HIL loop should be preserving.