What happened
On PR #1745, the fullsend review agent made several findings that revealed a misunderstanding of Rego's declarative/Datalog semantics:
- "Return values" language: The agent described Rego rules as having "return values" and suggested testing through a "public API" — concepts that don't apply in Rego. Rules evaluate to true/false via logical conjunction; there is no call/return mechanism.
- Integration test insistence (5 iterations): The agent raised "no integration test through
is_registry_dependency" in every review cycle (Jun 11, Jun 15 x3, Jun 16). The author repeatedly explained that in Rego, testing individual conjunction terms separately IS sufficient because rules compose declaratively. The agent never adapted.
- Missed idiomatic improvements: The human reviewer (simonbaird) provided concrete Rego-idiomatic suggestions (
some q in qualifiers, object.get, parsed_purl.version in {null, ""}) that significantly simplified the code. The bot never suggested these patterns despite having the code context.
The current AGENTS.md covers build commands, conventions, and architecture but includes no guidance about Rego's evaluation model or idiomatic testing patterns.
What could go better
The review agent's testing recommendations were calibrated for imperative languages where unit tests cover functions and integration tests cover APIs. In Rego, rules are logical propositions that compose via conjunction — testing each clause independently provides equivalent coverage to testing the composed rule. The agent's repeated insistence on integration tests was not just noise; it consumed author time across 5 dismissals and eroded trust in the bot's domain competence.
Confidence: High that adding Rego paradigm guidance would reduce these false positives. The agent already reads AGENTS.md for context. The pattern of the human reviewer providing better Rego-specific feedback confirms the gap is domain knowledge, not reasoning capability. Partially overlaps with fullsend-ai/fullsend#1609 (domain-specific semantics) but that issue targets Kubernetes RBAC specifically and proposes a platform-level fix, while this is repo-specific context that belongs in AGENTS.md.
Proposed change
Add a "Rego Language Model" or "How Rego Works" section to AGENTS.md covering:
- Evaluation model: Rego rules are declarative propositions, not functions. Multiple rule bodies for the same name are OR-ed; conditions within a body are AND-ed. There are no "return values" — rules evaluate to true/undefined.
- Testing idioms: Testing individual rule clauses separately is idiomatic and sufficient. Integration tests through higher-level rules are not required when each conjunction term has coverage. The repo enforces 100% coverage via
make test.
- Common patterns:
some x in collection, object.get(obj, key, default), set membership via x in {a, b, c}, and not for negation. Reviewers should suggest these patterns when they see verbose alternatives.
- What NOT to suggest: Don't recommend imperative patterns (early returns, try/catch, null guards for missing keys when the parser guarantees the key exists).
Example addition to AGENTS.md:
## Rego Evaluation Model (for AI reviewers)
Rego is a declarative policy language (Datalog-inspired), not imperative code:
- Multiple rule bodies with the same name are **disjunctions** (OR). Conditions within a body are **conjunctions** (AND).
- Rules evaluate to `true` or `undefined` — there are no "return values" or control flow.
- Testing each conjunction term independently is sufficient and idiomatic. Do not request integration tests through higher-level rules unless there is a specific composition bug.
- Prefer `some x in collection` over explicit iteration, `object.get(obj, key, default)` over key-existence checks, and `x in {a, b}` over chained equality.
Validation criteria
On the next 3-5 PRs to policy/lib/ files reviewed by fullsend-ai-review[bot]:
- The agent should not describe Rego rules as having "return values" or suggest imperative patterns.
- The agent should not request integration tests through higher-level rules when individual clause tests exist and coverage is 100%.
- The agent should suggest idiomatic Rego patterns (e.g.,
some x in collection, object.get) when it sees verbose alternatives.
Measurable: count false-positive test-adequacy findings on Rego PRs before vs. after the AGENTS.md update.
Generated by retro agent from #1745
What happened
On PR #1745, the fullsend review agent made several findings that revealed a misunderstanding of Rego's declarative/Datalog semantics:
is_registry_dependency" in every review cycle (Jun 11, Jun 15 x3, Jun 16). The author repeatedly explained that in Rego, testing individual conjunction terms separately IS sufficient because rules compose declaratively. The agent never adapted.some q in qualifiers,object.get,parsed_purl.version in {null, ""}) that significantly simplified the code. The bot never suggested these patterns despite having the code context.The current AGENTS.md covers build commands, conventions, and architecture but includes no guidance about Rego's evaluation model or idiomatic testing patterns.
What could go better
The review agent's testing recommendations were calibrated for imperative languages where unit tests cover functions and integration tests cover APIs. In Rego, rules are logical propositions that compose via conjunction — testing each clause independently provides equivalent coverage to testing the composed rule. The agent's repeated insistence on integration tests was not just noise; it consumed author time across 5 dismissals and eroded trust in the bot's domain competence.
Confidence: High that adding Rego paradigm guidance would reduce these false positives. The agent already reads AGENTS.md for context. The pattern of the human reviewer providing better Rego-specific feedback confirms the gap is domain knowledge, not reasoning capability. Partially overlaps with fullsend-ai/fullsend#1609 (domain-specific semantics) but that issue targets Kubernetes RBAC specifically and proposes a platform-level fix, while this is repo-specific context that belongs in AGENTS.md.
Proposed change
Add a "Rego Language Model" or "How Rego Works" section to
AGENTS.mdcovering:make test.some x in collection,object.get(obj, key, default), set membership viax in {a, b, c}, andnotfor negation. Reviewers should suggest these patterns when they see verbose alternatives.Example addition to AGENTS.md:
Validation criteria
On the next 3-5 PRs to
policy/lib/files reviewed by fullsend-ai-review[bot]:some x in collection,object.get) when it sees verbose alternatives.Measurable: count false-positive test-adequacy findings on Rego PRs before vs. after the AGENTS.md update.
Generated by retro agent from #1745