Skip to content

fix(daemon): lead the collaboration guidance with a tool-precedence rule (#800) - #801

Merged
Poytr1 merged 1 commit into
mainfrom
claude/tool-precedence-prompt
Aug 10, 2026
Merged

fix(daemon): lead the collaboration guidance with a tool-precedence rule (#800)#801
Poytr1 merged 1 commit into
mainfrom
claude/tool-precedence-prompt

Conversation

@Poytr1

@Poytr1 Poytr1 commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

What

Adds one leading bullet to the standing collaboration guidance (every session carries it): AgentConnect's MCP tools are the only channel that reaches other agents and humans; runtime built-ins with similar names (e.g. Claude Code's bare SendMessage) do not reach AgentConnect and anything sent through them is lost — never use them for messaging, reporting back, or collaboration. Cost: ~330 chars ≈ ~80 standing tokens per session. Pinned by a contract test that also asserts the bullet LEADS the section (the collision fires exactly when the model is choosing a messaging tool).

Why

Issue #800, measured in the tool-surface A/B (PR #791, 2026-08-09): on the Claude Code runtime a child session's tool list contains both mcp__agentconnect__sendMessage and the runtime's own built-in SendMessage (agent-teams messaging). In 3 of 6 parent-report trials the model's first attempt went to the built-in — a literal name match for the report-back instruction — and 2 of 6 trials lost the parent's answer entirely, silently.

Measured before/after (parent-session scenario, same harness as #791; criteria fixed before looking)

Metric Before (6 trials) After (10 trials, 5 per surface arm)
Trials with any built-in SendMessage attempt 3/6 0/10 (verified in scored attempts AND raw ACP events)
Answers lost to the built-in 2/6 0/10
Total success (parent actually woken with the answer) 4/6 10/10
First-attempt correct form 0/6 4/10

The collision is eliminated in this sample and no answer was lost. First-attempt behavior improved but is not perfect — the child still sometimes wraps the correct parent reply in extra (legal) postless calls — so #800's deeper fixes (suppress the built-in tool for daemon-owned sessions; reply-hint the exact pre-filled call; inferred reply) remain worthwhile hardening.

🤖 Generated with Claude Code

…cedence rule (#800)

Measured in the tool-surface A/B (2026-08-09, PR #791): a child session on
the Claude Code runtime also carries the runtime's own built-in
`SendMessage` (agent-teams messaging) — a literal name match for the
report-back instruction — and in 3 of 6 parent-report trials the model's
first attempt went there, silently losing the parent's answer in 2 of 6.

The standing context now says, before anything else about collaboration:
AgentConnect's MCP tools are the ONLY channel that reaches other agents
and humans, similarly-named runtime built-ins do not reach AgentConnect
and anything sent through them is lost. ~80 standing tokens per session.
This is the prompt-side half of #800; suppressing the built-in tool and
the reply-hint / inferred-reply hardening remain follow-ups.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@agentconnect-md-test agentconnect-md-test Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. The new precedence rule is placed before the existing collaboration guidance, so it applies through the shared standing-context path for fresh/resumed and Claude/non-Claude sessions. It names both the exact AgentConnect MCP tool and the conflicting bare runtime tool, and the added regression test pins the required content and ordering. git diff --check passes, and I found no blocking behavior issue.

Two non-blocking follow-ups:

  • “the ONLY channel that reaches other agents and humans here” is broader than the intended tool-precedence rule and is literally at odds with the later instruction that an ordinary reply reaches people and agents in the current thread. Consider tightening it to “Among messaging tools, only AgentConnect’s MCP tools…” while retaining the leading emphasis.
  • The newly added explanatory comments span multiple lines, while this repository’s AGENTS.md asks for one-line comments. They can be condensed without changing the test contract.

I could not execute the focused Vitest case because this isolated checkout has no installed dependencies and Corepack could not reach the npm registry (DNS failure), even after retrying with a writable cache. Static inspection confirms the test exercises the intended prompt block and ordering.

sent by review-bot (Codex · gpt-5.6-sol) · open in session

@Poytr1
Poytr1 merged commit 29feb9b into main Aug 10, 2026
9 checks passed
@Poytr1
Poytr1 deleted the claude/tool-precedence-prompt branch August 10, 2026 03:16
Poytr1 added a commit that referenced this pull request Aug 10, 2026
…ompt parity, tighten the channel-bare judge

Review findings on the exact head, all fixed:

1. The façade/classifier/fixture contract files were not in either CI
   contract command's explicit path list — added all three to
   eval:collab:contracts (now 18 files / 138 tests).
2. Prompt parity: merged main, which now carries the #800 tool-precedence
   bullet (#801), so BOTH arms' guidance includes it again. The doc's
   static table now records both revisions — the 24-run revision (parity
   held: neither arm had the bullet) and the current head (parity held:
   both do, A 3,099 / B 2,771 chars) — plus the separately measured
   bullet effect (built-in SendMessage attempts 3/6 -> 0/10, losses 0/10,
   success 10/10; issue #800).
3. channel-bare's judge now also requires that nobody was woken; the six
   recorded trials were re-verified under the stricter rule (peer ran
   zero turns in all six) and their 6/6 stands.

The substring FQN matching in matchesToolPermissionFqns is deliberate
consistency with the production isBuiltinSystemTool matcher (same
id-suffixed adapter variants), noted in the review reply.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Poytr1 added a commit that referenced this pull request Aug 11, 2026
…thread regression

This reverts commit 29feb9b (#801, the
prompt-side mitigation for #800). The bullet said AgentConnect MCP tools
are the ONLY channel that reaches agents and humans; that over-generalized
and taught models that ordinary in-thread replies reach nobody, so a live
turn-taking thread degraded into sendMessage hand-offs with meta-narration
and a broken count. The standing collaboration guidance returns to its
pre-#801 form byte-for-byte; #800 is reopened for scoped follow-ups.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Poytr1 added a commit that referenced this pull request Aug 11, 2026
…thread regression (#861)

This reverts commit 29feb9b (#801, the
prompt-side mitigation for #800). The bullet said AgentConnect MCP tools
are the ONLY channel that reaches agents and humans; that over-generalized
and taught models that ordinary in-thread replies reach nobody, so a live
turn-taking thread degraded into sendMessage hand-offs with meta-narration
and a broken count. The standing collaboration guidance returns to its
pre-#801 form byte-for-byte; #800 is reopened for scoped follow-ups.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Poytr1 added a commit that referenced this pull request Aug 11, 2026
…ion gate

The A/B matrix's four scenarios each demand one explicit send, so the #801
tool-precedence bullet could score 10/10 on parent-session while breaking
ordinary thread play (the live counting-game regression that #861 reverted;
issue #800 records the lesson). Scenario 5 is the missing coverage: one
plaza thread, BOTH agents on the arm's surface, a human kickoff
@-mentioning both, and the count carried by ordinary replies through the
platform echo and the #549 continuation ladder.

- `THREAD_COUNT_SCENARIO` + `judgeThreadCount` (daemon-judged): hard pass =
  count reaches 6 via delivered thread replies, ZERO messaging-tool calls
  by either participant (product surface, other arm's tool, or the runtime
  built-in — delivered or refused), no rejected participant reply. Soft
  (reported, never failed on): duplicates, skips, overshoot,
  meta-narration beyond the bare number, reply length, turns per number.
- Wired into the real-run matrix at index 4: same counterbalancing, seed
  derivation, validity rules (failed/timed-out turn => invalid), artifact
  layout, and summary; rides the existing scenario/arm env filters.
- Contract tests (eval:collab:contracts): the #801 handoff trace scores
  FAIL, a clean reply-only trace scores PASS, built-in SendMessage and
  arm-B `post` variants fail identically, soft-metric traces pass while
  measured; plus a scripted end-to-end fixture test proving a
  both-mentioned kickoff plus ordinary replies really carries the count
  peer-to-peer on a real daemon.
- Arm-B guidance parity: drop the #800 precedence bullet from
  POST_COLLAB_GUIDANCE, mirroring the production revert (#861) — both arms
  are back at the 24-run revision's text (2,718 / 2,394 chars, measured).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Poytr1 added a commit that referenced this pull request Aug 11, 2026
…nge gate

messaging-primitives-ab.md: §8 documents the in-thread turn-taking
scenario (the #801 incident, shape, daemon judge, and the 2026-08-11
post-revert baseline — 6/6 clean passes across both arms, zero
messaging-tool calls, zero meta-narration) plus the standing rule; §2/§4
updated for the scenario list and the parity revert (guidance back to the
24-run revision, re-measured at 2,718/2,394 chars).

collaboration-arena-baseline.md: §4.2 states the prompt-change gate — any
change to the standing collaboration guidance or parent-report append must
be validated against BOTH the parent-session scenario AND the in-thread
scenario before landing, citing #801/#861 as the incident that created the
rule.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant