Skip to content

fix(moa): add explicit warnings to reference prompt against claiming tool execution#61454

Open
liuhao1024 wants to merge 1 commit into
NousResearch:mainfrom
liuhao1024:liuhao/cron-bugfix-61452
Open

fix(moa): add explicit warnings to reference prompt against claiming tool execution#61454
liuhao1024 wants to merge 1 commit into
NousResearch:mainfrom
liuhao1024:liuhao/cron-bugfix-61452

Conversation

@liuhao1024

Copy link
Copy Markdown
Contributor

What does this PR do?

Prevents MoA (Mixture of Agents) reference models from fabricating tool execution in their text output. Reference models are advisory-only and should never claim to have executed commands, downloaded files, or accessed URLs — even as hypothetical statements.

Despite the existing _REFERENCE_SYSTEM_PROMPT stating "you cannot call tools", models still hallucinate execution in their responses (e.g., "I ran curl and got 404"). This fix adds:

  1. A prominent CRITICAL warning against claiming/implying execution
  2. Concrete bad/good examples showing what to avoid
  3. A trailing reminder "NEVER claim to have executed anything"

This addresses the aggregator's inability to distinguish real tool outputs from fabricated claims, which erodes trust in the entire MoA pipeline.

Related Issue

Fixes #61452

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

  • agent/moa_loop.py: Enhanced _REFERENCE_SYSTEM_PROMPT with explicit warnings and examples against claiming tool execution (12 lines added)
  • tests/agent/test_moa_reference_system_prompt.py: Added regression tests verifying the prompt contains necessary constraints (2 tests)

How to Test

Run the new regression tests to verify the prompt contains the required warnings:

pytest tests/agent/test_moa_reference_system_prompt.py -v

Observed result: both tests pass, confirming:

  • The prompt explicitly states reference models cannot execute anything
  • The prompt warns against claiming/implying execution
  • The prompt provides bad/good examples

End-to-end: When using MoA mode, reference model outputs should no longer contain claims like "I ran curl" or "I downloaded the file successfully."

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: macOS 15.2

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists labels Jul 9, 2026

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: LGTM

What the PR Does

Add explicit warnings to the MOA reference prompt against claiming tool execution.

Assessment

  • Small, focused doc/instruction fix.

Reviewed by Hermes Agent

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Reference models fabricate tool execution in text output, misleading aggregator

3 participants