fix(evaluator): avoid duplicating evaluator instructions

[asdaxzcwqsa]

Summary

• stop injecting the evaluator agent instruction into the evaluation user prompt
• keep the evaluator criteria in the evaluator's system instruction only
• add a regression test covering the duplicated-instruction case

Problem

EvaluatorOptimizerLLM._build_eval_prompt() was embedding self.evaluator.instruction directly into the evaluation prompt even though the evaluator LLM already receives that instruction through its own configured system prompt. That causes evaluator instructions to appear twice.

Notes

This keeps the existing evaluation rubric intact while avoiding duplicate evaluator guidance in the user message.

Fixes #381

Summary by CodeRabbit

• Bug Fixes

  • Simplified the evaluation prompt so it uses a generic instruction instead of repeating the evaluator’s own guidance.
  • Kept the rest of the response review format unchanged, including rating, feedback, improvement flags, and focus areas.

• Tests

  • Added coverage to ensure the evaluation prompt no longer includes duplicated instructions.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 75201682-3947-471b-a78c-f401d786b3f0

📥 Commits

Reviewing files that changed from the base of the PR and between f62d849 and cbbf452.

📒 Files selected for processing (2)

• src/mcp_agent/workflows/evaluator_optimizer/evaluator_optimizer.py
• tests/workflows/evaluator_optimizer/test_evaluator_optimizer.py

---

📝 Walkthrough

Walkthrough

In _build_eval_prompt, the opening line of the evaluation prompt no longer embeds self.evaluator.instruction (e.g., "based on these criteria: …"). It is replaced with the static string "Evaluate the following response." A new unit test confirms the generic instruction appears and the evaluator-specific instruction does not.

Changes

Evaluator prompt deduplication

Layer / File(s)	Summary
Remove evaluator instruction from eval prompt + test src/mcp_agent/workflows/evaluator_optimizer/evaluator_optimizer.py, tests/workflows/evaluator_optimizer/test_evaluator_optimizer.py	_build_eval_prompt replaces the criteria-embedding opener with a static "Evaluate the following response." string. A new test (test_build_eval_prompt_does_not_repeat_evaluator_instruction) asserts the generic opener is present and the evaluator's own instruction string is absent from the built prompt.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Poem

A bunny once found a repeated refrain,
The evaluator's words echoed again and again! 🐇
With one tiny snip of a prompt's first line,
The duplicate voice was removed — how divine!
Now "Evaluate this!" rings out just once clear,
And the test confirms silence where echoes used to appear. ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main change: removing duplicated evaluator instructions.
Linked Issues check	✅ Passed	The PR removes the duplicated evaluator instruction from the evaluation prompt and adds a regression test, matching issue #381.
Out of Scope Changes check	✅ Passed	The changes stay focused on the evaluator prompt fix and its test coverage, with no obvious unrelated additions.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}