[WIP] Add support for eval-level default expect checks

[Copilot]

Thanks for asking me to work on this. I will get started on it and keep this PR's description up to date as I form a plan and make progress.

---

This section details on the original issue you should resolve

<issue_title>feat(config): support eval-level default expect checks</issue_title>
<issue_description>## Background

Users want to define common expect checks once in eval.yaml instead of repeating the same block in every evals/cases/*.yaml file.

Today, expect is case-only:

• config.EvalConfig has no Expect field.
• config.CaseDefaults currently supports defaults such as timeout_seconds, max_turns, and collect_artifacts, but not expect.
• config.CaseConfig has Expect Expect yaml:"expect".
• defaultEvaluator.runExpectPreCheck() calls resolveExpectConfig(&caseCfg.Expect), so only per-case expect is evaluated.

User Need

Many eval suites have invariant zero-cost checks that should apply to every case, for example:

• exit_code: 0
• output must not contain generic failure text or placeholders;
• generated files must exist for every case;
• a common safety/format marker must be present or absent.

Repeating those checks in every case file is noisy and easy to drift. Since expect is the cheap pre-judge gate, making common checks reusable would reduce boilerplate while preserving the existing fast-fail behavior.

Possible Design

Prefer adding expect under cases.defaults, because it matches the existing case-defaults inheritance model:

# evals/eval.yaml
cases:
  files:
    - evals/cases/*.yaml
  defaults:
    timeout_seconds: 180
    max_turns: 1
    expect:
      exit_code: 0
      must_not_contain:
        - "TODO"
        - "I cannot"
      files_exist:
        - "result.json"

Each case could still add or override checks:

# evals/cases/special.yaml
id: special
input:
  prompt: Run the special task
expect:
  must_contain:
    - "SPECIAL_OK"

An alternative is a top-level expect: in eval.yaml, but cases.defaults.expect may be less ambiguous because expect is evaluated per case, after each case's agent run.

Merge Semantics To Define

The implementation should explicitly define how defaults and per-case expect combine. A reasonable starting point:

• If no case-level expect is set, use cases.defaults.expect.
• Slice fields such as must_contain, must_not_contain, files_exist, files_not_exist, and file_contains are appended/de-duplicated with defaults first.
• Scalar fields such as exit_code and golden_file can be overridden by the case when set.
• Existing case-only behavior remains backward compatible.
• expect still runs before judge, and a failing merged expect still skips judge execution.

Acceptance Criteria

• CaseDefaults or EvalConfig supports a documented eval-level way to define default expect checks.
• The evaluator applies the effective merged expect for every case.
• Per-case expect can add to or override defaults using documented rules.
• Existing evals with only case-level expect behave unchanged.
• Unit tests cover config loading, merge behavior, scalar override behavior, and expect pre-check execution.
• Documentation and templates are updated in docs/guide/writing-evals.md, docs/zh/guide/writing-evals.md, and skills/skill-upper references/templates.</issue_description>
• Closes feat(config): support eval-level default expect checks #135

Comments on the Issue (you are @copilot in this section)

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[roark47]

@copilot It's ok, do it

[Copilot]

@copilot It's ok, do it

✅ Implementation complete!

I've successfully implemented eval-level default expect checks for skill-up. Here's what was delivered:

Core Changes

• Added Expect field to CaseDefaults in the config schema
• Implemented merge logic that combines default and case-level expect checks
• Updated the evaluator to apply merged expect configurations before running judges

Merge Behavior

• Slice fields (must_contain, must_not_contain, files_exist, etc.) are appended and de-duplicated
• Scalar fields (exit_code, golden_file) are overridden by case-level values when set
• Backward compatible - existing evals with only case-level expect work unchanged

Testing

• Comprehensive unit tests for all merge scenarios
• Integration tests verifying default expect behavior in real evaluation contexts
• All tests pass including race detector

Documentation

• Updated English and Chinese user guides
• Updated skill-upper reference documentation
• Added clear examples showing usage patterns

The feature is ready for review and merging. All code review and security scan validations passed successfully.