Skip to content

Add Open Multi-Agent evaluation - #74

Open
JackChen-me wants to merge 1 commit into
benchflow-ai:mainfrom
JackChen-me:agent/add-open-multi-agent
Open

Add Open Multi-Agent evaluation#74
JackChen-me wants to merge 1 commit into
benchflow-ai:mainfrom
JackChen-me:agent/add-open-multi-agent

Conversation

@JackChen-me

Copy link
Copy Markdown

Summary

Add only the Open Multi-Agent evaluation surface to §5b · TypeScript/JS-native eval runners.

Why it clears the bar

This is a code-first TypeScript evaluation surface rather than a generic observability mention:

  • versioned EvalSets and scorers;
  • offline JSON, Markdown, and JUnit reports;
  • threshold and baseline-regression gates with CI exit-code behavior; and
  • opt-in production sampling that evaluates settled runs asynchronously and is isolated from the business result.

The public evaluation implementation and focused tests such as offline runner, gates, and online isolation show the work behind the one-line entry.

Verification

  • Checked the current README and open/closed issues and pull requests for the canonical repository URL and exact project name; no existing OMA submission was found.
  • Confirmed the repository and evaluation documentation links return HTTP 200.
  • Ran git diff --check.

Disclosure

I am a maintainer of Open Multi-Agent.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant