feat(benchmark): add swe-bench runner#1423
Merged
Merged
Conversation
Introduce the swe-runner package under src/benchmark/, providing a local batch runner for SWE-bench with cosh and OpenClaw adapters. It prepares isolated Docker workspaces, produces evaluator-compatible predictions, and records artifacts and traces. Signed-off-by: 爱鲲 <jiawa.syx@alibaba-inc.com>
alex-yuxi
approved these changes
Jul 10, 2026
alex-yuxi
left a comment
alex-yuxi
left a comment
Collaborator
There was a problem hiding this comment.
本PR为使用openclaw运行SWEBench的agent部分代码，功能对标mini-swe-agent，其稳定性和准确性高于后者，功能验证完整有效，代码审核未发现明显问题。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add a local SWE-bench batch runner with
coshand OpenClaw agent adapters. It prepares isolated Docker workspaces, writes evaluator-compatible predictions and run artifacts, and can export OpenClaw trace metrics.Related Issue
no-issue: initial benchmark runner addition is not linked to an issue
Type of Change
Scope
Checklist
Testing
Not run (not requested).
Additional Notes
None.