Skip to content

fix: isolate three-system benchmark environments - #13

Merged
wq19901103wq merged 1 commit into
mainfrom
codex/deepseek-official-benchmark
Jul 17, 2026
Merged

fix: isolate three-system benchmark environments#13
wq19901103wq merged 1 commit into
mainfrom
codex/deepseek-official-benchmark

Conversation

@wq19901103wq

Copy link
Copy Markdown
Owner

Summary

  • route Claude Code through the same configurable official DeepSeek endpoint and preflight every runner
  • isolate Claude Code and SWE-agent shell installs in per-task virtual environments
  • fetch enough Git history for correct setuptools-scm versions in local SWE-bench evaluation
  • preserve runner infrastructure errors and save partial reports before aborting

Validation

  • ruff check .
  • mypy agent tests
  • python -m pytest -q (351 passed, 3 skipped)
  • official DeepSeek smoke comparison: direct 1/1, Claude Code 1/1, SWE-agent 1/1

@wq19901103wq
wq19901103wq merged commit 4562cfe into main Jul 17, 2026
4 checks passed
@wq19901103wq
wq19901103wq deleted the codex/deepseek-official-benchmark branch July 17, 2026 22:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant