Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,14 @@ jobs:
pip install "mkdocs-material>=9.5" "mkdocs-static-i18n>=1.2"
mkdocs build --strict -d _site/docs

# ── Live demo: arbor replay --demo --html -> _site/demo.html ─────
# Regenerated from source on every deploy so the hosted, zero-install
# demo always matches the shipped code (no committed artifact to go stale).
- name: Build live demo page
run: |
pip install .
arbor replay --demo --html --no-open --out _site/demo.html

# ── Project page: Vite/React -> _site (root) ─────────────────────
- uses: actions/setup-node@v4
with:
Expand Down
45 changes: 29 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,15 @@
</p>


# Toward Generalist Autonomous Research via Hypothesis-Tree Refinement
<h1 align="center">🌳 Arbor</h1>

<h3 align="center">The autonomous research agent that beats Claude Code and Codex by <b>2.5×</b> on the same compute budget</h3>

<p align="center">
<a href="https://arxiv.org/pdf/2606.11926"><img src="https://img.shields.io/badge/Paper-arXiv-B31B1B?style=for-the-badge&logo=arxiv&logoColor=white" alt="Paper"></a>
<a href="https://github.com/RUC-NLPIR/Arbor"><img src="https://img.shields.io/badge/Code-GitHub-181717?style=for-the-badge&logo=github&logoColor=white" alt="GitHub"></a>
<a href="https://RUC-NLPIR.github.io/Arbor/"><img src="https://img.shields.io/badge/Project_Page-Live-0E9B9B?style=for-the-badge&logo=githubpages&logoColor=white" alt="Project Page"></a>
<a href="https://RUC-NLPIR.github.io/Arbor/demo.html"><img src="https://img.shields.io/badge/Live_Demo-Watch-FF5C8A?style=for-the-badge&logo=githubpages&logoColor=white" alt="Live Demo"></a>
<a href="https://RUC-NLPIR.github.io/Arbor/docs/"><img src="https://img.shields.io/badge/Docs-Material-526CFE?style=for-the-badge&logo=materialformkdocs&logoColor=white" alt="Docs"></a>
<a href="https://github.com/RUC-NLPIR/Arbor/discussions"><img src="https://img.shields.io/badge/Discussions-Join-5865F2?style=for-the-badge&logo=github&logoColor=white" alt="Discussions"></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/License-Apache_2.0-D22128?style=for-the-badge&logo=apache&logoColor=white" alt="License: Apache 2.0"></a>
Expand All @@ -19,18 +21,29 @@
<b>English</b> | <a href="README.zh-CN.md">简体中文</a>
</p>

**Arbor is an autonomous research agent that turns a long-horizon objective into a
cumulative search.** Give it a benchmark and a goal; it proposes hypotheses, edits
code, runs real experiments, learns from the results, and keeps the improvements that
hold up on held-out data. Instead of one-shot attempts that forget what failed, Arbor
grows a **hypothesis tree**: every idea becomes a branch — pruned if it fails,
harvested if it works — and insights propagate back so later ideas start smarter.
<p align="center">
<i>Give Arbor a benchmark and a goal. It proposes hypotheses, edits code, runs real
experiments, and keeps only the gains that survive held-out data — growing a
<b>hypothesis tree</b> instead of forgetting what failed.</i>
</p>

> **▶️ Try it in 30 seconds — no API key, no config:**
>
> ```bash
> pip install arbor-agent && arbor replay --demo # watch the hypothesis tree grow live
> ```
>
> Or **watch it right now in your browser** — nothing to install: **[▶️ Live Demo](https://RUC-NLPIR.github.io/Arbor/demo.html)**.

### 🏆 One controller, six tasks — wins the held-out test on all of them

| Task | Metric | Claude Code | Codex | **Arbor** |
| --- | --- | :---: | :---: | :---: |
| BrowseComp | acc ↑ | 53.33 | 50.00 | **67.67** |
| Terminal-Bench 2.0 | pass ↑ | 71.70 | 73.59 | **77.36** |
| Math-Reasoning Data | gap ↑ | 8.33 | 6.25 | **20.83** |

For more details, visit our [project page](https://RUC-NLPIR.github.io/Arbor/)
and read the [paper](https://arxiv.org/pdf/2606.11926). For a more detailed usage manual,
see our [documentation](https://RUC-NLPIR.github.io/Arbor/docs/). 🧭 You can also
choose the [CLI or Skill version](#-cli-and-skill-versions) depending on your
environment and workflow.
Plus **86.36% Any-Medal on MLE-Bench Lite** (GPT-5.5). → [See all six tasks](#-results) · [project page](https://RUC-NLPIR.github.io/Arbor/) · [paper](https://arxiv.org/pdf/2606.11926) · [docs](https://RUC-NLPIR.github.io/Arbor/docs/)

## 🎬 Demo

Expand All @@ -39,10 +52,10 @@ https://github.com/user-attachments/assets/49c1a306-d2e9-49d6-9c83-65e38a62df30

## 📣 News

- **2026-06** — **Built-in literature search & idea novelty checks.** Arbor can now ground its research in prior work via the public [alphaXiv](https://www.alphaxiv.org) API — zero config, no search endpoint or key. Novelty-check any idea before you build it with `arbor idea-check "<your idea>"`, or let the Coordinator vet every new branch automatically. See [Literature Search & Novelty Checks](#-literature-search--novelty-checks). 🔎
- **2026-06** — Arbor was featured by [VentureBeat](https://venturebeat.com/), one of the leading tech media outlets in the US: ["New AI optimization framework beats Claude Code and Codex by 2.5x on the same compute budget"](https://venturebeat.com/orchestration/new-ai-optimization-framework-beats-claude-code-and-codex-by-2-5x-on-the-same-compute-budget). 📰
- **2026-06** — Arbor's native CLI runtime and Agent Skill Suite (Codex / Claude Code) are released. 🚀
- **2026-06** — The Arbor paper is released on [arXiv](https://arxiv.org/abs/2606.11926). 🎉
- **2026-06-22** — **Built-in literature search & idea novelty checks.** Arbor can now ground its research in prior work via the public [alphaXiv](https://www.alphaxiv.org) API — zero config, no search endpoint or key. Novelty-check any idea before you build it with `arbor idea-check "<your idea>"`, or let the Coordinator vet every new branch automatically. See [Literature Search & Novelty Checks](#-literature-search--novelty-checks). 🔎
- **2026-06-18** — Arbor was featured by [VentureBeat](https://venturebeat.com/), one of the leading tech media outlets in the US: ["New AI optimization framework beats Claude Code and Codex by 2.5x on the same compute budget"](https://venturebeat.com/orchestration/new-ai-optimization-framework-beats-claude-code-and-codex-by-2-5x-on-the-same-compute-budget). 📰
- **2026-06-12** — Arbor's native CLI runtime and Agent Skill Suite (Codex / Claude Code) are released. 🚀
- **2026-06-11** — The Arbor paper is released on [arXiv](https://arxiv.org/abs/2606.11926). 🎉

## 💡 Why Arbor

Expand Down
39 changes: 28 additions & 11 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,15 @@
</p>


# 基于假设树的面向通用自主科研方法(Toward Generalist Autonomous Research via Hypothesis-Tree Refinement)
<h1 align="center">🌳 Arbor</h1>

<h3 align="center">在相同算力预算下,效果超越 Claude Code 与 Codex <b>2.5×</b> 的自主科研智能体</h3>

<p align="center">
<a href="https://arxiv.org/pdf/2606.11926"><img src="https://img.shields.io/badge/Paper-arXiv-B31B1B?style=for-the-badge&logo=arxiv&logoColor=white" alt="Paper"></a>
<a href="https://github.com/RUC-NLPIR/Arbor"><img src="https://img.shields.io/badge/Code-GitHub-181717?style=for-the-badge&logo=github&logoColor=white" alt="GitHub"></a>
<a href="https://RUC-NLPIR.github.io/Arbor/"><img src="https://img.shields.io/badge/Project_Page-Live-0E9B9B?style=for-the-badge&logo=githubpages&logoColor=white" alt="Project Page"></a>
<a href="https://RUC-NLPIR.github.io/Arbor/demo.html"><img src="https://img.shields.io/badge/Live_Demo-Watch-FF5C8A?style=for-the-badge&logo=githubpages&logoColor=white" alt="Live Demo"></a>
<a href="https://RUC-NLPIR.github.io/Arbor/docs/"><img src="https://img.shields.io/badge/Docs-Material-526CFE?style=for-the-badge&logo=materialformkdocs&logoColor=white" alt="Docs"></a>
<a href="https://github.com/RUC-NLPIR/Arbor/discussions"><img src="https://img.shields.io/badge/讨论区-加入-5865F2?style=for-the-badge&logo=github&logoColor=white" alt="Discussions"></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/License-Apache_2.0-D22128?style=for-the-badge&logo=apache&logoColor=white" alt="License: Apache 2.0"></a>
Expand All @@ -19,20 +21,35 @@
<a href="README.md">English</a> | <b>简体中文</b>
</p>

**Arbor 是一个自主科研智能体,可以把长周期目标转化为持续累积的搜索过程。** 给它一个基准
(benchmark)和一个目标,它会提出假设、修改代码、运行真实实验、从结果中学习,并保留那些在
留出(held-out)数据上经得起验证的改进。不同于“一次性尝试、过后即弃”的做法,Arbor 会逐步生长出一棵
**假设树**:每个想法都是一根分支,失败则剪枝,成功则保留;洞见会沿树反向传播,让后续想法
从更可靠的起点出发。
<p align="center">
<i>给 Arbor 一个基准和一个目标。它会提出假设、修改代码、运行真实实验,只保留经得起留出
数据验证的改进——生长出一棵<b>假设树</b>,而不是过后即弃、忘记失败的教训。</i>
</p>

> **▶️ 30 秒上手——无需 API key,无需配置:**
>
> ```bash
> pip install arbor-agent && arbor replay --demo # 实时观看假设树的生长过程
> ```
>
> 或者**现在就在浏览器里看**——无需安装任何东西:**[▶️ 在线 Demo](https://RUC-NLPIR.github.io/Arbor/demo.html)**。

### 🏆 一个控制器,六项任务——全部赢下留出测试集

| 任务 | 指标 | Claude Code | Codex | **Arbor** |
| --- | --- | :---: | :---: | :---: |
| BrowseComp | acc ↑ | 53.33 | 50.00 | **67.67** |
| Terminal-Bench 2.0 | pass ↑ | 71.70 | 73.59 | **77.36** |
| Math-Reasoning Data | gap ↑ | 8.33 | 6.25 | **20.83** |

更多详情,请访问我们的[项目主页](https://ruc-nlpir.github.io/Arbor/)并阅读[论文](https://arxiv.org/pdf/2606.11926)。如需详细的使用说明,请参阅[文档](https://ruc-nlpir.github.io/Arbor/docs/)。🧭 你也可以根据自己的环境和工作流选择使用 [CLI 版本或技能套件版本](https://claude.ai/chat/e7121091-ce2c-4970-a60f-16b54c453729#-cli-与技能套件版本)。
外加 **MLE-Bench Lite 上 86.36% Any-Medal**(GPT-5.5)。→ [查看全部六项任务](#-实验结果) · [项目主页](https://ruc-nlpir.github.io/Arbor/) · [论文](https://arxiv.org/pdf/2606.11926) · [文档](https://ruc-nlpir.github.io/Arbor/docs/)

## 📣 最新动态

- **2026-06** — **内置文献检索与想法新颖性审查。** Arbor 现在可以通过 [alphaXiv](https://www.alphaxiv.org) 公共 API 把研究建立在已有工作之上——零配置,无需搜索端点或密钥。动手前用 `arbor idea-check "<你的想法>"` 审查任意想法的新颖性,或让 Coordinator 自动为每个新分支把关。详见[文献检索与新颖性审查](#-文献检索与新颖性审查)。🔎
- **2026-06** — Arbor 被美国知名科技媒体 [VentureBeat](https://venturebeat.com/) 报道:[《New AI optimization framework beats Claude Code and Codex by 2.5x on the same compute budget》](https://venturebeat.com/orchestration/new-ai-optimization-framework-beats-claude-code-and-codex-by-2-5x-on-the-same-compute-budget)。📰
- **2026-06** — Arbor 原生 CLI 运行时与智能体技能套件(Codex / Claude Code)正式发布。🚀
- **2026-06** — Arbor 论文在 [arXiv](https://arxiv.org/abs/2606.11926) 发布。🎉
- **2026-06-22** — **内置文献检索与想法新颖性审查。** Arbor 现在可以通过 [alphaXiv](https://www.alphaxiv.org) 公共 API 把研究建立在已有工作之上——零配置,无需搜索端点或密钥。动手前用 `arbor idea-check "<你的想法>"` 审查任意想法的新颖性,或让 Coordinator 自动为每个新分支把关。详见[文献检索与新颖性审查](#-文献检索与新颖性审查)。🔎
- **2026-06-18** — Arbor 被美国知名科技媒体 [VentureBeat](https://venturebeat.com/) 报道:[《New AI optimization framework beats Claude Code and Codex by 2.5x on the same compute budget》](https://venturebeat.com/orchestration/new-ai-optimization-framework-beats-claude-code-and-codex-by-2-5x-on-the-same-compute-budget)。📰
- **2026-06-12** — Arbor 原生 CLI 运行时与智能体技能套件(Codex / Claude Code)正式发布。🚀
- **2026-06-11** — Arbor 论文在 [arXiv](https://arxiv.org/abs/2606.11926) 发布。🎉

## 💡 为什么选择 Arbor

Expand Down
46 changes: 46 additions & 0 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,52 @@ sources behind each idea) and a per-run cost breakdown.

---

## Direction 4 — Adoption, DX & community

The first three directions grow what Arbor *does*. This one lowers the barrier
between a curious visitor and a first successful run, and makes the project's
momentum visible. The goal is a shorter path to the "aha" moment, not more
surface area.

### 4.1 Live, zero-install demo

`arbor replay --demo --html` already emits a self-contained, dependency-free page
of a real run. Publish that page (e.g. on GitHub Pages alongside the docs) and
link it from the top of the README, so a visitor can watch the hypothesis tree
grow **without installing anything**. One recorded run, refreshed when the
dashboard changes.

### 4.2 Examples gallery

Today `examples/algotune_knn` is the only end-to-end task a newcomer can run.
Grow this into a small gallery so different audiences can self-identify, each
with a copy-pasteable command and a short asciinema recording:

- a Kaggle / MLE-style task (pairs with the `mle_kaggle` plugin),
- a prompt / harness-engineering task, and
- a small training / fine-tuning task.

Keep the bar at "runs in minutes on a laptop or a free key", reusing the
zero-config discipline of `algotune_knn`.

### 4.3 Regular releases and a changelog

Versions are already derived from `v*` tags via setuptools-scm, so the cost of a
release is low. Cut a release on a predictable cadence with human-readable notes,
and keep a `CHANGELOG.md` (or GitHub Releases as the source of truth) so the
project's progress is legible from the outside.

### 4.4 Public roadmap and issue board

Surface this document as a public board (GitHub Projects) and tag tracked work
with `good first issue` / `help wanted`. Several threads are already
contribution-shaped — growing the [benchmark zoo](#21-benchmark-zoo-organized-by-domain-format-tooling-shipped-collection-growing)
to 3–5 packs ([2.1](#21-benchmark-zoo-organized-by-domain-format-tooling-shipped-collection-growing))
and adding domain plugins ([2.2](#22-plugin-gallery)) — so opening them up turns
readers into contributors.

---

Have an idea or want to own one of these threads? Open a
[discussion](https://github.com/RUC-NLPIR/Arbor/discussions) or see
[Contributing](contributing.md).
43 changes: 41 additions & 2 deletions docs/roadmap.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@

## 方向二 —— 外部资源

### 2.1 按 domain 划分的 benchmark zoo 🚧 *(格式与工具已完成;集合扩充中)*
### 2.1 按 domain 划分的 benchmark zoo 🚧 *(格式与工具已完成;集合扩充中)* {#sec-2-1}

一个经过筛选、统一格式的任务集合,按领域分组(如 CV、NLP、时序、优化),每个任务用一篇已
发表论文的结果作为要超越的 baseline。它以 `arbor-zoo/` 放在仓库里,每个基准一个文件夹,
Expand Down Expand Up @@ -106,7 +106,7 @@
复用 `plugin` 词汇(`eval_contract` / `protected_paths`),应能几乎无返工地导出(与 2.2
配套)。

### 2.2 插件库
### 2.2 插件库 {#sec-2-2}

在 `mle_kaggle` 之外提供更多范例领域插件,与上面的 Task Pack 配对,让把 Arbor 重定向到
一个领域只需改一行 `plugin:`。
Expand Down Expand Up @@ -138,5 +138,44 @@

---

## 方向四 —— 采用、开发者体验与社区

前三个方向扩展 Arbor *能做什么*;这个方向降低"好奇的访客"到"第一次成功运行"
之间的门槛,并让项目的推进势头可见。目标是缩短到达"aha 时刻"的路径,而不是堆叠
更多功能面。

### 4.1 免安装的在线 Demo

`arbor replay --demo --html` 已经能导出一个自包含、零依赖的真实 run 页面。把它
发布出去(例如随文档一起部署到 GitHub Pages),并从 README 顶部链接过去,让访客
**无需安装任何东西**就能看假设树生长。一份录制好的 run,随仪表盘变化刷新即可。

### 4.2 示例画廊

目前 `examples/algotune_knn` 是新手唯一能端到端跑通的任务。把它扩成一个小型画廊,
让不同受众都能对号入座,每个都配可复制的命令和一段 asciinema 录屏:

- 一个 Kaggle / MLE 风格的任务(与 `mle_kaggle` 插件配套),
- 一个 prompt / harness 工程任务,以及
- 一个小规模训练 / 微调任务。

门槛保持在"笔记本上几分钟、或用免费 key 即可跑通",复用 `algotune_knn` 的零配置
纪律。

### 4.3 规律发布与更新日志

版本号已经通过 setuptools-scm 从 `v*` tag 自动派生,发布成本很低。以可预期的节奏
发布版本并附上人类可读的发布说明,并维护一份 `CHANGELOG.md`(或以 GitHub Releases
为准),让项目进展从外部看也一目了然。

### 4.4 公开路线图与 issue 看板

把本文档以公开看板(GitHub Projects)的形式呈现,并给跟踪中的工作打上
`good first issue` / `help wanted` 标签。有几条线索天然适合外部贡献——把
[benchmark zoo](#sec-2-1) 扩到 3–5 个 pack([2.1](#sec-2-1))、增加领域插件
([2.2](#sec-2-2))——把它们开放出来,就能把读者变成贡献者。

---

有想法,或想认领其中一条线索?开一个
[discussion](https://github.com/RUC-NLPIR/Arbor/discussions),或见 [贡献](contributing.md)。
Loading