-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathindex.html
More file actions
256 lines (239 loc) · 19.2 KB
/
Copy pathindex.html
File metadata and controls
256 lines (239 loc) · 19.2 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
<!--
SPDX-License-Identifier: Apache-2.0
Copyright (C) 2025, The Exgentic organization and its contributors.
-->
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<link rel="icon" type="image/png" href="/favicon.png?v=3" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="The Open Agent Leaderboard. Systematic evaluation across diverse environments without domain-specific tuning." />
<title>Open Agent Leaderboard</title>
<!-- Open Graph -->
<meta property="og:type" content="website" />
<meta property="og:title" content="Open Agent Leaderboard" />
<meta property="og:description" content="How good are general purpose AI agents? We test them across diverse domains. No hand-tuning. No shortcuts. Fairly Ranked." />
<meta property="og:url" content="https://exgentic.ai/" />
<meta property="og:image" content="https://exgentic.ai/og-image.png" />
<meta name="twitter:card" content="summary_large_image" />
<meta name="twitter:title" content="Open Agent Leaderboard" />
<meta name="twitter:description" content="How good are general purpose AI agents? We test them across diverse domains. No hand-tuning. No shortcuts. Fairly Ranked." />
<meta name="twitter:image" content="https://exgentic.ai/og-image.png" />
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@300;400;500;600;700;800;900&family=JetBrains+Mono:wght@400;500;600&display=swap" rel="stylesheet">
<script src="https://cdn.jsdelivr.net/npm/gsap@3.12.7/dist/gsap.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/gsap@3.12.7/dist/ScrollTrigger.min.js"></script>
<!-- Google Analytics 4 -->
<script async src="https://www.googletagmanager.com/gtag/js?id=G-XKC51P054L"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'G-XKC51P054L');
</script>
<!-- Apply theme before render to prevent flash -->
<script>
var t=localStorage.getItem('exgentic-theme');
if(t==='light'||(t===null&&window.matchMedia('(prefers-color-scheme:light)').matches))
document.documentElement.classList.add('light-mode-early');
</script>
<link rel="stylesheet" href="/styles.css">
</head>
<body class="page-landing">
<!-- Loading Screen -->
<div id="loadingScreen">
<div class="loading-content">
<div class="loading-fill" id="loadingFill"></div>
</div>
<div id="floatingLabels"></div>
</div>
<!-- Mesh Gradient Blobs -->
<!-- Header -->
<header class="site-header" id="header">
<div class="header-inner">
<a href="/" aria-label="Exgentic Home" class="nav-logo-link">
<svg class="nav-logo-icon" viewBox="0 0 40 48" fill="none" stroke="none">
<polygon points="4,8 36,8 20,30" class="tri-outer"/>
<polygon points="4,44 36,44 20,22" class="tri-outer"/>
<polygon points="9,42 31,42 20,35" class="tri-inner"/>
</svg>
<span class="nav-logo-text">Exgentic</span>
</a>
<button class="mobile-toggle" id="mobileToggle" aria-label="Toggle navigation">☰</button>
<nav class="header-nav" id="headerNav">
<a href="#leaderboard">Leaderboard</a>
<a href="#findings">Findings</a>
<a href="#faq">FAQ</a>
<a href="#blog">Blog</a>
<a href="https://arxiv.org/abs/2602.22953" target="_blank" rel="noopener">Paper</a>
<a href="https://github.com/Exgentic/exgentic" target="_blank" rel="noopener" class="btn-nav">
<svg width="16" height="16" viewBox="0 0 16 16" fill="currentColor" style="vertical-align:middle;margin-right:4px"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/></svg>
GitHub
</a>
<button class="theme-toggle" id="themeToggle" title="Toggle light/dark mode">☾</button>
</nav>
</div>
</header>
<!-- Hero -->
<section class="hero" id="hero">
<div class="hero-content">
<div class="hero-badge"><span class="dot"></span>Open Source</div>
<h1 class="hero-title" data-text="Open Agent Leaderboard">Open Agent<br>Leaderboard</h1>
<p class="hero-subtitle">How good are general purpose AI agents?<br>We test them across diverse domains.<br>No hand-tuning. No shortcuts. Fairly Ranked.</p>
<div class="hero-cards">
<div class="hero-card"><svg class="hero-icon" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"><path d="M20 21v-2a4 4 0 0 0-4-4H8a4 4 0 0 0-4 4v2"/><circle cx="12" cy="7" r="4"/></svg><span class="domain">Personal Assistance</span><span class="cite">AppWorld</span></div>
<div class="hero-card"><svg class="hero-icon" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"><path d="M21 15a2 2 0 0 1-2 2H7l-4 4V5a2 2 0 0 1 2-2h14a2 2 0 0 1 2 2z"/></svg><span class="domain">Customer Service</span><span class="cite">τ²-bench Airline, Retail</span></div>
<div class="hero-card"><svg class="hero-icon" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"><path d="M14.7 6.3a1 1 0 0 0 0 1.4l1.6 1.6a1 1 0 0 0 1.4 0l3.77-3.77a6 6 0 0 1-7.94 7.94l-6.91 6.91a2.12 2.12 0 0 1-3-3l6.91-6.91a6 6 0 0 1 7.94-7.94l-3.76 3.76z"/></svg><span class="domain">Technical Support</span><span class="cite">τ²-bench Telecom</span></div>
<div class="hero-card"><svg class="hero-icon" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"><circle cx="11" cy="11" r="8"/><line x1="21" y1="21" x2="16.65" y2="16.65"/></svg><span class="domain">Deep Research</span><span class="cite">BrowseComp+</span></div>
<div class="hero-card"><svg class="hero-icon" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"><polyline points="16 18 22 12 16 6"/><polyline points="8 6 2 12 8 18"/></svg><span class="domain">Software Engineering</span><span class="cite">SWE-Bench</span></div>
</div>
<div class="hero-cta">
<span><a href="#leaderboard" class="btn btn-primary">Explore Leaderboard</a></span>
<span><a href="https://github.com/Exgentic/exgentic" target="_blank" rel="noopener" class="btn btn-secondary">
<svg width="18" height="18" viewBox="0 0 16 16" fill="currentColor"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/></svg>
GitHub
</a></span>
</div>
</div>
<div class="scroll-hint" id="scrollHint">
<svg width="36" height="36" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="3" stroke-linecap="round" stroke-linejoin="round"><polyline points="6 9 12 15 18 9"/></svg>
</div>
</section>
<!-- Leaderboard (lead with the product) -->
<section class="section-alt" id="leaderboard">
<div class="container container-narrow">
<span class="section-label">Live Rankings</span>
<h2 class="section-title">The Agent Leaderboard</h2>
<p class="section-subtitle">Head-to-head performance across diverse benchmarks.</p>
<div class="leaderboard-controls"><h3>Leaderboard</h3><div><select class="filter-select" id="modelFilter"><option value="all">All Models</option></select></div></div>
<div class="table-wrap"><table class="lb-table" id="lbTable"><thead><tr id="lbHead"></tr></thead><tbody id="lbBody"></tbody></table></div>
<div class="lb-footer" style="display:flex;justify-content:space-between;align-items:center;flex-wrap:wrap">
<div style="display:flex;align-items:center;gap:12px"><a href="/results.csv" download>Download CSV</a><span style="color:var(--text-muted);font-size:1.2em">·</span><a href="/LICENSE-DATA.txt">License</a></div>
<span id="lastUpdated"></span>
</div>
</div>
</section>
<!-- Cost vs Performance -->
<section id="pareto">
<div class="container container-narrow">
<span class="section-label">Efficiency</span>
<h2 class="section-title">What Does Performance Cost?</h2>
<p class="section-subtitle">Which agents deliver the most for the least?</p>
<div id="paretoChartContainer"></div>
</div>
</section>
<!-- Key Findings -->
<section class="section-alt" id="findings">
<div class="container container-narrow">
<span class="section-label">Results</span>
<h2 class="section-title">What the Data Shows</h2>
<p class="section-subtitle">Five findings that challenge conventional assumptions about AI agents.</p>
<div class="findings-grid">
<div class="finding-card glass"><h3>General agents adapt without manual tuning</h3><p>Software engineering, customer service, web research, and everyday digital tasks, all without per-domain manual customization or training.</p></div>
<div class="finding-card glass"><h3>The model matters most, but so does the agent</h3><p>Model choice drives 28% of task variance, but agent choice still swings results by up to 11 percentage points.</p></div>
<div class="finding-card glass"><h3>General agents already rival specialists</h3><p>On half the tested benchmarks, general agents match or beat the top published scores from domain-specific systems, with none of the per-domain engineering.</p></div>
<div class="finding-card glass"><h3>Open-weight models aren't reliably general</h3><p>The open-weight models we tested can swing wildly depending on the agent, or collapse on entire tasks. Closed-source frontier models hold up.</p></div>
<div class="finding-card glass"><h3>Similar scores, very different failures</h3><p>Two agents with the same overall score can fail in very different ways: some stop too early, others skip past the evidence. Scores alone don't tell the whole story.</p></div>
</div>
<div class="findings-cta"><span><a href="https://arxiv.org/abs/2602.22953" target="_blank" rel="noopener" class="btn btn-primary">Read the Full Paper (arXiv) →</a></span></div>
</div>
</section>
<!-- Force-Directed Data Graph -->
<section class="graph-section" style="overflow:hidden">
<div style="text-align:center;padding:0 0">
<span class="section-label">Network Map</span>
<h2 class="section-title">The Evaluation Graph</h2>
<p class="section-subtitle" style="margin-bottom:0">Every agent. Every benchmark. Every connection.</p>
<canvas id="constellation"></canvas>
</div>
</section>
<!-- FAQ -->
<section class="section-alt" id="faq">
<div class="container">
<span class="section-label">FAQ</span>
<h2 class="section-title">Frequently Asked Questions</h2>
<p class="section-subtitle">Common questions about Exgentic's evaluation methodology and results.</p>
<div class="faq-container">
<div class="faq-item glass"><button class="faq-question">Why are your reported results different from those in the leaderboard or whitepaper?<span class="faq-chevron">▼</span></button><div class="faq-answer"><p>Many agent or model whitepapers rely on significant prompt optimization to maximize performance on specific benchmarks. In Exgentic, we intentionally avoid prompt optimization to provide a more neutral and comparable evaluation across agents.</p><p>In addition, we report results on 100 sampled tasks per benchmark. For some benchmarks this is a subset of the full dataset, but we believe this is not the primary factor driving differences in results.</p></div></div>
<div class="faq-item glass"><button class="faq-question">Do you modify benchmarks or agents?<span class="faq-chevron">▼</span></button><div class="faq-answer"><p>No. We do not modify the benchmarks or the agent implementations. In some cases, we needed to slightly adapt the interface to fit the unified protocol—for example: Externalizing prompts that are embedded inside the benchmark (e.g., TAU-Bench). Adding task instructions when required by the benchmark specification (e.g., SWE-Bench Verified). These adjustments do not change the task itself.</p></div></div>
<div class="faq-item glass"><button class="faq-question">How did you decide which agents to include?<span class="faq-chevron">▼</span></button><div class="faq-answer"><p>We selected agents that represent commonly used general-agent architectures and that can operate across multiple benchmarks with minimal task-specific customization. Our goal is to evaluate general-purpose agents, not agents tailored to a single benchmark. We will add more agents, and we welcome contributions from the community.</p></div></div>
<div class="faq-item glass"><button class="faq-question">How did you select the benchmarks?<span class="faq-chevron">▼</span></button><div class="faq-answer"><p>Since Exgentic focuses on evaluating generality, we chose benchmarks that cover a diverse range of domains and task types, including coding, tool use, reasoning, and interactive environments.</p></div></div>
<div class="faq-item glass"><button class="faq-question">Why do I only see closed models in the leaderboard?<span class="faq-chevron">▼</span></button><div class="faq-answer"><p>We started with a small set of widely used models to establish the initial leaderboard. We plan to expand the leaderboard soon with many open-weight models as well as additional closed models.</p></div></div>
<div class="faq-item glass"><button class="faq-question">Can I reproduce your results?<span class="faq-chevron">▼</span></button><div class="faq-answer"><p>Yes. Exgentic is fully open. The evaluation pipeline, agent implementations, and configuration are available in the repository so results can be reproduced. Minor variations may occur due to model version changes or nondeterminism in LLM outputs.</p></div></div>
</div>
</div>
</section>
<!-- Get Involved -->
<section class="get-started" id="start">
<div class="container container-narrow">
<span class="section-label">Get Involved</span>
<h2 class="section-title">Open Source. Open Data. Open Leaderboard.</h2>
<p class="section-subtitle">Everything is open. Jump in.</p>
<div class="gs-grid">
<div class="gs-card glass"><h3>Run Evaluations</h3><p>Clone the repo. Evaluate your agent against the field. Results in hours, not weeks.</p><span><a href="https://github.com/Exgentic/exgentic" target="_blank" rel="noopener" class="btn-outline">Get the Code</a></span></div>
<div class="gs-card glass"><h3>Submit Your Agent</h3><p>Put your agent on the board. See exactly where it stands against the field.</p><span><a href="https://github.com/Exgentic/exgentic/blob/main/README.md" target="_blank" rel="noopener" class="btn-outline">Submit</a></span></div>
<div class="gs-card glass"><h3>Add a Benchmark</h3><p>Plug your evaluation environment into the Unified Protocol. Instant cross-agent testing.</p><span><a href="https://github.com/Exgentic/exgentic/blob/main/README.md" target="_blank" rel="noopener" class="btn-outline">Integrate</a></span></div>
</div>
</div>
</section>
<!-- Blog -->
<section class="section-alt" id="blog">
<div class="container container-narrow">
<span class="section-label">Insights</span>
<h2 class="section-title">Blog</h2>
<p class="section-subtitle">Research updates, benchmarks, and findings from the Exgentic team.</p>
<div class="blog-cards-grid">
<div class="blog-card-home glass">
<div class="blog-card-home-meta">
<time>May 13, 2026</time>
<span>·</span>
<span>10 min read</span>
</div>
<h3><a href="/blog/open-weight-agents/">Are open-weight models fit for agentic workloads?</a></h3>
<p>We compared widely used open-weight models to frontier closed ones. We find that open models trail on quality, fail unpredictably, and only tie on cost.</p>
<a href="/blog/open-weight-agents/" class="blog-card-home-link">Read more →</a>
</div>
<div class="blog-card-home glass">
<div class="blog-card-home-meta">
<time>Mar 17, 2026</time>
<span>·</span>
<span>12 min read</span>
</div>
<h3><a href="/blog/eval-reporting/">Rethinking Agent Evaluation Reporting</a></h3>
<p>Why pass/fail scores hide what matters most about agent systems.</p>
<a href="/blog/eval-reporting/" class="blog-card-home-link">Read more →</a>
</div>
<div class="blog-card-home glass">
<div class="blog-card-home-meta">
<time>Mar 15, 2026</time>
<span>·</span>
<span>9 min read</span>
</div>
<h3><a href="/blog/open-leaderboard/">The Open Agent Leaderboard</a></h3>
<p>How good are general purpose AI agents? We built an open evaluation framework to find out.</p>
<a href="/blog/open-leaderboard/" class="blog-card-home-link">Read more →</a>
</div>
<div class="blog-card-placeholder" aria-hidden="true"></div>
</div>
</div>
</section>
<!-- Footer -->
<footer class="site-footer">
<div class="container">
<div class="footer-grid">
<div class="footer-brand"><h3>Exgentic</h3><p>The open community for evaluating AI agents.</p></div>
<div class="footer-col"><h4>Resources</h4><ul><li><a href="https://github.com/Exgentic/exgentic" target="_blank" rel="noopener">GitHub</a></li><li><a href="https://arxiv.org/abs/2602.22953" target="_blank" rel="noopener">Paper</a></li><li><a href="https://github.com/Exgentic/exgentic/blob/main/README.md" target="_blank" rel="noopener">Contribute</a></li><li><a href="https://github.com/Exgentic/open-agent-leaderboard/blob/main/CONTRIBUTING.md" target="_blank" rel="noopener">Submit</a></li></ul></div>
<div class="footer-col"><h4>Community</h4><ul><li><a href="https://github.com/Exgentic/exgentic/blob/main/LICENSE" target="_blank" rel="noopener">License</a></li><li><a href="https://github.com/Exgentic/exgentic/blob/main/README.md#-citing-exgentic" target="_blank" rel="noopener">Citation</a></li><li><a href="https://github.com/Exgentic/exgentic/issues" target="_blank" rel="noopener">Contact</a></li></ul></div>
</div>
<div class="footer-bottom"><span>© 2026 Exgentic</span></div>
</div>
</footer>
<script src="/scripts/loading.js"></script>
<script src="/scripts/main.js"></script>
<script src="/scripts/animations.js"></script>
<script src="/scripts/pareto.js"></script>
<script src="/scripts/leaderboard.js"></script>
</body>
</html>