AgentBench / results
Versioned and inspectable

A result should
keep its context.

Editions bind results, evidence and approved analysis together. Pinned comparisons identify the exact edition rather than silently switching to newer scores.

Current edition

release-7f182ae03670cb3cd9a8dfdd5f2b42552946efd7e93b13425348b3096ae317a0

Campaign began 2026-10-04T07:57:09Z. This is a captured results edition, not a live harness connection.

Download public facts manifest (JSON / gzip) · Release manifest and checksums

Download aggregate results (CSV)

CSV contains aggregate metadata and measurements only. Text cells that could be interpreted as spreadsheet formulas receive a leading apostrophe; JSON and exact evidence downloads preserve their permitted bytes.

Large editions split their full facts into checksum-verified parts listed in the facts manifest; the release manifest provides each part’s path. Older editions contain inline facts. Individual model and question pages load their own evidence as needed.

Retained editions

Withdrawn editions

No editions have been withdrawn.

Recorded amendments

roster · open-models-20261015-v3

Owner-approved cloud quality references alongside local results. Hosted provider/precision are labeled; no cloud speed or local memory metrics.

roster · cloud-recovery-reference-review-20261017-v1

Read-only provenance review: the retry3 recovery names retry2-v2 but contains the retry2-v1 parent hash. Exact retained bytes are allowlisted for this one reference; both original amendments and failed retries are preserved. The selected terminal amendment independently pins those failed retry outcomes. No answer, grade, retry allowance or source file was changed.

roster · 20261014-openrouter-luna-v4

Owner-approved cloud quality references alongside local results. Hosted provider/precision are labeled; no cloud speed or local memory metrics.

correction · ifeval-1174-letter-minimum-v1

Prompt requests at least six o characters; original grading metadata erroneously required fewer than six.

exclusion · exclude-rg-dice-2-0

Seven-dice exact arithmetic has no local passes and still failed Nemotron Super with a 12,288-token thinking allowance. Poor discrimination in the current tool-free 4,096-token core battery. Valid question retained for optional tool-assisted or longer-budget work; not claimed impossible.

exclusion · exclude-rg-dice-2-1

Seven-dice exact arithmetic has no local passes and still failed Nemotron Super with a 12,288-token thinking allowance. Poor discrimination in the current tool-free 4,096-token core battery. Valid question retained for optional tool-assisted or longer-budget work; not claimed impossible.

exclusion · exclude-real_linux2-R5

Owner removed the sheet-music categorization/organization task: time-consuming and not a helpful benchmark.

correction · tuned-ten-execution-publication-v1

Preserve recorded native medium effort for stock GPT-OSS. Exclude mixed Qwen MTP-file timing after flash-attention/cache recovery; speculative decoding was disabled. Original measurements and failed attempts retained in the private source snapshot. Quality scores unchanged.

correction · merge-ba905af6504c3fec387f3c99

Additional campaign results included with separate execution and hardware identities. Original trials and outcomes are retained.

correction · drop-adjudication-v1

Reviewed saved DROP answers: fixed decimal percentage normalization, removed incorrect 68-day and 2-yard reference alternatives, and excluded an ambiguous passer/receiver question. Original answers and grades remain linked; no model reruns.

correction · scoring-adjudication-v2

Reviewed saved Make-24 and competition-math answers; exact input numbers and requested answer representations enforced. Invalid cube, both versions of contradictory asterisk item, and flawed CPU monitor check with incomplete replay evidence excluded uniformly. Original answers, grades, and earlier DROP adjudication retained; no model reruns.