AgentBench / results
Results by category

Models at a glance

Compare strengths across tasks. Select a score to inspect the answers.

Edition 7f182ae0
Size, architecture & run location
49 models · sorted by Balanced index ↓
Score0%100%* Not ranked — see why below— No scored result

Click headings to sort. Overall: nine equally weighted categories. Ranges show possible scores for incomplete results, which stay unranked. All scored models are shown; setups with no scores are omitted. On narrow screens, swipe across for all categories.

What does * mean? This is a recorded score with a comparison limitation: some tasks remain unscored, the setup’s comparability is unverified, or its thinking control was not verified. It does not mean partial credit on an answer. These cells stay outside the rankings. Select a cell to see its exact reason and scored task count.

Model scores by category, with an equally weighted overall index. Qualified and incomplete results are unranked. Each row retains its actual reasoning mode and setup.
Tested model One setup across the row
Ternary Bonsai 2 27B PTQ1_0Unknown · 26.9BThinking / Reasoning versionLocal
Qwen3.6 35B-A3BQ4_K_M · 34.7B · MoEThinking onLocal
Qwen3.8 Flash-Next (IQ3_XXS)IQ3_XXS · 176.9B · MoEThinking offLocal
Gemma 4 26B-A4BUD-Q4_K_M · 25.2B · MoEThinking offLocal
Gemma 4 26B-A4B QAT Q4_K_XLQAT-UD-Q4_K_XL · 25.2B · MoEThinking offLocal
Ornith 1.5 9B Q4_K_MQ4_K_M · 9.2BThinking onLocal
Qwen3.5 9BQ4_0 · 9.0BThinking onLocal
GPT-OSS 120B (stock)Q4_K_M · 116.8B · MoEReasoning · mediumLocal
Qwen3.6 28B-REAP20-A3BQ4_K_M · 28.2B · MoEThinking onLocal
Gemma 4 31B · CoreWeave (cloud)Hosted FP4 (provider-reported) · 30.7BThinking / Reasoning versionCloud · CoreWeave
Gemma 4 12BQ6_K · 11.9BThinking offLocal
GPT-OSS 120B (cloud)Hosted BF16 (provider-reported) · 117.0B · MoEReasoning · mediumCloud · DeepInfra
Gemma 4 12B QAT Q4_0QAT-Q4_0 · 11.9BThinking offLocal
Nemotron 3.5 Lightning 30B-A3B (cloud)Hosted BF16 (provider-reported) · 30.0B · MoEThinking / Reasoning versionCloud · DeepInfra
Qwen3.8 27B GSQ-RCOIQ2_S · 26.9BThinking onLocal
Qwen3-Coder-Next 80B-A3B (cloud)Hosted BF16 (provider-reported) · 80.0B · MoEThinking offCloud · Parasail
Granite 4.2 8B Q4_0Q4_0 · 8.8BThinking onLocal
Qwen3.6 29B REAP Opus distillQ4_K_M · 28.9B · MoEThinking onLocal
Ministral 3 14B Reasoning 2512 Q4_K_MQ4_K_M · 13.5BThinking / Reasoning versionLocal
Ministral 3 14BQ4_K_M · 13.5BThinking offLocal
Qwen3-Coder-NextIQ3_M · 79.7B · MoEThinking offLocal
Ling 3.0 Flash Q2_KIQ2_XXS · Size unknownThinking offLocal
Qwen3.5 4BQ4_K_M · 4.2BThinking offLocal
Ling 3.0 Tiny Q4_0Q4_0 · 7.9B · MoEThinking onLocal
Ornith 1.5 35B-A3BQ5_K_M · 35.5B · MoEThinking offLocal
Falcon-H1R 7B Q4_K_MQ4_K_M · 7.6BReasoning enabled (observed; requested off)Local
GLM-4.7-Flash REAP 23B-A3BUD-Q4_K_XL · 23.0B · MoEThinking offLocal
Ministral 3 8B Reasoning 2512 Q4_K_MQ4_K_M · 8.5BThinking / Reasoning versionLocal
Ministral 3 8B Instruct Q4_K_MQ4_K_M · 8.5BThinking offLocal
Mellum 2.1 12B-A2.5B ThinkingQ4_K_M · 12.1B · MoEThinking offLocal
LFM2.5 8B-A1B Q4_0Q4_0 · 8.5B · MoEReasoning enabled (observed; requested off)Local
Qwen3.5 2B Q4_K_MQ4_K_M · 1.9BThinking offLocal
Nemotron Nano 9B v2 Q4_0Q4_0 · 8.9BThinking offLocal
Gemma 4 E2B Q4_0Q4_0 · 4.6BThinking offLocal
Granite 4.0 Micro 3.4B Q4_0Q4_0 · 3.4BThinking offLocal
Gemma 4 E4BQ4_0 · 7.5BThinking offLocal
LFM2.5 1.2B Thinking Q4_K_MQ4_K_M · 1.2BThinking / Reasoning versionLocal
Qwen2.5 Coder 7B Instruct Q4_K_MQ4_K_M · 7.6BThinking offLocal
SmolLM3 3B Q4_K_MQ4_K_M · 3.1BThinking offLocal
Qwen3.5 0.8B Q4_0Q4_0 · 0.8BThinking offLocal
Gemma 4 19B-A4B REAPQ4_K_S · 18.4B · MoEThinking offLocal
LFM2.5 1.2B Q4_0Q4_0 · 1.2BThinking offLocal
Phi-4 Mini Instruct 3.8B Q4_K_MQ4_K_M · 3.8BThinking offLocal
GPT-6 Luna (cloud)Hosted UNDISCLOSED (provider-reported) · Size unknownReasoning · lowCloud · OpenAI via OpenRouter
Ling 3.0 Flash (cloud)Hosted BF16 (provider-reported) · 124.0B · MoEThinking / Reasoning versionCloud · DeepInfra
Mistral Small 4 119BUD-Q4_K_XL · 119.0B · MoEThinking onLocal
Phi-4 Mini Reasoning 3.8B Q4_K_MQ4_K_M · 3.8BThinking offLocal
Qwen3.8 27B (cloud)Hosted FP8 (provider-reported) · 27.0BReasoning · mediumCloud · Chutes
Qwen3.8 Flash-Next Coder (IQ1_M)IQ1_M · 116.5B · MoEThinking onLocal
How to read this view

The balanced index averages the nine category scores, giving each category 1/9 of the weight regardless of its question count. It measures quality on these test banks, not speed or general model ability. It is not Elo. Only setups with complete, eligible coverage in all nine categories receive a ranked index.

For incomplete results, the range runs from zero to full credit on every unscored task, keeping recorded scores fixed. It is a possible-score range, not a statistical confidence interval. Fully scored but qualified setups show an asterisk and remain unranked. Cells show rounded percentages; select one for exact scores and answers.

By default, all tested thinking modes are included. Each model uses the setup with the most eligible categories, then greatest scored coverage, then its declared baseline. Neither score nor speed chooses it. The same quant, configuration and thinking condition are used throughout a row. Enable all setups to inspect every quant and reasoning effort. Withdrawn setups are omitted.

Size means total parameters, not required RAM. Cloud results describe hosted versions and providers. The two instruction campaigns use identical questions and scorers and share one quality column; evidence IDs remain separate, as do hardware-specific speed comparisons. New, unverified test revisions will not be silently merged. Policy: balanced-categories-v1.