AgentBench / results
← All models

Ministral 3 14B Reasoning 2512 Q4_K_M

1 tested weight format · Explore results, then inspect the answers.

Choose a tested configuration

Get this tested model

Q4_K_M · 8.24 GB download

Links follow the selected quant and configuration.

Setup, memory & provenance · Q4_K_M

Q4_K_M · 8.24 GB of weights · vulkan · context 16384 · GPU layers 99

Medium · 5–under 20B · dense · 13.5B total

Memory fit has not been verified for this configuration.

completed

Recorded settings and thinking verification

Temperature 0 · seed 42 · output limit 4096 · reasoning effort unrecorded

RAM labels describe the test computer, not this model’s memory requirement. Speed comparisons keep the 32 GB and 128 GB runs separate.

Runtime build: unrecorded · Local · test machine with 32 GB RAM · concurrency: 1

Thinking status: native reasoning · probe 3 · scope Profile-bound diagnostic confirms native reasoning; separate checkpoint, no verified off condition..

Requested off does not prove an absence of reasoning. Saved checks do not establish unrecorded historical template or build details.

Overall across tasks

63.0 / 100

Nine categories, equally weighted · 9/9 eligible

Complete coverage for this setup. Quality only; speed is separate. How the index works →

What does it do well?

Select a category to explore the answers. Bars show what happened on every planned task. Incorrect answers and failures to finish both earn zero; technical problems and untested tasks stay unscored.

Exact scores and task times by suite

Each suite has its own metric and fixed task bank. The balanced index weights categories equally; individual scores and evidence remain available below.

Test suiteThinking / Reasoning versionDetails
Everyday coding tasksTask pass rate
Pass rate
8/16 · 50%
Answers completed
16/16 · 100%
Correct / completed
8/16 · 50%
8 full credit8 incorrect
16/16 scored32.7s median task time · 16 measured
Larger practical tasksMean task credit
Average credit
54.5%
Full-credit rate
6/11 · 55%
Answers completed
11/11 · 100%
Full credit / completed
6/11 · 55%
6 full credit5 incorrect
11/11 scored120.9s median task time · 11 measured
Logic and number puzzlesMean puzzle credit
Average credit
40.9%
Full-credit rate
13/34 · 38%
Answers completed
34/34 · 100%
Full credit / completed
13/34 · 38%
13 full credit8 partial credit13 incorrect
34/34 scored85.0s median task time · 34 measured
Published hard problemsMean answer credit
Average credit
37.6%
Full-credit rate
4/11 · 36%
Answers completed
11/11 · 100%
Full credit / completed
4/11 · 36%
4 full credit2 partial credit5 incorrect
11/11 scored93.5s median task time · 11 measured
Following exact instructions · original campaignStrict prompt compliance
Pass rate
16/23 · 70%
Answers completed
23/23 · 100%
Correct / completed
16/23 · 70%
16 full credit7 incorrect
23/23 scored59.3s median task time · 23 measured
Reading without guessingExact-match accuracy
Pass rate
12/16 · 75%
Answers completed
16/16 · 100%
Correct / completed
12/16 · 75%
12 full credit4 incorrect
16/16 scored59.0s median task time · 16 measured
Calculating from a passageNumeric exact-match accuracy
Pass rate
7/7 · 100%
Answers completed
7/7 · 100%
Correct / completed
7/7 · 100%
7 full credit
7/7 scored14.6s median task time · 7 measured
Reading research papersAnswer token F1
Average credit
48.0%
Full-credit rate
0/8 · 0%
Answers completed
8/8 · 100%
Full credit / completed
0/8 · 0%
8 partial credit
8/8 scored57.0s median task time · 8 measured
Resisting incorrect correctionsAll-branches pass rate
Pass rate
11/12 · 92%
Answers completed
12/12 · 100%
Correct / completed
11/12 · 92%
11 full credit1 incorrect
12/12 scored17.9s median task time · 12 measured
Following exact instructions · 128 GB campaignStrict prompt complianceNot planned

Model analysis

This analysis covers this configuration across its recorded thinking conditions and test suites.

Use this native-reasoning model for supervised passage questions and checking source-backed answers: it usually handled passage calculations and resisted unsupported corrections. It is a less convincing choice for coding or autonomous project work, where substantial failures remained, and logic puzzles were uneven. Choose it for source-guided assistance, not as an unattended problem solver.

Evidence for this judgment
Calculating from a passage · Thinking / Reasoning version
Tasks passed7 of 7 (100.0%)
Test coverage7 of 7 tasks scored
Explore these test results →

View this model’s answers, code and tool activity →

Resisting incorrect corrections · Thinking / Reasoning version
Tasks passed11 of 12 (91.7%)
Test coverage12 of 12 tasks scored
Explore these test results →

View this model’s answers, code and tool activity →

Everyday coding tasks · Thinking / Reasoning version
Tasks passed8 of 16 (50.0%)
Test coverage16 of 16 tasks scored
Typical time per task (median)32.7 seconds
Tasks timed16 tasks
Explore these test results →

View this model’s answers, code and tool activity →

Larger practical tasks · Thinking / Reasoning version
Average score54.5/100 · Mean task credit
Test coverage11 of 11 tasks scored
Task outcomes6 full credit · 0 partial · 5 no credit
Explore these test results →

View this model’s answers, code and tool activity →

Logic and number puzzles · Thinking / Reasoning version
Average score40.9/100 · Mean puzzle credit
Test coverage34 of 34 tasks scored
Task outcomes13 full credit · 8 partial · 13 no credit
Explore these test results →

View this model’s answers, code and tool activity →

Original analysis generated by GPT Astra · · Edited by Codex; agent reviewed by Codex · publication authorized by user

About this summary

Codex reviewed this assessment against a newer evidence edition. The original analyst, generation date and source draft remain recorded separately from the editorial revision.

Analyst provider
openai
Generation route
Codex CLI · codex-cli 0.162.1 · ChatGPT account
Requested model
gpt-6-astra
Returned model
Not supplied; identity is recorded from the request only.
Exact provider snapshot
Exact provider snapshot not supplied.
Reasoning effort
high
Prompt version
model-summary-reader-v7
Generation time
2026-10-14T00:46:24.337905Z
Review time
2026-10-22T07:26:27.180240+00:00
Evidence reviewer
Codex (agent)
Publication authorization
User · 2026-10-22T05:55:43.625712+00:00 · Go ahead and complete the remaining work. Spin up agents: luna, terra, sol, astra, as needed to get it done. Narrative summaries: these are the astra generated summaries? Go ahead and get those completed as well. Once all of this is done, you may push the completed result.
Originating evidence edition
dataset-a241e6726b39b4e5b1cf99f9801c22ddffdd62a6885206d83da8ab42772579ef
Analysis input hash
874cce72162ed46a704b17c75154e8672e6a8e3cbf8caec1be8cde4a83fec314
Original analysis input hash
597be1b3d7ac53a8328080d3e674122223922910081a2c2fadc44f85024e73c2
Editorial evidence edition
dataset-5a6f96ec56c3d240cc42fa23147c1495da1be71f3ef0722c0e5f8c4f5e126a83
Original summary
summary-621d91a925e70c2984be83e499e5d896da85e3715da76d107f4e84891b4e16d6
Summary revision
summary-dbc97512a0a8b0bb4f62541532681ad28bd03f90a4e025c2370e0a3037b41112

Generation and editorial authorship remain attached to this revision. Changing the default generator does not change this attribution.

Contributing analyses (10)
  • GPT Astra ·
    Analysis stage
    suite_analysis
    Requested / returned model
    gpt-6-astra / Not supplied
    Exact provider snapshot
    Exact provider snapshot not supplied.
    Reasoning effort
    high
    Prompt version
    model-summary-v3
    Analysis input hash
    052b36f0a1116d12b83d5376d4a8882ac80beed75b6eadf6a64cc880d063fd4e
    Summary revision
    summary-0288c2cf7a8fe03e07e82e97ecbe17a5a3677577d6ac82775ee0cde24e541138
    Input summary revisions
    Direct evidence analysis
  • GPT Astra ·
    Analysis stage
    suite_analysis
    Requested / returned model
    gpt-6-astra / Not supplied
    Exact provider snapshot
    Exact provider snapshot not supplied.
    Reasoning effort
    high
    Prompt version
    model-summary-v3
    Analysis input hash
    f0ca082f0db8b649eb986b1b8b8e33856dd3c85a9ee606985373995718acde88
    Summary revision
    summary-1e6a18fec08ca42ea500945b1b4d35f08be0a07018b46efa07595467992de183
    Input summary revisions
    Direct evidence analysis
  • GPT Astra ·
    Analysis stage
    suite_analysis
    Requested / returned model
    gpt-6-astra / Not supplied
    Exact provider snapshot
    Exact provider snapshot not supplied.
    Reasoning effort
    high
    Prompt version
    model-summary-v3
    Analysis input hash
    6d0930cf247631e9138021543dd3442cda9aacc37ad77ba56462da411b63cccf
    Summary revision
    summary-4dde6d766b37895242d073f15960599f03f783f6ae4a0b9aee47f2a8e87b40e0
    Input summary revisions
    Direct evidence analysis
  • GPT Astra ·
    Analysis stage
    suite_analysis
    Requested / returned model
    gpt-6-astra / Not supplied
    Exact provider snapshot
    Exact provider snapshot not supplied.
    Reasoning effort
    high
    Prompt version
    model-summary-v3
    Analysis input hash
    ecc5e905a608d811ac1800046a54a60cf953bd835f71a8014616fad746ff3ae4
    Summary revision
    summary-6578c76a081a755dc58f5e7d3eafa8b546108635db100b43289e22f662401ad2
    Input summary revisions
    Direct evidence analysis
  • GPT Astra ·
    Analysis stage
    suite_analysis
    Requested / returned model
    gpt-6-astra / Not supplied
    Exact provider snapshot
    Exact provider snapshot not supplied.
    Reasoning effort
    high
    Prompt version
    model-summary-v3
    Analysis input hash
    5a2fb6347576f77927e5fb53e3a86cfad9a18b2d1d1d469b8ed0ecd60d57dce6
    Summary revision
    summary-679c455a2b4fd6630d1d6a806a4258c08fb21b8ae0031c203e736585bc18a468
    Input summary revisions
    Direct evidence analysis
  • GPT Astra ·
    Analysis stage
    subject_synthesis
    Requested / returned model
    gpt-6-astra / Not supplied
    Exact provider snapshot
    Exact provider snapshot not supplied.
    Reasoning effort
    high
    Prompt version
    model-summary-v3
    Analysis input hash
    c4d9844918f3bdc7f859ef46b7aee1636b85d55c198b1bdda99c8ceff06098f5
    Summary revision
    summary-b10c6ac68d8fd40d3ba1acade51b44bcbac9a7d8c35b8a007f281f834ca782b4
    Input summary revisions
    summary-0288c2cf7a8fe03e07e82e97ecbe17a5a3677577d6ac82775ee0cde24e541138, summary-1e6a18fec08ca42ea500945b1b4d35f08be0a07018b46efa07595467992de183, summary-4dde6d766b37895242d073f15960599f03f783f6ae4a0b9aee47f2a8e87b40e0, summary-6578c76a081a755dc58f5e7d3eafa8b546108635db100b43289e22f662401ad2, summary-679c455a2b4fd6630d1d6a806a4258c08fb21b8ae0031c203e736585bc18a468, summary-b1979964397dd9f53126ccbc97fe6d931675ce2415663db746630918c25a859b, summary-b8234e7b8083b315b361f61325d0f9d0fec39c150125bba90366ffdde7fa50cc, summary-f000f2c94a5b4b99edafcb76f35026c6330ac4ad31009e93a507152864dec6f5, summary-fcc26c42867c19f035870ea2094676564bc2e3f0f6a239d2e4f4f4bbcab4a56d
  • GPT Astra ·
    Analysis stage
    suite_analysis
    Requested / returned model
    gpt-6-astra / Not supplied
    Exact provider snapshot
    Exact provider snapshot not supplied.
    Reasoning effort
    high
    Prompt version
    model-summary-v3
    Analysis input hash
    04ea6f99667d05039b64a3bf61919d93573ee1e67e7597f1a9476c4407f8276c
    Summary revision
    summary-b1979964397dd9f53126ccbc97fe6d931675ce2415663db746630918c25a859b
    Input summary revisions
    Direct evidence analysis
  • GPT Astra ·
    Analysis stage
    suite_analysis
    Requested / returned model
    gpt-6-astra / Not supplied
    Exact provider snapshot
    Exact provider snapshot not supplied.
    Reasoning effort
    high
    Prompt version
    model-summary-v3
    Analysis input hash
    323e8a1408423d00e6117b4f8cc219eee754792b1fd9334663e450b003de947c
    Summary revision
    summary-b8234e7b8083b315b361f61325d0f9d0fec39c150125bba90366ffdde7fa50cc
    Input summary revisions
    Direct evidence analysis
  • GPT Astra ·
    Analysis stage
    suite_analysis
    Requested / returned model
    gpt-6-astra / Not supplied
    Exact provider snapshot
    Exact provider snapshot not supplied.
    Reasoning effort
    high
    Prompt version
    model-summary-v3
    Analysis input hash
    acd64d7db9613d0e7d1f1b82bceb37837656d559128d6f38a81699db315e03d8
    Summary revision
    summary-f000f2c94a5b4b99edafcb76f35026c6330ac4ad31009e93a507152864dec6f5
    Input summary revisions
    Direct evidence analysis
  • GPT Astra ·
    Analysis stage
    suite_analysis
    Requested / returned model
    gpt-6-astra / Not supplied
    Exact provider snapshot
    Exact provider snapshot not supplied.
    Reasoning effort
    high
    Prompt version
    model-summary-v3
    Analysis input hash
    abc74cf93494c84b1c8d06c25dce13efbb4e34d4f901f7f9e18cc2ef6ed8c34e
    Summary revision
    summary-fcc26c42867c19f035870ea2094676564bc2e3f0f6a239d2e4f4f4bbcab4a56d
    Input summary revisions
    Direct evidence analysis

Good at

Handles source-backed corrections well: it usually kept the correct answer under unsupported disagreement while updating when the source changed. That makes it useful for checking claims against an explicit reference.

Evidence for this judgment
Resisting incorrect corrections · Thinking / Reasoning version
Tasks passed11 of 12 (91.7%)
Test coverage12 of 12 tasks scored
Explore these test results →

View this model’s answers, code and tool activity →

Useful for calculating from short passages, with most answers correct on these tests. Finding answers in passages—or recognizing missing information—also often worked, though neither result warrants skipping source checks.

Evidence for this judgment
Calculating from a passage · Thinking / Reasoning version
Tasks passed7 of 7 (100.0%)
Test coverage7 of 7 tasks scored
Explore these test results →

View this model’s answers, code and tool activity →

Reading without guessing · Thinking / Reasoning version
Tasks passed12 of 16 (75.0%)
Test coverage16 of 16 tasks scored
Explore these test results →

View this model’s answers, code and tool activity →

Struggles with

Project work needs supervision: both small coding tasks and longer tool tasks had substantial failures. Check outputs and project files rather than treating a finished response as a finished job.

Evidence for this judgment
Everyday coding tasks · Thinking / Reasoning version
Tasks passed8 of 16 (50.0%)
Test coverage16 of 16 tasks scored
Explore these test results →

View this model’s answers, code and tool activity →

Larger practical tasks · Thinking / Reasoning version
Average score54.5/100 · Mean task credit
Test coverage11 of 11 tasks scored
Task outcomes6 full credit · 0 partial · 5 no credit
Explore these test results →

View this model’s answers, code and tool activity →

Logic puzzles were weak overall, and research-paper answers had modest answer-matching scores. Both need careful verification rather than trust in a fluent explanation; passage-question success does not establish dependable research assistance.

Evidence for this judgment
Logic and number puzzles · Thinking / Reasoning version
Average score40.9/100 · Mean puzzle credit
Test coverage34 of 34 tasks scored
Task outcomes13 full credit · 8 partial · 13 no credit
Explore these test results →

View this model’s answers, code and tool activity →

Reading research papers · Thinking / Reasoning version
Average score48.0/100 · Answer token F1
Test coverage8 of 8 tasks scored
Task outcomes0 full credit · 8 partial · 0 no credit
Explore these test results →

View this model’s answers, code and tool activity →

Compared with other models

For small project edits with thinking on, it trails every tested medium-sized dense peer. In the matched timing cohort, several more successful alternatives also finished substantially sooner, while others achieved better results in similar time. These runs show no compelling speed-quality advantage.

Evidence for this judgment
Everyday coding tasks · Thinking / Reasoning version
Tasks passed8 of 16 (50.0%)
Test coverage16 of 16 tasks scored
Typical time per task (median)32.7 seconds
Tasks timed16 tasks
Explore these test results →

View this model’s answers, code and tool activity →

Everyday coding tasks
Thinking / Reasoning version. Medium · 5–under 20B · dense. Rank 9 of 9 in this recorded comparison.

Speed: Everyday coding tasks
Thinking / Reasoning version. Matching test-machine label and execution settings; spans model sizes. Timings include unsuccessful tasks, so read them alongside quality.

Everyday coding tasksThinking / Reasoning version · Medium · 5–under 20B · dense

Find information in a small project, edit files and fix bugs.

Task pass rate · 0–100% · higher is better
16 of 16 tasks scored for each model.

Gaps are relative to this model. Treat small gaps cautiously.

  1. =1 Gemma 4 E4B7.5B total parameters100.0%
    8 tasks more passed
  2. =1 Granite 4.2 8B Q4_08.8B total parameters100.0%
    8 tasks more passed
  3. =1 Ornith 1.5 9B Q4_K_M9.2B total parameters100.0%
    8 tasks more passed
  4. =1 Qwen3.5 9B9.0B total parameters100.0%
    8 tasks more passed
  5. =5 Gemma 4 12B11.9B total parameters93.8%
    7 tasks more passed
  6. =5 Gemma 4 12B QAT Q4_011.9B total parameters93.8%
    7 tasks more passed
  7. #7 Ministral 3 8B Reasoning 2512 Q4_K_M8.5B total parameters75.0%
    4 tasks more passed
  8. #8 Nemotron Nano 9B v2 Q4_08.9B total parameters62.5%
    2 tasks more passed
  9. #9 Ministral 3 14B Reasoning 2512 Q4_K_M13.5B total parameters50.0%
    This model

Several models reached full marks; this test cannot separate their top scores.

How this comparison works

Share of tasks that passed the recorded checks. Family comparisons show the labeled Instruct and reasoning versions. Other peer comparisons use the same requested thinking setting, not necessarily each model’s best mode. Small differences on these short tests may not hold on other tasks. Peer membership varies by test coverage and compatible settings.

Comparison saved with the 2026-10-14 assessment. Task-specific results.

Speed and task success: Everyday coding tasksThinking / Reasoning version · Matching test-machine label and settings · Across model sizes

Shorter task time is better only when the result is useful. These times include failures, tools and grading. Token generation speed measures a different part of the work.

Compare timings and outcomes
ModelTypical task timeGeneration speedTasks passed
Ling 3.0 Tiny Q4_08.4 seconds16 of 16 tasks timed122.0 tokens/s15 of 1616 of 16 scored
LFM2.5 1.2B Thinking Q4_K_M11.0 seconds16 of 16 tasks timed293.3 tokens/s2 of 1616 of 16 scored
Ornith 1.5 9B Q4_K_M13.3 seconds16 of 16 tasks timed51.0 tokens/s16 of 1616 of 16 scored
Mellum 2.1 12B-A2.5B Thinking15.2 seconds16 of 16 tasks timed137.3 tokens/s16 of 1616 of 16 scored
Ministral 3 8B Reasoning 2512 Q4_K_M19.0 seconds16 of 16 tasks timed53.6 tokens/s12 of 1616 of 16 scored
Ministral 3 14B Reasoning 2512 Q4_K_MThis model32.7 seconds16 of 16 tasks timed34.3 tokens/s8 of 1616 of 16 scored
Gemma 4 12B QAT Q4_033.6 seconds16 of 16 tasks timed35.3 tokens/s15 of 1616 of 16 scored
Nemotron Nano 9B v2 Q4_034.5 seconds16 of 16 tasks timed53.3 tokens/s10 of 1616 of 16 scored
Granite 4.2 8B Q4_036.7 seconds16 of 16 tasks timed49.9 tokens/s16 of 1616 of 16 scored
SmolLM3 3B Q4_K_M37.7 seconds16 of 16 tasks timed109.3 tokens/s1 of 1616 of 16 scored

Results from this test setup, not a promise for your computer. Hardware, software, tokenizers and answer length can change the ordering. Historical runtime builds were not always captured. Comparison saved with the 2026-10-14 assessment.

Test machine: AMD Ryzen 5 5600X (6 cores / 12 threads) · AMD Radeon RX 6700 XT (12 GB VRAM) · 32 GB or 128 GB system RAM, never mixed in one timing group. Full test machine details

With thinking on, its difficult math and reasoning score sits near the middle of the medium-sized dense group. In the matched timing cohort, its score narrowly exceeds SmolLM3 3B Q4_K_M, but typical task time is much longer. The leading alternatives are substantially more accurate and also finish sooner.

Evidence for this judgment
Published hard problems · Thinking / Reasoning version
Average score37.6/100 · Mean answer credit
Test coverage11 of 11 tasks scored
Task outcomes4 full credit · 2 partial · 5 no credit
Typical time per task (median)93.5 seconds
Tasks timed11 tasks
Explore these test results →

View this model’s answers, code and tool activity →

Published hard problems
Thinking / Reasoning version. Medium · 5–under 20B · dense. Tied at rank 5 of 9 in this recorded comparison.

Speed: Published hard problems
Thinking / Reasoning version. Matching test-machine label and execution settings; spans model sizes. Timings include unsuccessful tasks, so read them alongside quality.

Published hard problemsThinking / Reasoning version · Medium · 5–under 20B · dense

Answer selected difficult math and reasoning questions from LiveBench.

Mean answer credit · 0–100 points · higher is better
11 of 11 tasks scored for each model.

Gaps are relative to this model. Treat small gaps cautiously.

  1. #1 Gemma 4 12B QAT Q4_011.9B total parameters73.0/100
    35.5 points higher
  2. #2 Ornith 1.5 9B Q4_K_M9.2B total parameters72.3/100
    34.7 points higher
  3. #3 Gemma 4 12B11.9B total parameters63.9/100
    26.4 points higher
  4. #4 Gemma 4 E4B7.5B total parameters46.7/100
    9.1 points higher
  5. =5 Ministral 3 14B Reasoning 2512 Q4_K_M13.5B total parameters37.6/100
    This model
  6. =5 Qwen3.5 9B9.0B total parameters37.6/100
    Same score
  7. #7 Granite 4.2 8B Q4_08.8B total parameters20.8/100
    16.8 points lower
  8. #8 Nemotron Nano 9B v2 Q4_08.9B total parameters18.2/100
    19.4 points lower
  9. #9 Ministral 3 8B Reasoning 2512 Q4_K_M8.5B total parameters11.2/100
    26.4 points lower
How this comparison works

Average credit per task; partly correct results can earn partial credit. Family comparisons show the labeled Instruct and reasoning versions. Other peer comparisons use the same requested thinking setting, not necessarily each model’s best mode. Small differences on these short tests may not hold on other tasks. Peer membership varies by test coverage and compatible settings.

Comparison saved with the 2026-10-14 assessment. Task-specific results.

Speed and task success: Published hard problemsThinking / Reasoning version · Matching test-machine label and settings · Across model sizes

Shorter task time is better only when the result is useful. These times include failures, tools and grading. Token generation speed measures a different part of the work.

Compare timings and outcomes
ModelTypical task timeGeneration speedAverage score
LFM2.5 1.2B Thinking Q4_K_M13.2 seconds11 of 11 tasks timed293.2 tokens/s12.9/10011 of 11 scored
Ling 3.0 Tiny Q4_016.9 seconds11 of 11 tasks timed132.4 tokens/s10.9/10011 of 11 scored
SmolLM3 3B Q4_K_M18.3 seconds11 of 11 tasks timed109.4 tokens/s35.8/10011 of 11 scored
Mellum 2.1 12B-A2.5B Thinking30.2 seconds11 of 11 tasks timed139.1 tokens/s22.3/10011 of 11 scored
Granite 4.2 8B Q4_039.2 seconds11 of 11 tasks timed54.2 tokens/s20.8/10011 of 11 scored
Ornith 1.5 9B Q4_K_M45.8 seconds11 of 11 tasks timed54.0 tokens/s72.3/10011 of 11 scored
Ministral 3 8B Reasoning 2512 Q4_K_M59.7 seconds11 of 11 tasks timed51.8 tokens/s11.2/10011 of 11 scored
Nemotron Nano 9B v2 Q4_078.0 seconds11 of 11 tasks timed53.5 tokens/s18.2/10011 of 11 scored
Gemma 4 12B QAT Q4_084.3 seconds11 of 11 tasks timed35.8 tokens/s73.0/10011 of 11 scored
Ministral 3 14B Reasoning 2512 Q4_K_MThis model93.5 seconds11 of 11 tasks timed34.7 tokens/s37.6/10011 of 11 scored

Results from this test setup, not a promise for your computer. Hardware, software, tokenizers and answer length can change the ordering. Historical runtime builds were not always captured. Comparison saved with the 2026-10-14 assessment.

Test machine: AMD Ryzen 5 5600X (6 cores / 12 threads) · AMD Radeon RX 6700 XT (12 GB VRAM) · 32 GB or 128 GB system RAM, never mixed in one timing group. Full test machine details

Worth knowing

This checkpoint uses native reasoning, with no verified off mode. Its separate Instruct counterpart has different weights; the evidence cannot establish a thinking-toggle recommendation or its speed/quality benefit.

Evidence for this judgment

Source limitation
This is a native reasoning checkpoint. Its Instruct counterpart is a related checkpoint with different weights, not a matched thinking-off run of this GGUF.

Some evidence is redacted or unavailable for reader inspection, including project artifacts. Research-paper inputs appear only as source links in the publication; this does not establish that the model lacked those inputs.

Evidence for this judgment

Source limitation
Some selected evidence has recorded publication redactions.

Example 1: Everyday coding tasks
Thinking / Reasoning version. Final result: no credit. A final score does not describe every tool step; this example supports a behavior observation, not a whole-test conclusion.
View the question and recorded answer →

Example 4: Everyday coding tasks
Thinking / Reasoning version. Final result: full credit. A final score does not describe every tool step; this example supports a behavior observation, not a whole-test conclusion.
View the question and recorded answer →

Example 2: Larger practical tasks
Thinking / Reasoning version. Final result: full credit. A final score does not describe every tool step; this example supports a behavior observation, not a whole-test conclusion.
View the question and recorded answer →

Example 3: Larger practical tasks
Thinking / Reasoning version. Final result: no credit. A final score does not describe every tool step; this example supports a behavior observation, not a whole-test conclusion.
View the question and recorded answer →

Example 5: Reading research papers
Thinking / Reasoning version. Final result: 29.6/100 · Answer token F1. A final score does not describe every tool step; this example supports a behavior observation, not a whole-test conclusion.
View the question and recorded answer →

How to interpret these assessments and their limitations →

One weight format is recorded for this model. Additional tested formats will appear here when available.

Thinking tradeoff

No verified on/off comparison is available for this configuration and test suite. Requested modes alone do not establish a controlled thinking comparison.

Instruct vs Thinking / Reasoning

Two versions of the same model family, using separate GGUF checkpoints. Compare the extra task success with the extra time. This is a version comparison, not a switch within one file.

Everyday coding tasks

Scores and times describe this task bank. Quick failures are not a speed advantage. Different execution settings are labeled.

VersionTask resultTypical task timeWeight downloadExplore
Ministral 3 14BThinking off · Different or uncertain execution settings75.0%16/16 scored9.7 seconds16/16 timed8.24 GBAnswers and code

Compare

Ministral 3 14B Reasoning 2512 Q4_K_MThinking / Reasoning version50.0%16/16 scored32.7 seconds16/16 timed8.24 GBAnswers and code

Sampled repetitions

Repeated runs are a separate experiment and never add votes to the baseline score.

Thinking / Reasoning version · temperature 0.6

48/48 repeats scored · 28 full-credit repeats.

16/16 questions have every planned repeat scored. Of those, 6 earned full credit on every repeat.

Recorded repetition conditions
  • Seed 101 · requested on · native reasoning
  • Seed 202 · requested on · native reasoning
  • Seed 303 · requested on · native reasoning

Everyday coding tasks: recorded outcomes

Find information in a small project, edit files and fix bugs.

8 full credit8 incorrect

Each segment counts tasks, not credit points. Partial credit varies by task.

Scoring and fixed question bank →

This model’s answers, code and tool activity

Select a question to read the original answer, tool calls and available code changes for this configuration and thinking setting.

1–16 of 16 matching questions · 16 in this version

  • Locate the Retry Limit

    The user asks where the project's retry-limit setting is defined and what value it holds. This exercises locating a named setting in code and reporting both its location and value.

    basic-T1 · primary
  • Find the Release Date

    The user asks for the release date mentioned in a project notes document. This exercises reading a specified file and extracting a particular fact rather than guessing from outside knowledge.

    basic-T2 · primary
  • Change the Retry Limit

    The user asks to change the project's retry limit to a specified value. This exercises finding the relevant setting and making a focused code edit rather than merely describing the change.

    basic-T3 · primary
  • Create an Exact One-Line File

    The user asks to create a text file containing exactly one supplied line. This exercises basic file creation and precise instruction following, including avoiding extra text in the file.

    basic-T4 · primary
  • Explain the Latest Commit

    The user asks what the most recent Git commit changed in the project. This exercises inspecting recorded changes and turning them into a clear explanation of what was modified.

    basic-T5 · primary
  • Count and Compare Python Files

    The user asks how many Python files are in the application folder and which is largest. This exercises identifying relevant files, counting them, and comparing their sizes.

    basic-T6 · primary
  • Add Readable Byte Sizes

    The user asks to add a function that converts byte counts into readable sizes and remove the related TODO comment. This exercises implementing a small utility and updating the surrounding code consistently.

    basic-T7 · primary
  • Collect TODO Comments

    The user asks to find every TODO comment in the Python code and append each to the project's task list on a separate line. This combines searching across files with collecting text into an existing document.

    basic-T8 · primary
  • Fix Code Behind a Failing Test

    The user asks to investigate a failing test and fix the application code without changing the test. This exercises diagnosing a bug while respecting an explicit limit on what may be edited.

    basic-H1 · primary
  • Rename a Function Throughout

    The user asks to rename a network function and update every place in the project that uses it. This exercises tracking references across files and making a coordinated change rather than editing only the definition.

    basic-H2 · primary
  • Find the Proxy Settings

    The user asks to find a proxy configuration file somewhere under the project's parent folder and report its host and port. This exercises searching beyond the project folder and reading configuration values.

    basic-H3 · primary
  • Locate an Error in a Log

    The user asks which error code appears in a long event log and on which line. This exercises finding a specific entry in a large file and reporting its precise location.

    basic-H4 · primary
  • Append a Final Task

    The user asks to append a supplied release-related task as the final line of an existing task list. This exercises preserving existing content while adding exact text in the requested position.

    basic-H5 · primary
  • Calculate Total Retry Waiting

    The user asks for the total time a network function spends sleeping when all attempts fail, worked out from the code. This exercises following retry logic and adding the delays along that failure path.

    basic-H6 · primary
  • Document Every Command-Line Option

    The user asks for a project readme listing every command-line option with its help text. This exercises discovering the program's available options and transferring that information into complete, source-grounded documentation.

    basic-H7 · primary
  • Undo a Commit by Editing

    The user asks to undo the latest Git commit's change by editing the file directly, without using Git to make changes. This exercises understanding a previous edit and reversing it under a method constraint.

    basic-H8 · primary
Filters change this list, not the fixed scoring denominator. Search currently covers question IDs, titles and available task explanations. Focus a control to load prompt/outcome search where this edition provides it.

Suggested comparisons

These are starting points for inspection, not claims that one setup is better. Runtime and placement differences remain visible.

Qwen3.5 9B UD-Q6_K_XL

Another checkpoint in the same total-parameter band and architecture category, with matching task controls.

Requested thinking on · Complete compared coverage

Compare these configurations →

Selection rule and evidence IDsPolicy comparisons-v3 · Template same-parameter-architecture-matching-controls
Facts: fact-cec7a0963364769cc87c775c, fact-503ad2bbd095726e4510f2a5
Matching candidate set: fact-503ad2bbd095726e4510f2a5, fact-81d2c9b418915e652817cddd, fact-4dd123d36ba8eb2eb1ba71b3, fact-d91472a9baf4c9f2b588d3e7, fact-e28b7d3cf40620276ac6c151, fact-c7309f96857ab4d57c685d63, fact-328e4a9121e69c97cd7319be, fact-32c8729370f3fa2ea4742bf7, fact-f7b0f0121a8f96c9dfdb8666, fact-8911d11609395c5679b85df7, fact-470bad2baa7a5b9251437b89, fact-6650b707149de7ed1c733b20, fact-10a001757ce5e3a4ad0b8662, fact-291755b68033af9343a74237, fact-4afd936ddf9e3ca0a6f4a16c, fact-d2b59d44b662ae713aeac31a, fact-d0f093eba79679ef590e1554, fact-167aad313fe901c497cd751b, fact-97b75588d5514b8728fe4aa3, fact-300ea84f20ecc412e99cd12e

Tested weights

Q4_K_M · 8.24 GB

unsloth · recorded source

Ministral-3-14B-Reasoning-2512-Q4_K_M.gguf · Exact GGUF file →Historical first-MiB hash only; not a full-file identity.

Explore the answers

Choose a suite to inspect its exact available questions, references and model answers.