AgentBench / results
← All models

Ministral 3 8B Reasoning 2512 Q4_K_M

1 tested weight format · Explore results, then inspect the answers.

Choose a tested configuration

Get this tested model

Q4_K_M · 5.2 GB download

Links follow the selected quant and configuration.

Setup, memory & provenance · Q4_K_M

Q4_K_M · 5.2 GB of weights · vulkan · context 16384 · GPU layers 99

Medium · 5–under 20B · dense · 8.5B total

Memory fit has not been verified for this configuration.

completed

Recorded settings and thinking verification

Temperature 0 · seed 42 · output limit 4096 · reasoning effort unrecorded

RAM labels describe the test computer, not this model’s memory requirement. Speed comparisons keep the 32 GB and 128 GB runs separate.

Runtime build: unrecorded · Local · test machine with 32 GB RAM · concurrency: 1

Thinking status: native reasoning · probe 3 · scope Profile-bound diagnostic confirms native reasoning; separate checkpoint, no verified off condition..

Requested off does not prove an absence of reasoning. Saved checks do not establish unrecorded historical template or build details.

Overall across tasks

51.2 / 100

Nine categories, equally weighted · 9/9 eligible

Complete coverage for this setup. Quality only; speed is separate. How the index works →

What does it do well?

Select a category to explore the answers. Bars show what happened on every planned task. Incorrect answers and failures to finish both earn zero; technical problems and untested tasks stay unscored.

Exact scores and task times by suite

Each suite has its own metric and fixed task bank. The balanced index weights categories equally; individual scores and evidence remain available below.

Test suiteThinking / Reasoning versionDetails
Everyday coding tasksTask pass rate
Pass rate
12/16 · 75%
Answers completed
16/16 · 100%
Correct / completed
12/16 · 75%
12 full credit4 incorrect
16/16 scored19.0s median task time · 16 measured
Larger practical tasksMean task credit
Average credit
18.2%
Full-credit rate
2/11 · 18%
Answers completed
10/11 · 91%
Full credit / completed
2/10 · 20%
2 full credit8 incorrect1 failed to finish
11/11 scored90.8s median task time · 11 measured
Logic and number puzzlesMean puzzle credit
Average credit
37.8%
Full-credit rate
12/34 · 35%
Answers completed
34/34 · 100%
Full credit / completed
12/34 · 35%
12 full credit4 partial credit18 incorrect
34/34 scored60.9s median task time · 34 measured
Published hard problemsMean answer credit
Average credit
11.2%
Full-credit rate
1/11 · 9%
Answers completed
11/11 · 100%
Full credit / completed
1/11 · 9%
1 full credit2 partial credit8 incorrect
11/11 scored59.7s median task time · 11 measured
Following exact instructions · original campaignStrict prompt compliance
Pass rate
17/23 · 74%
Answers completed
23/23 · 100%
Correct / completed
17/23 · 74%
17 full credit6 incorrect
23/23 scored39.6s median task time · 23 measured
Reading without guessingExact-match accuracy
Pass rate
12/16 · 75%
Answers completed
16/16 · 100%
Correct / completed
12/16 · 75%
12 full credit4 incorrect
16/16 scored39.4s median task time · 16 measured
Calculating from a passageNumeric exact-match accuracy
Pass rate
7/7 · 100%
Answers completed
7/7 · 100%
Correct / completed
7/7 · 100%
7 full credit
7/7 scored40.1s median task time · 7 measured
Reading research papersAnswer token F1
Average credit
52.7%
Full-credit rate
1/8 · 13%
Answers completed
8/8 · 100%
Full credit / completed
1/8 · 13%
1 full credit7 partial credit
8/8 scored25.2s median task time · 8 measured
Resisting incorrect correctionsAll-branches pass rate
Pass rate
2/12 · 17%
Answers completed
12/12 · 100%
Correct / completed
2/12 · 17%
2 full credit10 incorrect
12/12 scored3.7s median task time · 12 measured
Following exact instructions · 128 GB campaignStrict prompt complianceNot planned

Model analysis

This analysis covers this configuration across its recorded thinking conditions and test suites.

Use this native-reasoning configuration for supervised small-project edits or calculations from a passage, not as an autonomous general assistant. Most small tasks succeeded, but longer tool-based work and handling challenged or updated sources were major weaknesses. Its separate Instruct counterpart is the better everyday-coding choice, matching task success with shorter task times; consider this checkpoint when its better passage-calculation results matter more than waiting.

Evidence for this judgment
Everyday coding tasks · Thinking / Reasoning version
Tasks passed12 of 16 (75.0%)
Test coverage16 of 16 tasks scored
Tasks timed16 tasks
Typical time per task (median)19.0 seconds
Explore these test results →

View this model’s answers, code and tool activity →

Calculating from a passage · Thinking / Reasoning version
Tasks passed7 of 7 (100.0%)
Test coverage7 of 7 tasks scored
Tasks timed7 tasks
Typical time per task (median)40.1 seconds
Explore these test results →

View this model’s answers, code and tool activity →

Larger practical tasks · Thinking / Reasoning version
Average score18.2/100 · Mean task credit
Test coverage11 of 11 tasks scored
Task outcomes2 full credit · 0 partial · 9 no credit
Explore these test results →

View this model’s answers, code and tool activity →

Resisting incorrect corrections · Thinking / Reasoning version
Tasks passed2 of 12 (16.7%)
Test coverage12 of 12 tasks scored
Explore these test results →

View this model’s answers, code and tool activity →

Everyday coding tasks
Thinking / Reasoning version. Ministral 3 8B · Instruct versus reasoning checkpoint. Tied at rank 1 of 2 in this recorded comparison.

Calculating from a passage
Thinking / Reasoning version. Ministral 3 8B · Instruct versus reasoning checkpoint. Rank 1 of 2 in this recorded comparison.

Original analysis generated by GPT Astra · · Edited by Codex; agent reviewed by Codex · publication authorized by user

About this summary

Codex reviewed this assessment against a newer evidence edition. The original analyst, generation date and source draft remain recorded separately from the editorial revision.

Analyst provider
openai
Generation route
Codex CLI · codex-cli 0.162.1 · ChatGPT account
Requested model
gpt-6-astra
Returned model
Not supplied; identity is recorded from the request only.
Exact provider snapshot
Exact provider snapshot not supplied.
Reasoning effort
high
Prompt version
model-summary-reader-v7
Generation time
2026-10-14T00:35:47.831451Z
Review time
2026-10-22T07:26:27.183412+00:00
Evidence reviewer
Codex (agent)
Publication authorization
User · 2026-10-22T05:55:43.625712+00:00 · Go ahead and complete the remaining work. Spin up agents: luna, terra, sol, astra, as needed to get it done. Narrative summaries: these are the astra generated summaries? Go ahead and get those completed as well. Once all of this is done, you may push the completed result.
Originating evidence edition
dataset-a241e6726b39b4e5b1cf99f9801c22ddffdd62a6885206d83da8ab42772579ef
Analysis input hash
aaca3d987542c37adbd0faa8a25b5fc3c27ec504cc0c1519aadcd2a5647fe2a5
Original analysis input hash
f2cc152251d275086c6d156f8089ec718532114000cfd608aee4717af0162f13
Editorial evidence edition
dataset-5a6f96ec56c3d240cc42fa23147c1495da1be71f3ef0722c0e5f8c4f5e126a83
Original summary
summary-e43d41f73817de1a0854e02cc67bddf001564726686305b242c7bac8c95008ad
Summary revision
summary-3e19a27bf64a781228fb5e047716161db1b5f75e4e6f4dff70c9ce752d81f493

Generation and editorial authorship remain attached to this revision. Changing the default generator does not change this attribution.

Contributing analyses (10)
  • GPT Astra ·
    Analysis stage
    suite_analysis
    Requested / returned model
    gpt-6-astra / Not supplied
    Exact provider snapshot
    Exact provider snapshot not supplied.
    Reasoning effort
    high
    Prompt version
    model-summary-v3
    Analysis input hash
    287628cde6a625b5bdc5cac1fe6575b2107a4e9dd9d1b45f39da514055eea6f1
    Summary revision
    summary-03d91b9c8049e0bc322d15dfa827e85feeadb8d94c667a1a85d8dec648a35f0c
    Input summary revisions
    Direct evidence analysis
  • GPT Astra ·
    Analysis stage
    suite_analysis
    Requested / returned model
    gpt-6-astra / Not supplied
    Exact provider snapshot
    Exact provider snapshot not supplied.
    Reasoning effort
    high
    Prompt version
    model-summary-v3
    Analysis input hash
    7833f222333507ca810147c824c647c58c3d80cbe1582d1a4d4c1a88c358ff5c
    Summary revision
    summary-0689cd6096cc7429a0f9990671c41c4f8c53e118c67d45911f4e4a6bb8c73f6b
    Input summary revisions
    Direct evidence analysis
  • GPT Astra ·
    Analysis stage
    suite_analysis
    Requested / returned model
    gpt-6-astra / Not supplied
    Exact provider snapshot
    Exact provider snapshot not supplied.
    Reasoning effort
    high
    Prompt version
    model-summary-v3
    Analysis input hash
    c8075c8287ea101932224f3fcfad7ccbd642569db0707997bbce50d521e2ef7e
    Summary revision
    summary-14929f93f72a467c39d3459a3abbed192fb24e874924c0f9f308a7da3b1feb9a
    Input summary revisions
    Direct evidence analysis
  • GPT Astra ·
    Analysis stage
    suite_analysis
    Requested / returned model
    gpt-6-astra / Not supplied
    Exact provider snapshot
    Exact provider snapshot not supplied.
    Reasoning effort
    high
    Prompt version
    model-summary-v3
    Analysis input hash
    ef2f9cb201df0bfbf2569d72df480c184e1ce164e0362b9f8e14b25a19e8d0c1
    Summary revision
    summary-29fa8edbb2e8ae6321428d65625d65cd3af0e92638045c4f2986a032d3a50aba
    Input summary revisions
    Direct evidence analysis
  • GPT Astra ·
    Analysis stage
    suite_analysis
    Requested / returned model
    gpt-6-astra / Not supplied
    Exact provider snapshot
    Exact provider snapshot not supplied.
    Reasoning effort
    high
    Prompt version
    model-summary-v3
    Analysis input hash
    9255a957326792aa6b97370e2959dea0d4ec728fcdcefb5a0191dd9f25a5a6bc
    Summary revision
    summary-5d06535eaf859c72f81f6bf400da572ba0d3957592be1e56296556dc4d3dd685
    Input summary revisions
    Direct evidence analysis
  • GPT Astra ·
    Analysis stage
    subject_synthesis
    Requested / returned model
    gpt-6-astra / Not supplied
    Exact provider snapshot
    Exact provider snapshot not supplied.
    Reasoning effort
    high
    Prompt version
    model-summary-v3
    Analysis input hash
    49f3e7fe7daf953c648120e5c51675d3cbb1e789c6633b5e37a7d777dbf1ec41
    Summary revision
    summary-78e9759f9c8057c493e5c0d07ec2f4715eeced647df6fd61509828283ba5ffc5
    Input summary revisions
    summary-03d91b9c8049e0bc322d15dfa827e85feeadb8d94c667a1a85d8dec648a35f0c, summary-0689cd6096cc7429a0f9990671c41c4f8c53e118c67d45911f4e4a6bb8c73f6b, summary-14929f93f72a467c39d3459a3abbed192fb24e874924c0f9f308a7da3b1feb9a, summary-29fa8edbb2e8ae6321428d65625d65cd3af0e92638045c4f2986a032d3a50aba, summary-5d06535eaf859c72f81f6bf400da572ba0d3957592be1e56296556dc4d3dd685, summary-7c2f8c5277ed3138d72b0e4a71e2637dc19053d83b651373c1f719dbd7edd77f, summary-87ed2912d38c375d675712b4f20b14dbaee7c0db9a449aaf73570f241575ad34, summary-e4adb2825d308fbe5aa44879d6c2c76f276931ff9dea16d8171129fe593dfb5d, summary-f78886c36b4d6d7087306d1f74682143363c8b820609bfae405c6d62e0982ab8
  • GPT Astra ·
    Analysis stage
    suite_analysis
    Requested / returned model
    gpt-6-astra / Not supplied
    Exact provider snapshot
    Exact provider snapshot not supplied.
    Reasoning effort
    high
    Prompt version
    model-summary-v3
    Analysis input hash
    63e19729a9750a2a50508627605a90d5af17892004ea6c4df14976a3792470d2
    Summary revision
    summary-7c2f8c5277ed3138d72b0e4a71e2637dc19053d83b651373c1f719dbd7edd77f
    Input summary revisions
    Direct evidence analysis
  • GPT Astra ·
    Analysis stage
    suite_analysis
    Requested / returned model
    gpt-6-astra / Not supplied
    Exact provider snapshot
    Exact provider snapshot not supplied.
    Reasoning effort
    high
    Prompt version
    model-summary-v3
    Analysis input hash
    088a33b636de6a4077f0de1a24b81de272b7b27106ffc266d9d8625471250481
    Summary revision
    summary-87ed2912d38c375d675712b4f20b14dbaee7c0db9a449aaf73570f241575ad34
    Input summary revisions
    Direct evidence analysis
  • GPT Astra ·
    Analysis stage
    suite_analysis
    Requested / returned model
    gpt-6-astra / Not supplied
    Exact provider snapshot
    Exact provider snapshot not supplied.
    Reasoning effort
    high
    Prompt version
    model-summary-v3
    Analysis input hash
    9cca582ff09dde16d8a13ea4e359993b075d70cb7e472a6bb4f43bdea7c05b0d
    Summary revision
    summary-e4adb2825d308fbe5aa44879d6c2c76f276931ff9dea16d8171129fe593dfb5d
    Input summary revisions
    Direct evidence analysis
  • GPT Astra ·
    Analysis stage
    suite_analysis
    Requested / returned model
    gpt-6-astra / Not supplied
    Exact provider snapshot
    Exact provider snapshot not supplied.
    Reasoning effort
    high
    Prompt version
    model-summary-v3
    Analysis input hash
    bf91d0191fc2a544aa44f535a6132246dd77567b12d82b470028052cf8a1bee6
    Summary revision
    summary-f78886c36b4d6d7087306d1f74682143363c8b820609bfae405c6d62e0982ab8
    Input summary revisions
    Direct evidence analysis

Good at

Handles calculations from passages well on this small test: all included answers matched exactly. That supports counting and comparing facts in text, while leaving broader reliability unestablished.

Evidence for this judgment
Calculating from a passage · Thinking / Reasoning version
Tasks passed7 of 7 (100.0%)
Test coverage7 of 7 tasks scored
Explore these test results →

View this model’s answers, code and tool activity →

Useful for small-project edits and bug fixes, with most tasks succeeding in the baseline run. Success did not consistently hold across repeated attempts, so keep tests and human oversight in the workflow.

Evidence for this judgment
Everyday coding tasks · Thinking / Reasoning version
Tasks passed12 of 16 (75.0%)
Test coverage16 of 16 tasks scored
Explore these test results →

View this model’s answers, code and tool activity →

Everyday coding tasks · Thinking on · sampled repetitions
Questions successful every repetition7 questions
Questions with all repetitions scored16 questions
Distinct questions planned16 questions
Repetitions earning full credit30 repetitions
Repetitions scored48 repetitions
Repetitions planned48 repetitions
Explore these test results →

View this model’s answers, code and tool activity →

Struggles with

Poor at keeping answers anchored to a source through challenges and source updates. Most tests failed the combined requirements, so verify both its corrections and its decisions not to change an answer.

Evidence for this judgment
Resisting incorrect corrections · Thinking / Reasoning version
Tasks passed2 of 12 (16.7%)
Test coverage12 of 12 tasks scored
Explore these test results →

View this model’s answers, code and tool activity →

Longer tasks with tools are a poor fit: few completed successfully. Difficult math and reasoning questions, plus logic and number puzzles, also produced weak results; the reasoning label does not establish broad problem-solving strength.

Evidence for this judgment
Larger practical tasks · Thinking / Reasoning version
Average score18.2/100 · Mean task credit
Test coverage11 of 11 tasks scored
Task outcomes2 full credit · 0 partial · 9 no credit
Explore these test results →

View this model’s answers, code and tool activity →

Published hard problems · Thinking / Reasoning version
Average score11.2/100 · Mean answer credit
Test coverage11 of 11 tasks scored
Task outcomes1 full credit · 2 partial · 8 no credit
Explore these test results →

View this model’s answers, code and tool activity →

Logic and number puzzles · Thinking / Reasoning version
Average score37.8/100 · Mean puzzle credit
Test coverage34 of 34 tasks scored
Task outcomes12 full credit · 4 partial · 18 no credit
Explore these test results →

View this model’s answers, code and tool activity →

Compared with other models

For passage calculations with thinking on, it ties the highest result in the tested medium-sized dense group. Equally successful alternatives in the matched timing cohort finished much sooner, and every timing peer had a shorter median task time. This checkpoint offers no responsiveness advantage for that use.

Evidence for this judgment
Calculating from a passage · Thinking / Reasoning version
Tasks passed7 of 7 (100.0%)
Test coverage7 of 7 tasks scored
Tasks timed7 tasks
Typical time per task (median)40.1 seconds
Explore these test results →

View this model’s answers, code and tool activity →

Calculating from a passage
Thinking / Reasoning version. Medium · 5–under 20B · dense. Tied at rank 1 of 9 in this recorded comparison.

Speed: Calculating from a passage
Thinking / Reasoning version. Matching test-machine label and execution settings; spans model sizes. Timings include unsuccessful tasks, so read them alongside quality.

Calculating from a passageThinking / Reasoning version · Medium · 5–under 20B · dense

Read a passage and count, compare or calculate using its facts.

Numeric exact-match accuracy · 0–100% · higher is better
7 of 7 tasks scored for each model.

Gaps are relative to this model. Treat small gaps cautiously.

  1. =1 Gemma 4 12B11.9B total parameters100.0%
    Same score
  2. =1 Gemma 4 E4B7.5B total parameters100.0%
    Same score
  3. =1 Ministral 3 14B Reasoning 2512 Q4_K_M13.5B total parameters100.0%
    Same score
  4. =1 Ministral 3 8B Reasoning 2512 Q4_K_M8.5B total parameters100.0%
    This model
  5. =1 Qwen3.5 9B9.0B total parameters100.0%
    Same score
  6. =6 Gemma 4 12B QAT Q4_011.9B total parameters85.7%
    1 task fewer passed
  7. =6 Granite 4.2 8B Q4_08.8B total parameters85.7%
    1 task fewer passed
  8. =6 Nemotron Nano 9B v2 Q4_08.9B total parameters85.7%
    1 task fewer passed
  9. =6 Ornith 1.5 9B Q4_K_M9.2B total parameters85.7%
    1 task fewer passed

Several models reached full marks; this test cannot separate their top scores.

How this comparison works

Share of tasks that passed the recorded checks. Family comparisons show the labeled Instruct and reasoning versions. Other peer comparisons use the same requested thinking setting, not necessarily each model’s best mode. Small differences on these short tests may not hold on other tasks. Peer membership varies by test coverage and compatible settings.

Comparison saved with the 2026-10-14 assessment. Task-specific results.

Speed and task success: Calculating from a passageThinking / Reasoning version · Matching test-machine label and settings · Across model sizes

Shorter task time is better only when the result is useful. These times include failures, tools and grading. Token generation speed measures a different part of the work.

Compare timings and outcomes
ModelTypical task timeGeneration speedTasks passed
Ling 3.0 Tiny Q4_01.5 seconds7 of 7 tasks timed138.4 tokens/s7 of 77 of 7 scored
Ornith 1.5 9B Q4_K_M3.0 seconds7 of 7 tasks timed54.7 tokens/s6 of 77 of 7 scored
SmolLM3 3B Q4_K_M3.5 seconds7 of 7 tasks timed121.9 tokens/s5 of 77 of 7 scored
Mellum 2.1 12B-A2.5B Thinking3.7 seconds7 of 7 tasks timed143.5 tokens/s7 of 77 of 7 scored
LFM2.5 1.2B Thinking Q4_K_M4.4 seconds7 of 7 tasks timed300.3 tokens/s4 of 77 of 7 scored
Nemotron Nano 9B v2 Q4_06.0 seconds7 of 7 tasks timed53.8 tokens/s6 of 77 of 7 scored
Gemma 4 12B QAT Q4_010.9 seconds7 of 7 tasks timed37.1 tokens/s6 of 77 of 7 scored
Ministral 3 14B Reasoning 2512 Q4_K_M14.6 seconds7 of 7 tasks timed35.6 tokens/s7 of 77 of 7 scored
Granite 4.2 8B Q4_018.9 seconds7 of 7 tasks timed56.1 tokens/s6 of 77 of 7 scored
Ministral 3 8B Reasoning 2512 Q4_K_MThis model40.1 seconds7 of 7 tasks timed53.5 tokens/s7 of 77 of 7 scored

Results from this test setup, not a promise for your computer. Hardware, software, tokenizers and answer length can change the ordering. Historical runtime builds were not always captured. Comparison saved with the 2026-10-14 assessment.

Test machine: AMD Ryzen 5 5600X (6 cores / 12 threads) · AMD Radeon RX 6700 XT (12 GB VRAM) · 32 GB or 128 GB system RAM, never mixed in one timing group. Full test machine details

Choose Ministral 3 8B Instruct Q4_K_M for everyday coding and straightforward passage questions: it matched coding success, answered more passage questions correctly, and finished both types much sooner in these runs. This means choosing a separate checkpoint. The native-reasoning checkpoint performed better on calculations from passages, a different task bank.

Evidence for this judgment

Everyday coding tasks
Thinking / Reasoning version. Ministral 3 8B · Instruct versus reasoning checkpoint. Tied at rank 1 of 2 in this recorded comparison.

Reading without guessing
Thinking / Reasoning version. Ministral 3 8B · Instruct versus reasoning checkpoint. Rank 2 of 2 in this recorded comparison.

Everyday coding tasks · Thinking / Reasoning version
Tasks passed12 of 16 (75.0%)
Test coverage16 of 16 tasks scored
Tasks timed16 tasks
Typical time per task (median)19.0 seconds
Explore these test results →

View this model’s answers, code and tool activity →

Reading without guessing · Thinking / Reasoning version
Tasks passed12 of 16 (75.0%)
Test coverage16 of 16 tasks scored
Tasks timed16 tasks
Typical time per task (median)39.4 seconds
Explore these test results →

View this model’s answers, code and tool activity →

Everyday coding tasksThinking / Reasoning version · Ministral 3 8B · Instruct versus reasoning checkpoint

Find information in a small project, edit files and fix bugs.

Task pass rate · 0–100% · higher is better
16 of 16 tasks scored for each model.

Gaps are relative to this model. Treat small gaps cautiously.

  1. =1 Ministral 3 8B Instruct Q4_K_M8.5B total parameters75.0%
    Same score
  2. =1 Ministral 3 8B Reasoning 2512 Q4_K_M8.5B total parameters75.0%
    This model
How this comparison works

Share of tasks that passed the recorded checks. Family comparisons show the labeled Instruct and reasoning versions. Other peer comparisons use the same requested thinking setting, not necessarily each model’s best mode. Small differences on these short tests may not hold on other tasks. Peer membership varies by test coverage and compatible settings.

Comparison saved with the 2026-10-14 assessment. Task-specific results.

Reading without guessingThinking / Reasoning version · Ministral 3 8B · Instruct versus reasoning checkpoint

Answer from a passage—or say that the passage does not contain the answer.

Exact-match accuracy · 0–100% · higher is better
16 of 16 tasks scored for each model.

Gaps are relative to this model. Treat small gaps cautiously.

  1. #1 Ministral 3 8B Instruct Q4_K_M8.5B total parameters87.5%
    2 tasks more passed
  2. #2 Ministral 3 8B Reasoning 2512 Q4_K_M8.5B total parameters75.0%
    This model
How this comparison works

Share of tasks that passed the recorded checks. Family comparisons show the labeled Instruct and reasoning versions. Other peer comparisons use the same requested thinking setting, not necessarily each model’s best mode. Small differences on these short tests may not hold on other tasks. Peer membership varies by test coverage and compatible settings.

Comparison saved with the 2026-10-14 assessment. Task-specific results.

Worth knowing

Native reasoning is verified, but this file has no verified thinking-off condition. Its Instruct counterpart uses different weights, so their differences cannot isolate the effect of a thinking toggle.

Evidence for this judgment

Source limitation
This is a native reasoning checkpoint. Its Instruct counterpart is a related checkpoint with different weights, not a matched thinking-off run of this GGUF.

Publication redactions, withheld practical-task artifacts and link-only paper exports limit reader inspection. These restrictions do not establish that the model lacked the source material.

Evidence for this judgment

Source limitation
Some selected evidence has recorded publication redactions.

How to interpret these assessments and their limitations →

One weight format is recorded for this model. Additional tested formats will appear here when available.

Thinking tradeoff

No verified on/off comparison is available for this configuration and test suite. Requested modes alone do not establish a controlled thinking comparison.

Instruct vs Thinking / Reasoning

Two versions of the same model family, using separate GGUF checkpoints. Compare the extra task success with the extra time. This is a version comparison, not a switch within one file.

Everyday coding tasks

Scores and times describe this task bank. Quick failures are not a speed advantage. Different execution settings are labeled.

VersionTask resultTypical task timeWeight downloadExplore
Ministral 3 8B Reasoning 2512 Q4_K_MThinking / Reasoning version75.0%16/16 scored19.0 seconds16/16 timed5.2 GBAnswers and code
Ministral 3 8B Instruct Q4_K_MThinking off75.0%16/16 scored8.6 seconds16/16 timed5.2 GBAnswers and code

Compare

Sampled repetitions

Repeated runs are a separate experiment and never add votes to the baseline score.

Thinking / Reasoning version · temperature 0.6

48/48 repeats scored · 30 full-credit repeats.

16/16 questions have every planned repeat scored. Of those, 7 earned full credit on every repeat.

Recorded repetition conditions
  • Seed 101 · requested on · native reasoning
  • Seed 303 · requested on · native reasoning
  • Seed 202 · requested on · native reasoning

Everyday coding tasks: recorded outcomes

Find information in a small project, edit files and fix bugs.

12 full credit4 incorrect

Each segment counts tasks, not credit points. Partial credit varies by task.

Scoring and fixed question bank →

This model’s answers, code and tool activity

Select a question to read the original answer, tool calls and available code changes for this configuration and thinking setting.

1–16 of 16 matching questions · 16 in this version

  • Locate the Retry Limit

    The user asks where the project's retry-limit setting is defined and what value it holds. This exercises locating a named setting in code and reporting both its location and value.

    basic-T1 · primary
  • Find the Release Date

    The user asks for the release date mentioned in a project notes document. This exercises reading a specified file and extracting a particular fact rather than guessing from outside knowledge.

    basic-T2 · primary
  • Change the Retry Limit

    The user asks to change the project's retry limit to a specified value. This exercises finding the relevant setting and making a focused code edit rather than merely describing the change.

    basic-T3 · primary
  • Create an Exact One-Line File

    The user asks to create a text file containing exactly one supplied line. This exercises basic file creation and precise instruction following, including avoiding extra text in the file.

    basic-T4 · primary
  • Explain the Latest Commit

    The user asks what the most recent Git commit changed in the project. This exercises inspecting recorded changes and turning them into a clear explanation of what was modified.

    basic-T5 · primary
  • Count and Compare Python Files

    The user asks how many Python files are in the application folder and which is largest. This exercises identifying relevant files, counting them, and comparing their sizes.

    basic-T6 · primary
  • Add Readable Byte Sizes

    The user asks to add a function that converts byte counts into readable sizes and remove the related TODO comment. This exercises implementing a small utility and updating the surrounding code consistently.

    basic-T7 · primary
  • Collect TODO Comments

    The user asks to find every TODO comment in the Python code and append each to the project's task list on a separate line. This combines searching across files with collecting text into an existing document.

    basic-T8 · primary
  • Fix Code Behind a Failing Test

    The user asks to investigate a failing test and fix the application code without changing the test. This exercises diagnosing a bug while respecting an explicit limit on what may be edited.

    basic-H1 · primary
  • Rename a Function Throughout

    The user asks to rename a network function and update every place in the project that uses it. This exercises tracking references across files and making a coordinated change rather than editing only the definition.

    basic-H2 · primary
  • Find the Proxy Settings

    The user asks to find a proxy configuration file somewhere under the project's parent folder and report its host and port. This exercises searching beyond the project folder and reading configuration values.

    basic-H3 · primary
  • Locate an Error in a Log

    The user asks which error code appears in a long event log and on which line. This exercises finding a specific entry in a large file and reporting its precise location.

    basic-H4 · primary
  • Append a Final Task

    The user asks to append a supplied release-related task as the final line of an existing task list. This exercises preserving existing content while adding exact text in the requested position.

    basic-H5 · primary
  • Calculate Total Retry Waiting

    The user asks for the total time a network function spends sleeping when all attempts fail, worked out from the code. This exercises following retry logic and adding the delays along that failure path.

    basic-H6 · primary
  • Document Every Command-Line Option

    The user asks for a project readme listing every command-line option with its help text. This exercises discovering the program's available options and transferring that information into complete, source-grounded documentation.

    basic-H7 · primary
  • Undo a Commit by Editing

    The user asks to undo the latest Git commit's change by editing the file directly, without using Git to make changes. This exercises understanding a previous edit and reversing it under a method constraint.

    basic-H8 · primary
Filters change this list, not the fixed scoring denominator. Search currently covers question IDs, titles and available task explanations. Focus a control to load prompt/outcome search where this edition provides it.

Suggested comparisons

These are starting points for inspection, not claims that one setup is better. Runtime and placement differences remain visible.

Nemotron Nano 9B v2 Q4_0

Another checkpoint in the same total-parameter band and architecture category, with matching task controls.

Requested thinking on · Complete compared coverage

Compare these configurations →

Selection rule and evidence IDsPolicy comparisons-v3 · Template same-parameter-architecture-matching-controls
Facts: fact-10a001757ce5e3a4ad0b8662, fact-6650b707149de7ed1c733b20
Matching candidate set: fact-6650b707149de7ed1c733b20, fact-291755b68033af9343a74237, fact-4afd936ddf9e3ca0a6f4a16c, fact-470bad2baa7a5b9251437b89, fact-8911d11609395c5679b85df7, fact-d2b59d44b662ae713aeac31a, fact-f7b0f0121a8f96c9dfdb8666, fact-d0f093eba79679ef590e1554, fact-328e4a9121e69c97cd7319be, fact-e28b7d3cf40620276ac6c151, fact-d91472a9baf4c9f2b588d3e7, fact-4dd123d36ba8eb2eb1ba71b3, fact-81d2c9b418915e652817cddd, fact-cec7a0963364769cc87c775c, fact-503ad2bbd095726e4510f2a5, fact-c7309f96857ab4d57c685d63, fact-32c8729370f3fa2ea4742bf7, fact-97b75588d5514b8728fe4aa3, fact-300ea84f20ecc412e99cd12e, fact-167aad313fe901c497cd751b

Tested weights

Q4_K_M · 5.2 GB

unsloth · recorded source

Ministral-3-8B-Reasoning-2512-Q4_K_M.gguf · Exact GGUF file →Historical first-MiB hash only; not a full-file identity.

Explore the answers

Choose a suite to inspect its exact available questions, references and model answers.