1 tested weight format · Explore results, then inspect the answers.
Q4_K_M · 18.97 GB download
Links follow the selected quant and configuration.16 not yet recorded · setup comparability unverified
Q4_K_M · 18.97 GB of weights · vulkan · context 16384 · GPU layers unrecorded
Large · 20B and above · dense · 26.9B total
Memory fit has not been verified for this configuration.
pending
Execution details are incomplete. This configuration is not eligible for ranking.
Temperature 0 · seed 42 · output limit 4096 · reasoning effort unrecorded
RAM labels describe the test computer, not this model’s memory requirement. Speed comparisons keep the 32 GB and 128 GB runs separate.
Runtime build: unrecorded · Local · test machine with 32 GB RAM · concurrency: 1
Thinking status: unknown · probe unrecorded · scope unrecorded.
Requested off does not prove an absence of reasoning. Saved checks do not establish unrecorded historical template or build details.
0–100 / 100
Nine categories, equally weighted · 0/9 eligible
Possible score range with unscored tasks receiving zero to full credit. Incomplete results remain unranked; this is not a confidence interval. Quality only; speed is separate. How the index works →
Select a category to explore the answers. Bars show what happened on every planned task. Incorrect answers and failures to finish both earn zero; technical problems and untested tasks stay unscored.
Each suite has its own metric and fixed task bank. The balanced index weights categories equally; individual scores and evidence remain available below.
| Test suite | Thinking off | Details |
|---|---|---|
| Everyday coding tasksTask pass rate | Not scored0/16 scored · PreviewNo measured time | |
| Larger practical tasksMean task credit | Not scored0/11 scored · PreviewNo measured time | |
| Logic and number puzzlesMean puzzle credit | Not scored0/34 scored · PreviewNo measured time | |
| Published hard problemsMean answer credit | Not scored0/11 scored · PreviewNo measured time | |
| Following exact instructions · original campaignStrict prompt compliance | Not scored0/23 scored · PreviewNo measured time | |
| Reading without guessingExact-match accuracy | Not scored0/16 scored · PreviewNo measured time | |
| Calculating from a passageNumeric exact-match accuracy | Not scored0/7 scored · PreviewNo measured time | |
| Reading research papersAnswer token F1 | Not scored0/8 scored · PreviewNo measured time | |
| Resisting incorrect correctionsAll-branches pass rate | Not scored0/12 scored · PreviewNo measured time | |
| Following exact instructions · 128 GB campaignStrict prompt compliance | Not planned |
No reviewed summary yet. The charts show recorded benchmark results.
One weight format is recorded for this model. Additional tested formats will appear here when available.
No verified on/off comparison is available for this configuration and test suite. Requested modes alone do not establish a controlled thinking comparison.
Repeated runs are a separate experiment and never add votes to the baseline score.
No repetition experiment is planned for this selection.
Find information in a small project, edit files and fix bugs.
0 full credit · 0 partial credit · 0 zero credit
16 pending
Ranking unavailable: incomplete coverage, incompatible or uncertain cohort.
Scoring and fixed question bank →
Select a question to read the original answer, tool calls and available code changes for this configuration and thinking setting.
1–16 of 16 matching questions · 16 in this version
The user asks where the project's retry-limit setting is defined and what value it holds. This exercises locating a named setting in code and reporting both its location and value.
basic-T1 · primaryThe user asks for the release date mentioned in a project notes document. This exercises reading a specified file and extracting a particular fact rather than guessing from outside knowledge.
basic-T2 · primaryThe user asks to change the project's retry limit to a specified value. This exercises finding the relevant setting and making a focused code edit rather than merely describing the change.
basic-T3 · primaryThe user asks to create a text file containing exactly one supplied line. This exercises basic file creation and precise instruction following, including avoiding extra text in the file.
basic-T4 · primaryThe user asks what the most recent Git commit changed in the project. This exercises inspecting recorded changes and turning them into a clear explanation of what was modified.
basic-T5 · primaryThe user asks how many Python files are in the application folder and which is largest. This exercises identifying relevant files, counting them, and comparing their sizes.
basic-T6 · primaryThe user asks to add a function that converts byte counts into readable sizes and remove the related TODO comment. This exercises implementing a small utility and updating the surrounding code consistently.
basic-T7 · primaryThe user asks to find every TODO comment in the Python code and append each to the project's task list on a separate line. This combines searching across files with collecting text into an existing document.
basic-T8 · primaryThe user asks to investigate a failing test and fix the application code without changing the test. This exercises diagnosing a bug while respecting an explicit limit on what may be edited.
basic-H1 · primaryThe user asks to rename a network function and update every place in the project that uses it. This exercises tracking references across files and making a coordinated change rather than editing only the definition.
basic-H2 · primaryThe user asks to find a proxy configuration file somewhere under the project's parent folder and report its host and port. This exercises searching beyond the project folder and reading configuration values.
basic-H3 · primaryThe user asks which error code appears in a long event log and on which line. This exercises finding a specific entry in a large file and reporting its precise location.
basic-H4 · primaryThe user asks to append a supplied release-related task as the final line of an existing task list. This exercises preserving existing content while adding exact text in the requested position.
basic-H5 · primaryThe user asks for the total time a network function spends sleeping when all attempts fail, worked out from the code. This exercises following retry logic and adding the delays along that failure path.
basic-H6 · primaryThe user asks for a project readme listing every command-line option with its help text. This exercises discovering the program's available options and transferring that information into complete, source-grounded documentation.
basic-H7 · primaryThe user asks to undo the latest Git commit's change by editing the file directly, without using Git to make changes. This exercises understanding a previous edit and reversing it under a method constraint.
basic-H8 · primaryThese are starting points for inspection, not claims that one setup is better. Runtime and placement differences remain visible.
Another checkpoint in the same total-parameter band and architecture category, with matching task controls.
Requested thinking off · Exploratory: incomplete or uncertain evidenceCompare these configurations →
ggml-org · recorded source
Qwen3.8-27B-Q4_K_M.gguf · Exact GGUF file →Historical first-MiB hash only; not a full-file identity.
Choose a suite to inspect its exact available questions, references and model answers.