For small coding tasks, Q4_K_M is a sensible starting point when download size matters. Q5_K_M, Q6_K and Q8_0 also did well here. With reasoning requested on, every format passed every task; requested off, Q6_K and Q8_0 led by only a single task. That narrow gap offers little reason to download a larger format for small edits alone, but this limited bank cannot settle performance on other tasks.
Evidence for this judgment
Everyday coding tasks · Thinking requested on
Matching task controls and 16,384 context. These quality results describe the recorded setups; they do not isolate the effect of bit depth.
| Weight format | Task pass rate | Task time | Download | Inspect |
|---|---|---|---|---|
Q4_K_MSelected formatornith-aivulkan · 32 GB test machineCPU MoE layers: Not recordedRecorded execution settings
| 16/16 passed16/16 scored | 13.3s16/16 timed | 5.78 GBWeights, not required RAM | Answers and code |
Q5_K_Mornith-aivulkan · 32 GB test machineCPU MoE layers: Not recordedRecorded execution settings
| 16/16 passed16/16 scored | 12.6s16/16 timed | 6.64 GBWeights, not required RAM | Answers and code |
Q6_Kornith-aivulkan · 32 GB test machineCPU MoE layers: Not recordedRecorded execution settings
| 16/16 passed16/16 scored | 16.2s16/16 timed | 7.56 GBWeights, not required RAM | Answers and code |
Q8_0ornith-aivulkan · 32 GB test machineCPU MoE layers: Not recordedRecorded execution settings
| 16/16 passed16/16 scored | 27.2s16/16 timed | 9.79 GBWeights, not required RAM | Answers and code |
Everyday coding tasks · Thinking requested off
Matching task controls and 16,384 context. These quality results describe the recorded setups; they do not isolate the effect of bit depth.
| Weight format | Task pass rate | Task time | Download | Inspect |
|---|---|---|---|---|
Q4_K_MSelected formatornith-aivulkan · 32 GB test machineCPU MoE layers: Not recordedRecorded execution settings
| 15/16 passed16/16 scored | 6.9s16/16 timed | 5.78 GBWeights, not required RAM | Answers and code |
Q5_K_Mornith-aivulkan · 32 GB test machineCPU MoE layers: Not recordedRecorded execution settings
| 15/16 passed16/16 scored | 10.0s16/16 timed | 6.64 GBWeights, not required RAM | Answers and code |
Q6_Kornith-aivulkan · 32 GB test machineCPU MoE layers: Not recordedRecorded execution settings
| 16/16 passed16/16 scored | 8.8s16/16 timed | 7.56 GBWeights, not required RAM | Answers and code |
Q8_0ornith-aivulkan · 32 GB test machineCPU MoE layers: Not recordedRecorded execution settings
| 16/16 passed16/16 scored | 12.7s16/16 timed | 9.79 GBWeights, not required RAM | Answers and code |