Not every model that claims to be “good for coding” actually is. I tested 9 different LLMs on my RTX 3060 Ti, ran 32 benchmark configurations, and rejected 3 of them outright. The failures taught me more about what matters in local AI than the successes.
Here’s the honest breakdown – what worked, what didn’t, and why.
The Full Roster
| Model | Size | Type | Quantization | Verdict |
| Gemma 4 E4B | 4B active | MoE | Q4_K_M | Kept – fast worker |
| Qwen3-4B Thinking | 4B | Dense | Q4_1 | Kept – simple planner |
| Qwen3.5 9B | 9B | Dense | Q4_K_M | Maybe – too heavy for speed |
| Gemma 4 26B-A4B | 4B active | MoE | Q4_K_M / UD-IQ4_NL | Kept – reasoning model |
| GLM-4.6V-Flash | – | Dense | Q4_K_M + mmproj | Rejected |
| Qwen3-Coder 30B-A3B | 3B active | MoE | IQ4_XS | Maybe – not worth the complexity |
| Qwen2.5-Coder 7B | 7B | Dense | Q4_K_M | Kept – daily coder |
| Qwen3.6 35B-A3B MTP | 3B active | MoE + MTP | UD-IQ4_NL | Kept – strong planner |
| Ornith 35B-A3B | 3B active | MoE + MTP | UD-IQ4_NL | Kept – senior planner |
Out of 9 models, 6 earned a permanent spot, 2 are “maybe” (useful in specific scenarios), and 1 was rejected completely.
The 3 Rejections
GLM-4.6V-Flash: Dead on Arrival
I tried GLM-4.6V-Flash three times. Three rejections.
The model is Zhipu’s multimodal offering – it has vision capabilities through the mmproj (multi-modal projector) component. On paper, it sounds interesting. In practice, it was unusable.
Run 1 (Q4_K_M + mmproj, 32k context): 9.75 t/s generation. Output was garbled Chinese text despite English prompts. Tokenizer warnings in the server log.
Run 2: 12.31 t/s. Slightly better but still producing mixed Chinese/English output. The model clearly wasn’t trained to handle the prompt format I was using.
Run 3: 18.91 t/s. Speed improved but output quality was still broken. Repeated phrases, incoherent code suggestions, and occasional character soup.
The lesson: model speed means nothing if the output is garbage. GLM-4.6V-Flash is optimized for Chinese language tasks. Running it for English coding was the wrong model for the job. I should have checked the training data language distribution before downloading.
Qwen3-Coder 30B: Too Much Complexity for Too Little Gain
Qwen3-Coder 30B-A3B was my attempt at a large dedicated coding model. 30 billion parameters, 3 billion active per token, with IQ4_XS quantization to fit in VRAM.
The model worked. The speed was acceptable (14-19 t/s depending on MoE layer count). The code quality was good. But it never justified its complexity over simpler alternatives.
The MoE tuning nightmare: Getting acceptable speed required tuning the -ncmoe parameter through 8 different values (20, 25, 30, 36, 37, 40, 41, 37 at 128k). At ncmoe 20-25, speed was 10-14 t/s – too slow. At ncmoe 37, it hit 18.57 t/s – acceptable but not impressive.
The context scaling problem: At 32k context, Qwen3-Coder ran at 18.57 t/s. At 128k context, it dropped to 16.71 t/s. The speed degradation with context size was worse than Ornith, which maintained 30-36 t/s at 128k.
The comparison problem: At 37 MoE layers, Qwen3-Coder hit 18.57 t/s. Ornith at ncmoe 34 hits 30-36 t/s. Ornith is faster, has more parameters, supports MTP, and requires less tuning. There’s no reason to use Qwen3-Coder when Ornith exists.
The lesson: a model that works isn’t the same as a model that’s worth the effort. Qwen3-Coder 30B is a perfectly functional model, but the maintenance overhead of tuning it for 8GB VRAM doesn’t justify its existence when better options are available.
Qwen3.5 9B: Technically Good, Practically Slow
Qwen3.5 9B is a solid general-purpose model with vision capabilities. It ran correctly, produced good output, and handled both text and image inputs. I kept it as a “maybe” because the speed never justified daily use.
At 125k context with Q4_K_M quantization, the model generated at 22.58 t/s. Not slow in absolute terms, but slow compared to the alternatives. Gemma E4B does the same quick tasks at 83 t/s. Qwen2.5-Coder handles coding tasks at 37 t/s. The 9B model sits in an awkward middle ground – too slow for quick tasks, not deep enough for hard problems.
The vision capability is interesting, but I don’t need local vision enough to justify the speed penalty. If I need image analysis, I use a cloud API.
The lesson: a model that’s “good at everything” is often “not the best at anything.” Specialization wins on limited hardware.
What the 6 Keepers Have in Common
The models I kept share three characteristics:
1. Clear role assignment: Each model does something specific well. Gemma is fast. Qwen Coder is code-focused. Ornith is deep. None of them tries to be everything.
2. Acceptable speed for its role: A “fast worker” needs to be genuinely fast (83 t/s). A “daily coder” needs to be interactive (37 t/s). A “senior planner” needs to be fast enough to not interrupt flow (30+ t/s). Models below 15 t/s get demoted to “maybe.”
3. Minimal tuning overhead: If a model requires 8 rounds of ncmoe tuning to be usable, it’s not worth the effort. The models that earned a permanent spot worked with reasonable defaults.
The Surprises
MTP Changes Everything for Large Models
Before testing MTP-enabled models, I thought speculative decoding was a minor optimization – maybe 10-20% speed improvement. The reality: MTP nearly doubles generation speed on models that support it.
The Qwen3.6 35B without MTP runs at ~18 t/s. With MTP n=1, it hits 27.19 t/s. That’s a 50% improvement from a single flag change. For Ornith with optimized MTP settings, the improvement is even larger – from an estimated 18-19 t/s base to 30-36 t/s with MTP.
If your model supports MTP and you’re not using it, you’re running at half speed.
The 4B MoE Model Outperforms the 7B Dense Model on Speed
Gemma 4 E4B at 83 t/s versus Qwen2.5-Coder 7B at 37 t/s. The smaller MoE model is nearly twice as fast. This makes sense – MoE activates fewer parameters per token – but the quality gap is smaller than you’d expect. For simple coding tasks, Gemma’s output is 80-90% as good as Qwen’s, at nearly twice the speed.
TurboQuant+ KV Compression Is Not Optional
The first configs I tested used symmetric q8_0/q8_0 KV cache. Maximum context was 32k on 8GB VRAM. After switching to asymmetric q8_0/turbo2, context jumped to 128k. The KV cache is the bottleneck, not the model weights. This optimization alone is worth more than any model swap.
The Final Scorecard
| Model | Speed | Quality | Tuning Effort | Role | Verdict |
| Gemma 4 E4B | 83 t/s | Good | Low | Fast worker | Daily driver |
| Qwen3-4B | 86 t/s | Basic | Low | Quick planner | Useful |
| Qwen3.5 9B | 22 t/s | Good | Low | Vision/general | Maybe |
| Gemma 4 26B | 18 t/s | Good | Medium | Reasoning | Useful |
| GLM-4.6V | 9-18 t/s | Broken | N/A | Vision | Rejected |
| Qwen3-Coder 30B | 18 t/s | Good | High | Coding | Maybe |
| Qwen2.5-Coder 7B | 37 t/s | Excellent | Low | Code specialist | Useful |
| Qwen3.6 35B MTP | 27 t/s | Excellent | Medium | Planning | Keep |
| Ornith 35B | 30-36 t/s | Excellent | Low | Daily coder + senior planner | Daily driver |
The local AI space moves fast. New models drop every month. But the methodology stays the same: test on your hardware, measure what matters, reject what doesn’t work, and build a stack around roles instead of chasing the single “best” model.
32 test runs taught me that the answer isn’t one model. It’s three models, each doing what it does best.
For the complete breakdown of how I use these three models in daily work, see My 3-Model Local AI Stack: Fast Worker + Coder + Senior Planner.
