My 3-Model Local AI Stack: Fast Worker + Coder + Senior Planner

I used to run one model for everything. ChatGPT for code generation, debugging, architecture questions, writing boilerplate – all through a single $20/month subscription. The model was good at some things and mediocre at others, but I didn’t think about it. It was just “the AI.”

Running local LLMs changed how I think about AI-assisted development. Instead of one model doing everything, I now run three models, each assigned to a specific role. The result is faster responses for simple tasks, better quality for complex ones, and zero monthly cost.

Here’s the stack, how it works, and why three models beats one.

The Three Roles

Every coding task falls into one of three categories:

1. Quick tasks – Generate a function, rename a variable, explain a concept, write a comment. Speed matters more than depth.
2. Daily coding – Write tests, refactor code, implement features, handle multi-file edits. Quality matters more than speed.
3. Hard problems – Architectural decisions, code review, debugging complex issues, large-context analysis. Depth matters more than both.

Matching the right model to each category is the difference between a local AI setup that feels like a superpower and one that feels like a toy.

Role 1: Fast Worker – Gemma 4 E4B

What I use it for:

• Generating simple functions and utilities
• Explaining code snippets
• Writing inline comments and documentation
• Quick refactoring (rename, extract, restructure)
• Answering factual questions about APIs or libraries

Why Gemma for this role:

At 68 t/s, Gemma generates output faster than I can read it. For quick tasks, speed is the only metric that matters – I don’t need the model to think deeply about generating a strlen wrapper or explaining a CSS flexbox issue. I need it done in 2 seconds so I can move on.

Gemma 4 E4B is Google’s small MoE model. It activates 4 billion parameters per token out of roughly 12 billion total. The MoE architecture means it’s computationally cheap despite having a decent-sized knowledge base. The Q4_K_M quantization keeps it well within VRAM limits.

Example usage:

# Quick task: generate a helper function
"Write a PHP function that validates a Moroccan phone number format"

# Gemma responds in ~2 seconds with:
function validateMoroccanPhone(string $phone): bool {
    return (bool) preg_match('/^(?:\\+212|0)[5-7]\\d{8}$/', $phone);
}

Two seconds. Done. No need to burn 30 seconds waiting for a 35B model to produce the same output.

Role 2: Daily Coder – Qwen2.5-Coder 7B

What I use it for:

• Writing tests (unit tests, integration tests, feature tests)
• Multi-file refactoring
• Implementing features that span multiple components
• Writing complex database queries
• Generating code that follows specific patterns or conventions

Why Qwen for this role:

Qwen2.5-Coder is a dense model – all 7 billion parameters fire on every token. This means it has consistent quality across all coding tasks, not the variable quality you sometimes see with small MoE models. The entire model was fine-tuned on code, so it understands programming patterns, naming conventions, and common idioms better than general-purpose models.

At 37 t/s with 128k context, it’s fast enough for interactive use and large enough to hold entire codebases in memory. When I’m refactoring a Laravel module with 15 files, Qwen sees the full picture.

Example usage:

# Daily coding task: implement a feature
"Add rate limiting to this Laravel API route. Use the token bucket algorithm.
Here are the 3 files that need changes..."

# Qwen responds with:
# - RateLimiter middleware class
# - Updated routes/api.php
# - Config for rate limits per endpoint
# - Feature tests for the rate limiter

Qwen handles multi-file changes cleanly. It understands the relationship between files, maintains consistent coding style, and produces output that doesn’t need heavy editing.

Role 3: Senior Planner – Ornith 35B-A3B

What I use it for:

• Architectural decisions (“Should I use repositories or direct Eloquent?”)
• Code review (security, performance, maintainability)
• Debugging complex issues (race conditions, memory leaks, performance bottlenecks)
• Large-context analysis (review an entire module)
• Planning multi-sprint feature implementations

Why Ornith for this role:

Ornith has 35 billion parameters – 3 billion active per token through MoE. The total parameter space gives it more “world knowledge” than Gemma or Qwen, even though the active compute is similar. For tasks that require reasoning across multiple concepts, connecting dots across a codebase, or identifying subtle issues, the larger parameter space matters.

The MTP speculative decoding keeps it fast enough for interactive use. At 30-36 t/s, I’m never waiting for the model to catch up. And the 128k context means it holds the entire codebase plus the conversation history.

Example usage:

# Gemma for quick tasks
llama-server -m gemma4-e4b-Q4_K_M.gguf -ngl 999 -fa on -c 65536 -np 4

# Qwen for daily coding
llama-server -m qwen2.5-coder-7b-Q4_K_M.gguf -ngl 999 -fa on -c 131072 -np 1

# Ornith for hard problems
llama-server -m Ornith-1.0-35B-MTP-APEX-I-Mini.gguf \\
  --spec-type draft-mtp --spec-draft-n-max 2 \\
  -ngl 999 -ncmoe 34 -fa on \\
  -ctk q8_0 -ctv turbo2 -c 128300 -np 1

The -ngl 999 flag loads all layers to GPU. -fa on enables Flash Attention for faster context processing. The -np flag sets parallel slots – I use 4 for Gemma (multiple quick tasks can queue) but only 1 for Qwen and Ornith (larger models, one task at a time).

For Ornith specifically, the MTP flags enable speculative decoding, which boosts generation speed by 20-40%. The -ctk q8_0 -ctv turbo2 flags enable asymmetric KV compression – the subject of a dedicated deep-dive post. The -ncmoe 34 flag configures the MoE expert count.

For a detailed explanation of how MTP speculative decoding works under the hood, see Speculative Decoding Explained: How MTP Turns 18 t/s Into 36 t/s.

The total setup time was one afternoon. The daily workflow improvement has been permanent. Three models, three roles, zero cost, full privacy. That’s the local AI stack that actually works on 8GB VRAM.

What It Costs

The hardware cost is the only real investment. My setup – RTX 3060 Ti, Ryzen 7 5800X, 32GB RAM – costs around $800-1000 depending on current pricing. That pays for itself in about 40-50 months compared to ChatGPT Plus at $20/month.

But the real cost comparison isn’t vs ChatGPT Plus. It’s vs the hours I used to spend switching between browser tabs, copying code into ChatGPT, waiting for responses, copying output back, and debugging the output. The local stack eliminates the context switching. The model is right there in my IDE, at 68 t/s for quick questions, at 30-36 t/s for complex ones. No browser, no API, no waiting, no copy-paste dance.

The electricity cost is negligible. Running a local LLM on an RTX 3060 Ti consumes about 200-250 watts under load. Over an 8-hour coding day at $0.12/kWh, that’s roughly $0.24. Over a month of daily use, that’s about $5. Compare that to $20/month for ChatGPT Plus, and the local stack saves $15/month just in subscription costs – before counting the latency and privacy benefits.

For the full financial and privacy argument for running local AI instead of paying for ChatGPT or Claude, see Why I Use Local AI Instead of Paying for ChatGPT or Claude.