I pay for an AI coding assistant every month, and one day I did the obvious thing and asked: what if I just ran the model myself? The open-weight coders are good now. A spot GPU is a few pence an hour. It felt like it should be cheaper.
So I priced it out — properly, across the models I'd actually use (Qwen3.5, GLM 4.7 Flash, DeepSeek V3.2), for my real workload of roughly 12M tokens a month with a heavy cache-hit rate. The conclusion isn't the one I set out to prove. At my volume, self-hosting is not the way to save money. It's the way to buy something an API won't sell you.
The cheapest option is an API — by a mile
Here's the number that ended the "to save money" version of this project before it started. DeepSeek V3.2 Chat, at my usage, costs about $2.84 a month — and it scores 72–74% on SWE-bench Verified, which is genuinely strong for writing and fixing code. There's no server, no GPU, nothing to babysit. GLM 4.7 Flash has a free tier that costs literally $0.
Against that, my best self-hosted plan comes in around $11 a month. So the very first honest thing to say is: if the only axis you care about is cost, and your usage is modest, you stop here. You do not spin up a GPU to replace a $3 API bill. I nearly talked myself out of it on this line alone.
So when does self-hosting actually win?
The answer is a crossover. An API bill is variable — it scales with every token. A self-hosted spot box is a fixed cost — roughly the same whether you send it 12M tokens or 100M. So the more you use it, the better fixed looks.
| Monthly usage | DeepSeek Chat (API, variable) | Qwen3.5-27B on a GCP L4 spot (fixed) |
|---|---|---|
| 12M tokens | ~$2.84 | ~$11.20 |
| 50M tokens | ~$11.83 | ~$11.20 |
| 100M tokens | ~$23.66 | ~$11.20 |
The lines cross at roughly 50M tokens a month. Below that, the API is cheaper and less hassle. Above it, the fixed box pulls ahead and keeps pulling. But notice what that framing quietly admits: at 12M tokens, cost is the wrong reason to self-host. The real reasons are the ones that don't show up in the table — no rate limits, nothing leaving your own hardware, and open weights (Apache 2.0) you fully control. Those are worth paying a small premium for. "Saving money" is not why you'd do it at my scale.
The best self-hosted coder — and the catch I only found by measuring
On paper, the pick is easy. Qwen3.5-27B scores 72.4% on SWE-bench — level with DeepSeek — fits on a single 24GB L4 at a 6-bit quant, and costs about $11 a month on GCP spot. Same accuracy as the API, on hardware I own, for pocket change. Done, surely.
Then I put a stopwatch on it. On the L4, that 27B dense model generates at around 13 tokens per second. For a chat-style question that's fine. For an agentic coding loop — the kind that re-reads files, calls tools, and takes several turns to land a change — 13 tok/s is painful. The spec sheet sold me a model; the stopwatch sold me a much slower one. This is the whole lesson of the project in one line: never trust a throughput number you didn't measure yourself.
There was a second trap underneath it. Qwen3.5's hybrid architecture has known KV-cache bugs in llama.cpp right now (and therefore in Ollama), which make multi-turn sessions reprocess the whole prompt each turn. The workaround is real and worth knowing: serve it with vLLM, which handles the architecture properly.
The trade you're really making: speed vs. reasoning
Once you accept that a slow-but-accurate model is frustrating to code with, the interesting options are the fast Mixture-of-Experts ones. Qwen3.5-35B-A3B runs several times faster than the 27B; GLM 4.7 Flash does 100–145 tok/s and is built for running a few agents at once. More speed means more attempts in the same minute, and for iterative coding, more attempts often beats a single more-accurate one.
The catch — there's always a catch — is that the fast MoE models keep only a few billion parameters "active" per token, so their reasoning on hard, tangled problems is noticeably weaker. Which points at the setup that actually works: a fast local model for the 80% that's iteration, and a strong API reasoner (DeepSeek, or Claude Sonnet when it has to be right) for the 20% that's genuinely hard. Not either/or. Both, on purpose.
The transferable bit
Two habits travel far beyond this one project. First: separate the reason from the number. "It's cheaper" and "it's unlimited/private/mine" are different goals, and only one of them survives contact with a $2.84 API bill — be clear which you're buying. Second: measure the thing that matters to you (here, tokens per second in a real agent loop), because the headline benchmark and the number that decides whether a tool is pleasant to use are rarely the same number.
Where I actually landed
Start on DeepSeek Chat at ~$3 a month — validate that the quality is good enough for your work at essentially zero cost and zero risk. Reach for a self-hosted Qwen3.5 on a GCP L4 spot the day you actually need what the API can't give you: unlimited tokens, or your code staying on your own box. Serve it with vLLM, expect to pair it with an API reasoner for the hard problems, and go in knowing you're buying control, not a discount.
If you're weighing build-vs-buy on AI infrastructure and want the honest version — costs, crossovers, and the caveats the spec sheets leave out — that's the kind of work I do at twentytwotensors. Get in touch.