I wanted something modest: OCR some PDFs, cheaply, without babysitting a server. And because there's a good open-weight vision-language OCR model now — GLM-OCR, 0.9B parameters, top of the OmniDocBench leaderboard — my instinct was to self-host it. That instinct cost me an afternoon and taught me the actual lesson, which is about the question, not the model.
The self-hosting rabbit hole
Serving a 0.9B vision model is a GPU job, and GPUs don't like being idle-and-cheap at the same time. I walked the usual ladder — spot VM, then a container pipeline, then Cloud Run for scale-to-zero — and hit the tax at every rung. On Cloud Run, a cold start is 30–60 seconds before the first page is even read. AWS Lambda, the natural "cheap and serverless" pick, is out entirely for a model like this: 15-minute timeout, a 6 MB response cap, billing for init. I was engineering hard to make an expensive shape fit a cheap budget.
The question I'd skipped: do I even need a vision model?
Then I asked the thing I should have asked first. There are really two OCR jobs, and they get conflated constantly:
- Text + coordinates — "what words are here, and where?" A classic detect-then-recognise pipeline. Plain text out, bounding boxes out.
- Document understanding — tables reconstructed, reading order, structured Markdown or JSON. This is what a vision-language model buys you, and it's genuinely harder.
I'd assumed I needed the second and was paying for it. I needed the first. And once you only need text and coordinates, the model gets absurdly small. PP-OCRv6 (PaddleOCR) is a detect→recognise pipeline whose small tier is ~31 MB and its tiny tier ~6 MB — hundreds of times smaller than the vision model. A 6 MB CPU model doesn't just save money, it deletes the entire problem I'd been solving: no GPU, no meaningful cold start, and Lambda and the Cloud Run free tier become trivially sufficient. The infrastructure pain was self-inflicted by the model choice.
The plot twist: sometimes you should just pay the API
If you do need real document understanding — tables, structure, clean Markdown — there's a third option that beats self-hosting anything: a hosted OCR API. Mistral's OCR runs about $2 per 1,000 pages (roughly half that in batch mode), returns structured Markdown with reconstructed tables, and has a zero-second cold start because it's someone else's always-warm GPU. For comparison, AWS Textract is around $65 per 1,000 pages — a hosted API isn't automatically the expensive option; this one is ~30× cheaper than the incumbent. When your volume is a few thousand pages, "pay pennies per page and write no infrastructure" is very hard to beat.
Reading the numbers honestly
Two honesty notes, because this is where these comparisons usually mislead. First: don't reflexively pick the smallest model. For personal batch work, latency is irrelevant — nobody's waiting — so you should optimise accuracy, not size. Going from PP-OCRv6 tiny to small lifts recognition accuracy from ~73.5% to ~81.3% (about +8 points) for a cost that's still effectively nothing. The tiny tier is for when milliseconds matter, which for a batch job they don't.
Second, and I want to be straight about it: I haven't run the bake-off yet. The vision-model CPU throughput figures are estimates, the hosted-API accuracy numbers are the vendor's own benchmarks (not independently verified), and the real test — quality on my actual documents — is still to do. This is a scoping-and-costing exercise, not a validated result, and the honest next step is to run the same twenty pages through all three and look.
The transferable bit
Before you optimise how you deploy a model, check you've picked the right class of model. Half the infrastructure problems in an ML project are self-inflicted by reaching for a bigger model than the task needs — I spent an afternoon fighting GPU cold starts for a job a 6 MB CPU model does on a free tier. The cheapest deployment is the one you delete by asking "what do I actually need out of this?" first.
What I'd actually run
Model first, then deployment. For plain text and positions: PP-OCRv6 small, self-hosted on CPU — a spot VM for batches, or Cloud Run's free tier for the occasional job. For real structured document understanding: the Mistral OCR API — pennies a page, zero infrastructure. Self-host the vision model only in the one case that actually forces it: you need document understanding and privacy rules out sending pages to an API. That's a real case — but it's the exception, not the default I started with.
If you've got a document-processing pipeline to cost out — and you want the honest split between "self-host this" and "just pay for that" — it's the kind of work I do at twentytwotensors. Get in touch.