Most tutorials stop at "it works on my machine." This is the part that decides whether you'd actually let it talk to a customer. And testing an LLM system is genuinely different from testing normal software, so it's worth doing properly.
Why you can't test it the normal way
The usual move is to assert that output equals an expected string. That doesn't work here. Ask the agent the same question twice and you get two correct answers worded differently. A string-equality test would fail both. So the question isn't "did it return exactly this text?" — it's "was the answer correct, grounded in the real policy, and appropriate?"
Part of the system is deterministic, though, and I test that the boring, reliable way. The database tools — search, book, cancel, refund maths — have exact right answers. Those get ordinary unit tests. It's only the model's judgement that needs a different approach.
How I test it: scenarios, plus a judge
Two things do the heavy lifting.
Realistic scenarios. I wrote 15 end-to-end customer scenarios — the simple policy question, the booking change that needs both agents, the ambiguous location, the request that should be refused. They run the whole system, not one piece, because the bugs that matter live in the hand-offs between agents.
LLM-as-a-judge. Instead of matching strings, I have a separate model grade each response against a rubric: is it correct, is it grounded in the actual policy, is the tone right? That turns a fuzzy "seems fine" into a score I can track and fail a build on.
15 scenarios → run the full system → capture each response
↓
LLM judge grades vs. rubric
↓
correct? grounded? appropriate? → reportThe scenarios and the evaluation harness are in the open-source repo. The judge isn't perfect — but it catches regressions a human reviewer would miss on the hundredth run, and it never gets bored.
Next in the series
The system is built and tested. The last part is the honest one: this is a project, not a client deployment — so what can I actually say it's worth, and what would I refuse to claim?
Evaluating LLM systems so you can trust them in production — that's a big part of what I do at twentytwotensors. Get in touch.