Everyone wants to talk about the model. But for a RAG system, the answer is only ever as good as what you retrieved, and what you retrieve is decided here — in the pipeline that turns source documents into vectors. For the CKD assistant (Part 1) that source is clinical guideline documents from NICE, KDIGO, the UK Kidney Association and Kidney Care UK.
Update, September 2026
The corpus has been rebuilt twice since this post — the "24 documents" below is a specific historical snapshot, not the current number. A later dedup pass found one more near-duplicate and a September re-chunk (2,000-token chunks → 800) settled the canonical corpus at 22 documents. The OCR engine also changed: this post describes Docling, but a later provenance audit found the live pipeline actually runs PaddleOCR-VL, not Docling — the two-stage layout-detector-plus-VLM architecture, kept because it handles rotated/vertical axis text better on scanned pages. See the evaluation post for the September fix/rebuild numbers.
OCR that respects tables
Clinical guidelines are full of tables — dosing thresholds, staging tables — and a naive text extract turns those into mush. The pipeline runs table-structure-aware OCR so a table comes out as a table, not a scrambled run of numbers, exporting both markdown (readable) and a structured JSON tree. Some pages are scanned, so OCR is on for those too.
Splitting out the parts that aren't clinical content
A guideline PDF is maybe 60% clinical content and 40% noise for my purposes — title pages, conflict-of-interest declarations, a long bibliography, abbreviations. If you embed all of it, your retriever happily returns a citation list when someone asks about potassium. So each document is split into main_text, references, and front/end matter, and only the clinical content goes forward.
I did the classification with a hybrid approach: MedGemma classifies each heading, with a regex heuristic as a fallback when the model isn't available. The fallback matters — it means the pipeline still runs without a GPU, just a bit more bluntly.
Cleaning and block-aware chunking
Then a manual cleaning pass for OCR artifacts (I tracked each document's review status — pass, needs-check, or delete; one KDIGO guideline was superseded by a newer edition and dropped). Finally, chunking. The rule I care about: don't slice through a coherent block. A chunk that ends mid-table or mid-recommendation retrieves badly, so the chunker is block-aware rather than a blind fixed-size window.
The clean chunks are embedded with EmbeddingGemma into 768-dimension vectors and stored in ChromaDB under one collection. That collection is the shared foundation every level in this series reads from.
The point
Most "the RAG isn't working" problems are pipeline problems — bad OCR, references polluting the index, chunks cut through the middle of a table. Spend your effort here before you reach for a bigger model. I found this out the hard way in the evaluation: nearly all the failures were retrieval, which traces straight back to this stage.
Next: the simplest system that uses this store — Level 1, plain RAG. (Part 3)