Abstract
When we talk about making AI systems better at finding and using information from documents, we often focus on fancy algorithms and cutting-edge language models. But here’s the thing: if your text extraction is garbage, everything else falls apart. This paper looks at how OCR quality impacts retrieval-augmented generation (RAG) systems, particularly when dealing with scanned documents and PDFs.
We explore the cascading effects of OCR errors through the RAG pipeline and present a modern solution using SmolDocling, an ultra-compact vision-language model that processes documents end-to-end. The recent OHRBench study (Zhang et al., 2024) provides compelling evidence that even modern OCR solutions struggle with real-world documents. We demonstrate how SmolDocling (Nassar et al., 2025), with just 256M parameters, offers a practical path forward by understanding documents holistically rather than character-by-character, outputting structured data that dramatically improves downstream RAG performance.