AI and software 5 min read
Retrieval-augmented generation, explained for business owners
Why the most useful business assistants look things up before they answer — and what that means for accuracy and privacy.
By the Smile Software team
Language models are trained on general information. They don't know your opening hours, your returns policy or last month's sales. Retrieval-augmented generation, or RAG, is the technique of looking up relevant information first and giving it to the model with the question.
How it works
- Your documents are split into passages and indexed for search.
- When someone asks a question, the most relevant passages are retrieved.
- The model receives the question and those passages, and answers using them.
- The answer can cite the passages it used.
Why it matters
Grounding answers in your content reduces made-up responses and lets people check the source. It also means you can update the assistant's knowledge by updating documents, without retraining anything.
Where it goes wrong
- Out-of-date or contradictory documents produce confused answers
- Poor search retrieves the wrong passages
- Permissions are ignored, so people see what they shouldn't
What to ask a vendor
Ask how answers are evaluated, how sources are shown, how access permissions are enforced and where your documents are processed and stored.
