AI customer support that answers from your own documents

Picture an outdoor gear shop with a busy website. The same questions arrive all day: can I return boots I have worn once, when will my tent ship, does this jacket's warranty cover a torn zipper? A general AI chatbot will answer every one of them fluently, and some of those answers will be wrong, because it has never read the shop's return policy or warranty terms. The fix is not a smarter model. It is giving the model your documents and making it answer from them.

That approach has a name, retrieval-augmented generation or RAG, and it sits behind most useful support AI. This guide explains it in plain terms, shows the published results from companies that measured it, and gives a ladder for deciding how much retrieval machinery a small business actually needs.

Why support is still the best first AI project

Support has everything a first AI project needs: high volume, repeating patterns and numbers you already track, such as time to resolution, repeat contacts and satisfaction. The best-known early result came from Klarna. In its first month, February 2024, its AI assistant handled 2.3 million conversations, two-thirds of Klarna's customer service chats, doing work Klarna put at the equivalent of 700 full-time agents across 23 markets and more than 35 languages. ¹

Two details in Klarna's announcement matter as much as the headline. Customer satisfaction was on par with human agents, not higher. And these were launch-month figures from a company with millions of customers. A small business should treat them as proof the pattern works, not as a forecast, and should keep an easy path to a person for anything the assistant cannot settle. ¹

What an AI support agent actually does

A useful support agent does more than chat. It sorts incoming questions, looks up the relevant policy or order, resolves what it safely can, and hands the rest to a person with the context already gathered. The division of labor below is the practical version for a small team.

RAG in plain English

Retrieval-augmented generation works like an open-book exam. When a question comes in, the system first searches your own material, such as the return policy, warranty terms, help articles and past tickets, and pulls out the few passages that look relevant. It then hands those passages to the AI model with an instruction: answer using this. Because the answer comes from your documents, the agent can cite the passage it used, and a person can check it.

RAG is not always needed. Anthropic points out that if your whole knowledge base is under about 200,000 tokens, roughly 500 pages, you can simply include all of it in the prompt and skip the search step. Many small businesses fit comfortably under that line: a return policy, a warranty page, shipping rules and fifty help articles. That guidance dates from 2024, and newer models accept far more, so the threshold has only grown. ³

When plain search is not enough

Basic RAG usually searches by meaning, using what are called embeddings: a question about a torn zipper finds the warranty passage about damaged fasteners even though the words differ. Meaning-based search has a blind spot, though. It can miss exact identifiers. Anthropic's example is an error code: a search by meaning might miss the exact match for TS-999, where a plain keyword search would find it at once. ³

Anthropic's fix combines three ideas. Hybrid search runs keyword and meaning-based search together. Contextual retrieval adds a short description of where each passage sits in its document before indexing it, so a lone paragraph about refunds still knows it belongs to the boots policy. Reranking then re-scores the top results before they reach the model. In its tests, the combination cut failed retrievals by 67%. ³

When a knowledge graph is worth it

Some questions need more than one passage. Microsoft Research found that baseline RAG struggles to connect the dots when an answer depends on relationships spread across many documents, or on understanding a whole collection. Its GraphRAG approach first builds a map of the entities in your data and how they relate, then answers from that map. On simple lookups, Microsoft notes, both approaches perform well; the graph pays off on the multi-step questions. ⁴

LinkedIn published the clearest support example. Its customer service team treated past tickets as a knowledge graph, linking each issue to related issues, rather than as loose text. In LinkedIn's tests the method beat a plain-text baseline by 77.6% on a standard retrieval score, and after about six months in use by its support team, it reduced the median time to resolve an issue by 28.6%. ²

What to measure

Track the same numbers the published cases report, so you can compare honestly. Resolution time and repeat contacts show whether customers actually got their answer; Klarna reported both. Median resolution time for your own team shows whether agent assist is helping staff; that is LinkedIn's metric. Satisfaction should hold steady or rise, never fall. Add one AI-specific check: how often the agent's cited passage actually supports its answer, sampled weekly by a person. ¹²

Start this week

• Export last month's support messages and group them into the ten most common questions.
• Collect the documents that answer them and fix anything outdated first; the agent will repeat your mistakes.
• Measure your baseline: average time to resolve and how many customers contact you twice about the same thing.
• Start with the smallest rung on the ladder that fits, often the whole knowledge base in the prompt.
• Require the agent to cite the passage it used and to hand off with a summary whenever it is unsure.
• Keep refunds, exceptions and complaints with a person, and review a sample of conversations every week.