What is RAG?

The short version

RAG means retrieval-augmented generation. Before a model writes an answer, the application searches a defined set of documents and adds the most relevant passages to the model's context.

The model still generates the response. The difference is that it receives the material it should use for that response at request time, rather than relying only on what it learned during training.

Why teams use it

Company policies, product documentation, support records, and internal procedures change frequently. Retraining a model whenever one of those sources changes is slow and expensive. A RAG system can use the current documents instead.

It also gives the team a way to inspect an answer. If an assistant tells a customer how to return an order, the operator can see which policy paragraph was retrieved and decide whether the answer is grounded in the right source.

How a RAG request works

Most systems follow a small sequence:

1. Documents are split into manageable sections and stored with metadata, such as source, owner, date, and access rules.
2. When a user asks a question, the system finds the sections most likely to answer it.
3. The application sends those sections, the question, and clear instructions to the model.
4. The model drafts a response. The application can return the source links with it or send uncertain cases to a person.

The retrieval step matters as much as the model. A capable model cannot give a reliable answer when it receives the wrong document, an outdated version, or too much unrelated context.

What RAG does not solve by itself

RAG does not make every answer factual. The system can retrieve a weak source, miss the right passage, or ask the model to infer more than the material supports. It can also expose information to the wrong person if permissions are not applied before retrieval.

For operational work, treat RAG as one part of the system. Define the approved sources, set access controls, record the retrieved context, and test real questions that have known answers. Add an escalation path when the documents do not support a confident response.

A simple example

A support assistant receives the question: "Can I change the delivery address after dispatch?" The RAG system searches the current shipping policy, retrieves the relevant section, and supplies it to the model. The model writes the response and links to that policy section.

Without retrieval, the same model may give a plausible answer based on a general pattern. That may sound useful, but it does not prove that the answer matches the company's actual policy.

When RAG is a good fit

RAG works well when the answer should come from a known body of material that changes over time. Common examples include internal knowledge bases, product manuals, legal templates, customer support policies, and technical documentation.

It is a poor fit when there is no trustworthy source material, the task requires a transaction rather than an answer, or the information must be calculated from live systems. In those cases, the application may need a database query, a workflow action, or human review instead.