RAG, or retrieval-augmented generation, is the process by which LLM-based tools access external information. This information, which is not contained in the LLM’s training parameters, is added to the relevant input context and used in answer generation.
RAG is one of the most important components of modern AI systems. It’s used by all the mainstream agentic tools (ChatGPT, Claude, Perplexity, etc.) and has a wide range of consumer and enterprise use cases. RAG has undergone several innovations since it first became widespread and continues to evolve in compelling ways.
Retrieval-augmented generation (RAG) is a method of improving the outputs of large language models (LLMs) by providing them with relevant external information.
A typical RAG “pipeline,” a term used to refer to the broad underlying RAG architecture, has two paths, an ingestion path and a retrieval path. The ingestion path takes data and converts it into a searchable format. The retrieval path builds a request based on a user’s prompt and queries the information source.
Often, RAG systems will be built around vector databases. In this scenario, the orchestrator component of an LLM tool, which manages user prompts, sends queries, and retrieves results from the searchable index, will generate a vector embedding. It uses this to query a database of vectors that represent the ingested data. It will then add the retrieved information to the existing context input for the LLM with instructions about how to use it to generate an output.
RAG doesn’t refer exclusively to the type of dense vector search described above. Knowledge graph traversal, semantic search, traditional database lookups, or any mix of these methods also qualify.
Benefits of RAG
One of the reasons RAG has become so popular is that it solves two core issues associated with LLM-only answer generation: hallucinations and lack of access to information not contained in training materials.
Fact checking and grounding
LLMs have a reputation for struggling with factual accuracy. Often, if they cannot answer a question reliably, they will confidently hallucinate. This can be either because information was not contained in training materials or the training materials themselves contain inaccuracies. The process of RAG-enabled fact checking involves grounding an answer in information from a trusted third-party source.
Up-to-date information
LLMs can only answer questions about topics that run up to their training data cut-off point, called the knowledge cut-off. RAG systems can connect to highly current sources of information, such as news materials using a web search API or internal databases with up-to-date product information. This makes it possible to generate responses on topics that aren’t covered in the training data.
Access to private external data
RAG systems allow enterprises to connect AI infrastructure to sources of sensitive, proprietary, and domain-specific information that they cannot share publicly. Many popular tools, such as Perplexity, now include the option to upload confidential data and build private knowledge bases, and it is a common enterprise use case.
RAG overview and evolution: Main types of architecture
While the specific term retrieval-augmented generation first appeared in a major 2020 paper titled Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, information retrieval systems had been in use for many decades prior. Early search engines are a good example. However, the real AI innovation was to combine parametric model knowledge with a dense vector index (the paper used Wikipedia to build this).
Here’s an overview of the main types of RAG:
- Basic RAG: Also called naive RAG, basic RAG is the simplest iteration of a RAG pipeline. It follows a retrieve and generate flow, with vector database search as the primary use case. It is still suitable for building a straightforward RAG implementation, especially for limited internal use.
- Hybrid RAG: This involves combining more than one type of RAG. It usually refers to a combination of simple keyword searches and dense vector retrieval.
- Corrective RAG: A form of RAG in which retrieved information is evaluated by a fine-tuned model to determine relevance to the user prompt.
- Self-RAG: While similar to corrective RAG, in self-RAG the LLM evaluates its own outputs and initiates retrieval if needed. It uses specialized reflection tokens to do this.
- Agentic RAG: Agentic RAG refers to the way agentic AI platforms can call on different retrieval methods. The LLM can reason and choose from various RAG options via the orchestrator.
- Multimodal RAG: This type of RAG incorporates other forms of media, like images and audio, into its ingestion and retrieval pipelines.
- GraphRAG: GraphRAG incorporates knowledge graph traversal (entity search) to retrieve information. This may be in conjunction with dense vector retrieval but doesn’t have to be.
- Branched RAG: This RAG architecture involves running multiple retrieval queries at the same time, often using different architecture patterns. Branched RAG is a common feature in AI tools that run complex workflows.
- HyDE (Hypothetical Document Embeddings): Hypothetical document embeddings are AI-generated responses that simulate what an ideal answer to a user prompt might be. These are used to refine the RAG query, thereby improving the accuracy of retrieved information.
Top RAG use cases with examples
What does RAG look like in the real world? If you’ve asked an AI chatbot a question about a time-sensitive topic, queried a customer support bot, or used a personal assistant for administrative tasks, you’ve made use of RAG.
The top RAG examples across consumer and enterprise contexts are web search, personal assistants (including in coding), coding assistants, customer support, and enterprise knowledge management.
Web search
Web search is a widely used form of RAG and arguably the most pervasive consumer-facing application. All of the mainstream chatbots offer web search functionality, in which the LLM uses up-to-date sources to supplement the information contained in its training materials.
The orchestrator in the AI tool connects to a third-party search engine using an API call, and results, in the form of snippets, are added to the LLM’s context window. This allows users to access current events, recent research, product information, and other topics where information changes frequently.
Practically, the API request might look like the following:
json
{ "query": "latest US inflation news", "max_results": 1, "language": "en", "country": "US" }
The returned information would include the source and snippet:
json
{ "query": "latest US inflation news", "results": [ { "title": "US inflation rises slightly in latest report", "url": "https://example.com/us-inflation-report", "snippet": "Consumer prices increased in the latest monthly data, with housing and energy contributing to the rise.", "published_date": "2026-11-05" } ] }
Personal AI assistants
Personal assistants use RAG to connect to information sources within a personal or shared enterprise environment. Examples of information sources include a user’s emails, files, calendar, and databases.
Because information can be stored in various ways, personal assistants are best understood as fully agentic RAG systems. They may use API calls, keyword search, dense vector retrieval, or knowledge graph traversal, depending on the prompt.
Increasingly, enterprise RAG systems will encode knowledge into vector databases so that it can be retrieved easily by company-wide personal assistants, and many enterprise AI products are designed with this in mind.
Coding assistants are also notable because they represent another major use case. A code assistant, such as GitHub Copilot, can connect to a repository and project-specific documentation to retrieve relevant files, functions, symbols, guidance, APIs, and commit history.
Customer support
Customer support is another major RAG use case, and adoption of AI chatbots has increased rapidly in recent years. In its State of AI Customer Support report, Crisp found that 40% of conversations are resolved without the need for a human, showing a high level of effectiveness.
When a user asks a customer support bot a question, the orchestrator will usually search the company’s internal knowledge base. To ensure accuracy, corrective RAG and self-RAG are also often employed, with human rep hand-off initiated where appropriate.
Enterprise knowledge bases
The rollout of AI workflows and generative AI tools across medium and large businesses has occurred in conjunction with the building of internal knowledge bases. This allows company-wide assistants to access internal data while maintaining privacy, whether it be sensitive customer information, private corporate data, or specialized domain knowledge.
These systems are usually built around vector databases, but other RAG methods are also used. Microsoft 365 Copilot, for example, has an API layer that exposes various information sources, including vector databases, knowledge graphs, and semantic indexes, to provide broad access to a company’s data.
These RAG systems are also typically built into the enterprise packages of general-purpose agentic tools. Perplexity Enterprise, for example, allows users to upload large amounts of documentation for secure internal use.
Users can also upload their own files within self-contained projects, essentially creating mini RAG systems. Searches can be coupled with other forms of RAG, like web searches, for hybrid augmentation.
What’s the difference between RAG and an LLM?
A large language model (LLM) can only generate outputs using information represented in its parameters. It assigns probabilities to an internal vocabulary to predict the next word, or token, in a sequence. As such, it cannot provide accurate answers if its parameters weren’t trained on relevant information.
In a general-use or specialized AI platform, RAG pipelines function alongside an LLM or group of LLMs. They connect to external sources of information to supplement the input context that is provided to the language model, and this is the core difference between RAG and LLMs.
The decision of whether or not to retrieve additional information or rely exclusively on parametric knowledge can be determined at either the level of the orchestrator, typically using rule-based logic, or even by the LLM itself.
What’s the difference between RAG and MCP?
RAG and model context protocol (MCP) are best understood as complementary technologies. MCP is an open standard for building architecture that allows AI tools to access external functionality, resources, and prompts using a single endpoint.
When an AI tool connects to an MCP server, the server exposes all of the primitives (MCP tools, resources, and prompts) that the client has permission to access. The client is the component that establishes the connection between the orchestrator on the client side and the server. Sometimes, this might involve exposing databases and information sources, in which case the MCP acts as an intermediary in the retrieval layer of a RAG pipeline.
What’s the difference between RAG and an API?
An API is a defined interface that two applications can use to send requests to each other. RAG is the process by which LLMs can access external information to improve and expand their outputs.
RAG often uses API calls to retrieve information. In web search information retrieval, for example, the orchestrator will formulate an API request to the relevant search engine based on the user’s prompt.
Why RAG matters and how it will continue to evolve
RAG matters because LLMs are inherently restricted by their training data. There are whole swaths of information, from current events to private company knowledge, that they don’t have access to. RAG addresses this issue without requiring a prohibitive amount of resources.
However, information retrieval on its own is not enough, and this is why RAG will continue to evolve. RAG forms a large component of current AI research and is focused on improving systems so that they return the right information across a wide range of formats and judge and prioritize it effectively.
As RAG grows more powerful, so does the usefulness of AI systems across important consumer and enterprise applications. These include customer support, web search, coding, personal assistants, specialized research, and more.












