Mitigation 2: RAG — Retrieval-Augmented Generation

THE QUESTION THIS PAGE ANSWERS

ANSWER FIRST

What is the key idea behind “Mitigation 2: RAG — Retrieval-Augmented Generation”?

Inject real documents into the context; 5-step animated flow; compare with and without RAG

DECISION RULE

Inspect what the model is being shown. The practical move is to separate instructions, source material, history, tools, and output rules. Once the context is visible, the right fix is usually easier to choose.

TRY NEXT

Draw the input and output of one small workflow before changing its prompt or model.

WATCH FOR

Adding more text when the real issue is relevance, ordering, or a missing boundary.

Choose a Business Scenario

Scenario A · Legal / Compliance

Contract Review / Regulatory Q&A Assistant

Regulations update frequently; the model's training data has a cutoff, making it easy to cite outdated or repealed clauses.

Scenario B · Healthcare

Drug Information / Medication Advice Assistant

Medication plans are highly personalized; the model's general knowledge cannot reflect a specific patient's condition or hospital protocols.

Scenario C · Product Docs

SaaS Product Technical Support Bot

Products iterate rapidly; version numbers and navigation paths change often, so the model only remembers the old version it was trained on.

PM Decision Framework: When is RAG Required?

✅

Time-sensitive knowledge: Policies, products, or prices update frequently — the model's training data can't keep up

✅

Private knowledge base: Internal company docs, product manuals — the model has never seen them

✅

High cost of errors: Legal, medical, or financial contexts — one wrong answer can cause serious harm

⚠️

Not needed for chat/creative scenarios: RAG can actually make responses feel rigid and overly constrained

Conversation Comparison

Without RAG (Closed-Book)

With RAG (Open-Book)

Without RAG: Closed-Book Exam

Question → × Guessing from training memory

The context contains only the question itself, so the model can only predict each Token based on impressions from its training data

With RAG: Open-Book Exam

Real document chunks + Question → ✓ Accurate answer

The retrieved documents become part of the context, directly shifting the probability distribution of each subsequent Token

At its core, a model is Token predicting Token — what comes next depends entirely on what came before. RAG works by "injecting retrieved real documents as high-quality context": with the same model, change the context and you change the probability distribution of everything that follows. The model hasn't gotten smarter — it just has a better starting point.

⚠ Limitations: RAG Is Not a Silver Bullet

Retrieval itself can fail

When retrieved chunks are inaccurate, the model treats the wrong document as ground truth — hallucinations don't decrease; they gain the appearance of an authoritative source

The model still fills gaps beyond the retrieved docs

The retrieved context window is limited; anything outside it is still completed from training memory, mixing RAG + hallucination in ways that are harder for users to detect than pure hallucination

PM mitigation strategy

RAG quality ceiling = knowledge base quality. Build retrieval hit-rate evaluation, and use mandatory citation display so users can verify sources

Why “Choose a Business Scenario” can find relevant content

“Inject real documents into the context;” moves retrieval beyond storing material: the real question is how to find what is relevant. That decision shapes the input quality of RAG, recommendation, and image-search systems.

Similarity is not the answer

In the flow described by “Inject real documents into the context;”, embeddings place items in a comparable semantic space and a neighbor index narrows the search. The final answer still depends on whether the retrieved chunks cover the question, whether the distance metric fits, and whether the evidence is current.

Separate findable from relevant

Turn “Inject real documents into the context;” into a small test: prepare queries with known answers, record relevance, misses, and distractors, then decide whether chunking, the index, or reranking needs to change.

From “Choose a Business Scenario” to “Conversation Comparison”

“Choose a Business Scenario” grounds the problem in “Scenario A · Legal / Compliance Contract Review / Regulatory Q&A Assistant Regulations update frequently; the model's training data has a cutoff, making it easy to cite outdated or repealed clauses. Scenario B…”. “Conversation Comparison” then moves it toward “↺ Replay Comparison View RAG Flow Why It Works Without RAG (Closed-Book) With RAG (Open-Book) Next → Reset Step 0 / 7 Step 1: Click "Next" to begin Click "Next" on the left to see each step in detail → Without…”. Together, they show that the lesson is not just a conclusion to remember, but a claim with conditions.

Carry the judgment into the next situation

The same logic applies to retrieval: define what counts as relevant, check whether recall covers the question, and then inspect whether ranking, chunking, or freshness pushed useful evidence out.

“Choose a Business Scenario”: Scenario A · Legal / Compliance Contract Review / Regulatory Q&A Assistant Regulations update frequently; the model's training data has a cutoff, making it easy to cite outdated or repealed clauses. Scenario B…
“Conversation Comparison”: ↺ Replay Comparison View RAG Flow Why It Works Without RAG (Closed-Book) With RAG (Open-Book) Next → Reset Step 0 / 7 Step 1: Click "Next" to begin Click "Next" on the left to see each step in detail → Without…

The final “Finish by testing the claim” brings the discussion to “Inject real documents into the context”. The useful thing to carry forward is knowing which judgments must be revisited when input, scale, or risk changes.

INTERACTIVE PRACTICE

Turn a vague request into a useful prompt

Clarify the goal, context, and constraints, then carry the finished prompt into the AI tool you use.

What do you want to do Context Output format Quality bar

Fill in the fields above and your prompt will appear here.

Mark as learned Your reading progress updates automatically

ARTICLE DISCUSSION

Leave one useful thought here.

Keep the idea that clicked, the question that stayed open, or a small note for the next learner.

↗

Discussing Mitigation 2: RAG — Retrieval-Augmented Generation The Model Under the Product

3discussionsArticle discussion · synced with the Circle

View in the learning circle

Asha MorganContent editorAug 20, 2026

INSIGHTField note

I turned one judgment from this article into a small experiment I could run today. Knowing what to observe next is more useful than simply remembering the conclusion.

ARTICLE DISCUSSION7 helpful

Lin HarperIndie developerAug 18, 2026

INSIGHTInsight

After reading this, I first looked for the conditions behind the idea instead of copying the method into a project. That order made the later trade-offs much clearer.

ARTICLE DISCUSSION5 helpful

Kiki MooreProduct operationsAug 15, 2026

QUESTIONQuestion

When this judgment reaches real work, which constraint should be added first? I am curious which step matters most between reading and the first practical attempt.

ARTICLE DISCUSSION4 helpful