Mitigation 1: Prompt Engineering

THE QUESTION THIS PAGE ANSWERS

ANSWER FIRST

What is the key idea behind “Mitigation 1: Prompt Engineering”?

Constraint instructions + limitations: the model doesn't know what it doesn't know

DECISION RULE

Inspect what the model is being shown. The practical move is to separate instructions, source material, history, tools, and output rules. Once the context is visible, the right fix is usually easier to choose.

TRY NEXT

Draw the input and output of one small workflow before changing its prompt or model.

WATCH FOR

Adding more text when the real issue is relevance, ordering, or a missing boundary.

Select Business Scenario

Scenario A · Customer Service

E-commerce / Finance Customer Service Bot

Users ask about product details and refund policies — the model easily fabricates non-existent rules.

Scenario B · Content Creation

AI Writing Assistant / Document Generation

When generating reports, the model may fabricate data or cite non-existent regulatory documents.

Scenario C · Knowledge Q&A

Enterprise Internal Knowledge Base Q&A

When employees ask about internal processes, the model fills gaps with generic knowledge, which may not match actual company policy.

PM Key Takeaways

Loading…

Conversation Comparison

Unconstrained Prompt

Constrained System Prompt

No System Prompt

User Question → Fabricated answer…

The model predicts tokens directly from training memory; when it hits knowledge gaps, it fabricates fluently.

With a Constrained System Prompt

User Question + If unsure say you don't know → I'm not sure…

The blue constraint words become part of the context, raising the probability of the "admit I don't know" sequence.

When the model predicts the next token, every token in the current context influences the probability distribution. The constraint words injected via System Prompt act as "high-quality prior context": they raise the probability of the "I'm not sure" sequence and lower the probability of fabrication sequences. The essence: use carefully crafted prior context to directionally bias the probability distribution of subsequent output.

⚠ Limitation: The model doesn't know what it doesn't know

When constraint instructions work

When the model has some uncertainty about a question, constraint words can push it toward admitting it doesn't know.
→ Works for: questions beyond knowledge scope, ambiguous queries, time-sensitive information

When constraint instructions fail

When the model is highly confident in a wrong answer, its top-ranked token is already wrong, and constraint words cannot affect that probability.
→ Fails for: systematic errors in training data, outdated knowledge treated as fact

PM Countermeasure

For domains where "the model is highly confident but may be wrong," Prompt constraints alone are insufficient.
→ Combine with RAG (inject real knowledge) or human review as fallback

How “Select Business Scenario” becomes executable

“Constraint instructions + limitations: the model doesn't know what it doesn't know” is not about a magic phrase. It is about giving the model enough information to know who the work is for, what must be done, and what counts as acceptable.

Background sets direction; constraints set the boundary

“Constraint instructions + limitations: the model doesn't know what it doesn't know” shows why a useful request separates the task, audience, source material, output format, and constraints. Without background, the model guesses. Without acceptance criteria, fluent text is not evidence that the task is complete.

More words do not guarantee a better result

Turn “Constraint instructions + limitations: the model doesn't know what it doesn't know” into a small experiment: change only one of background, requirements, or constraints while keeping the rest fixed, then observe which layer actually changes the output.

From “Select Business Scenario” to “Conversation Comparison”

“Select Business Scenario” grounds the problem in “Scenario A · Customer Service E-commerce / Finance Customer Service Bot Users ask about product details and refund policies — the model easily fabricates non-existent rules. Scenario B · Content Creation AI Wri…”. “Conversation Comparison” then moves it toward “Compare View Prompt Template Why It Works Three Strategies Unconstrained Prompt Constrained System Prompt System Prompt Template · Ready to reuse No Context Attach Document Force "I Don't Know" Example: "Commer…”. Together, they show that the lesson is not just a conclusion to remember, but a claim with conditions.

Carry the judgment into the next situation

Build a request layer by layer: task and audience first, material and output rules next, constraints and acceptance checks last. Change one layer at a time so you know what actually helped.

“Select Business Scenario”: Scenario A · Customer Service E-commerce / Finance Customer Service Bot Users ask about product details and refund policies — the model easily fabricates non-existent rules. Scenario B · Content Creation AI Wri…
“Conversation Comparison”: Compare View Prompt Template Why It Works Three Strategies Unconstrained Prompt Constrained System Prompt System Prompt Template · Ready to reuse No Context Attach Document Force "I Don't Know" Example: "Commer…

The final “Finish by testing the claim” brings the discussion to “Constraint instructions + limitations: the model doesn't know what it doesn't know”. The useful thing to carry forward is knowing which judgments must be revisited when input, scale, or risk changes.

INTERACTIVE PRACTICE

Turn a vague request into a useful prompt

Clarify the goal, context, and constraints, then carry the finished prompt into the AI tool you use.

What do you want to do Context Output format Quality bar

Fill in the fields above and your prompt will appear here.

Mark as learned Your reading progress updates automatically

ARTICLE DISCUSSION

Leave one useful thought here.

Keep the idea that clicked, the question that stayed open, or a small note for the next learner.

↗

Discussing Mitigation 1: Prompt Engineering The Model Under the Product

3discussionsArticle discussion · synced with the Circle

View in the learning circle

Asha MorganContent editorAug 20, 2026

INSIGHTField note

I turned one judgment from this article into a small experiment I could run today. Knowing what to observe next is more useful than simply remembering the conclusion.

ARTICLE DISCUSSION7 helpful

Lin HarperIndie developerAug 18, 2026

INSIGHTInsight

After reading this, I first looked for the conditions behind the idea instead of copying the method into a project. That order made the later trade-offs much clearer.

ARTICLE DISCUSSION5 helpful

Kiki MooreProduct operationsAug 15, 2026

QUESTIONQuestion

When this judgment reaches real work, which constraint should be added first? I am curious which step matters most between reading and the first practical attempt.

ARTICLE DISCUSSION4 helpful