Make Five Runs in a Row All Usable
Same-input five-run demo; three tasks: write the first prompt, classify the instability, add examples and a ban list then rerun and compare
THE QUESTION THIS PAGE ANSWERS
ANSWER FIRSTWhat is the key idea behind “Make Five Runs in a Row All Usable”?
Same-input five-run demo; three tasks: write the first prompt, classify the instability, add examples and a ban list then rerun and compare
Follow the handoffs, not the demo. A system becomes dependable at the boundaries between model, tools, state, permissions, and people. Read each handoff as a place where you can observe, test, and recover.
Name the input, owner, approval, and recovery action for one automated step.
A successful run that cannot explain what happened or be safely repeated.
A lot of people stall here like this: "I tried it. Sometimes good, sometimes bad." Sometimes-good-sometimes-bad usually means the prompt never nailed the requirements down. The model is getting a bit of a raw deal. Ask the same sentence five times in a row and you'll see whether it's actually stable.
The same sentence, asked five times
Stable does not mean word-for-word
LLMs are born with randomness. Chasing verbatim replay is a waste of effort. What you nail down is three things: structure holds (three sections means three), fields hold (if you asked for a "next-week plan" it must be there), boundaries hold (when the material isn't enough it says "not mentioned in the docs," it doesn't invent one). Those three can be welded on with this chapter's output-format constraints plus Few-Shot examples. And wrap user input in delimiters — that's the first line of defense against Prompt injection.
This Chapter's Hands-on Checklist
0 / 3 done
Write M0's job as a first-version prompt
15 min EveryoneWrite it as the four-piece set: role + background + requirements + output format. Don't chase perfect. First get a version that runs. That version is the control for every later tweak.
What counts as done
Run it five times. Classify the instability
1 hour Want to know what's offSame input, five runs. Save every result. Anything you can't use as-is goes into one of three buckets: format drifted, content invented, length ran away. The fixes differ: format drift → add examples; invented content → narrow the material; length runaway → hard word limits.
What counts as done
Add examples and a ban list, then rerun and compare
Half a day Need it in a real workflowAdd one gold example (Few-Shot) and a ban list (e.g. "no pleasantries," "write 'not mentioned' for anything not in the material"). Run five more times and compare with the last version. Do one extra thing while you're at it: wrap user input in delimiters, then sneak "ignore all previous instructions" into the input and see if it takes the bait.
What counts as done
Why “See it first · What kind of disease is "sometimes it works pretty well"” depends on the operation
“A lot of people stall here like this: "I tried it.” makes the structure concrete. The useful comparison is not which name sounds more advanced, but how the data is arranged and how far the most common operation has to travel.
Read a structure through access and change
“LLMs are born with randomness.” exposes a trade-off that is easy to miss: reading by position, looking up by key, adding at either end, inserting in the middle, and traversing relationships do not favor the same organization. A structure that is fast for one operation is not automatically fast for all of them.
Count scale and update frequency together
Use “M1 only needs two things: the final prompt, and the five-run score.” as a boundary check. Write down the data size, the dominant operation, and the latency you can accept before deciding whether an AI-generated structure actually fits.
From “See it first · What kind of disease is "sometimes it works pretty well"” to “Hands-on checklist · Pick one, start it, tick it off”
“See it first · What kind of disease is "sometimes it works pretty well"” grounds the problem in “A lot of people stall here like this: "I tried it. Sometimes good, sometimes bad." Sometimes-good-sometimes-bad usually means the prompt never nailed the requirements down . The model is getting a bit of a raw…”. “Hands-on checklist · Pick one, start it, tick it off” then moves it toward “Write it as the four-piece set: role + background + requirements + output format. Don't chase perfect. First get a version that runs . That version is the control for every later tweak”. Together, they show that the lesson is not just a conclusion to remember, but a claim with conditions.
Carry the judgment into the next situation
When you meet a new data structure, do not begin by memorizing its definition. Write down the most frequent operation, estimate scale and update behavior, and check whether the structure satisfies all three conditions.
- “See it first · What kind of disease is "sometimes it works pretty well"”: A lot of people stall here like this: "I tried it. Sometimes good, sometimes bad." Sometimes-good-sometimes-bad usually means the prompt never nailed the requirements down . The model is getting a bit of a raw…
- “Hands-on checklist · Pick one, start it, tick it off”: Write it as the four-piece set: role + background + requirements + output format. Don't chase perfect. First get a version that runs . That version is the control for every later tweak
- “The closing point”: M1 only needs two things: the final prompt, and the five-run score. After the next chapter adds tools, you'll come back and change it
The final “The closing point” brings the discussion to “M1 only needs two things: the final prompt, and the five-run score. After the next chapter adds tools, you'll come back and change it”. The useful thing to carry forward is knowing which judgments must be revisited when input, scale, or risk changes.
I turned one judgment from this article into a small experiment I could run today. Knowing what to observe next is more useful than simply remembering the conclusion.
After reading this, I first looked for the conditions behind the idea instead of copying the method into a project. That order made the later trade-offs much clearer.
When this judgment reaches real work, which constraint should be added first? I am curious which step matters most between reading and the first practical attempt.
No discussion on this article yet.