Part 2 · The Harness Around the Model

Long-Term Memory: Vector Retrieval

Embedding → vector database → semantic search; design decisions for topK and minScore

THE QUESTION THIS PAGE ANSWERS

ANSWER FIRST

What is the key idea behind “Long-Term Memory: Vector Retrieval”?

Embedding → vector database → semantic search; design decisions for topK and minScore

DECISION RULE

Make the claim earn its place. Use this page as a decision aid, not a definition to memorize. Connect the idea to one real task, one observable result, and one failure that would change your mind.

TRY NEXT

Write one question you could answer with evidence after trying this idea.

WATCH FOR

A conclusion that sounds complete but leaves the key assumption untested.

An Analogy
🗂
Short-term Memory = Your Desk
The context window — limited space for information
🗄
Long-term Memory = Your Filing Cabinet
Vector database — retrieve relevant files onto the desk when needed
The desk can't hold everything, but you can pull the most relevant files from the cabinet at any time
Interactive Demo: Click a Question to Observe the Retrieval Process
🗄 记忆库 8 memory entries
Embedding gemini-embedding-001
维度 768
存储 LanceDB
topK 5
minScore 0.3
Click to simulate different user queries
🔍 Vector Retrieval Process
Three Design Decisions
📌 Design Decision ①
What information should be stored in long-term memory? User preferences, project config, historical bugs, frequently used operations — these determine the Agent's level of personalization.
📌 Design Decision ②
Retrieval quality depends on the Embedding model. "Fix the login API" and "login API returning 500 under concurrent load" — can the model match both to the same memory entry?
📌 Design Decision ③
Too many memories is also a problem: topK=5 means at most 5 entries are recalled each time. How do you ensure the most important memories rank first?
Takeaway
Takeaway Short-term memory (context window) lets the Agent remember the current conversation; long-term memory (vector retrieval) lets the Agent remember last month. Together they allow the Agent to behave like a real assistant: understanding what you're doing now while also remembering your past preferences.

Why “An Analogy” can find relevant content

“Embedding → vector database → semantic search;” moves retrieval beyond storing material: the real question is how to find what is relevant. That decision shapes the input quality of RAG, recommendation, and image-search systems.

Similarity is not the answer

In the flow described by “Embedding → vector database → semantic search;”, embeddings place items in a comparable semantic space and a neighbor index narrows the search. The final answer still depends on whether the retrieved chunks cover the question, whether the distance metric fits, and whether the evidence is current.

Separate findable from relevant

Turn “Embedding → vector database → semantic search;” into a small test: prepare queries with known answers, record relevance, misses, and distractors, then decide whether chunking, the index, or reranking needs to change.

From “An Analogy” to “Interactive Demo: Click a Question to Observe the Retrieval Process”

“An Analogy” grounds the problem in “🗂 Short-term Memory = Your Desk The context window — limited space for information → 🗄 Long-term Memory = Your Filing Cabinet Vector database — retrieve relevant files onto the desk when needed The desk can't…”. “Interactive Demo: Click a Question to Observe the Retrieval Process” then moves it toward “🗄 记忆库 8 memory entries Embedding gemini-embedding-001 维度 768 存储 LanceDB topK 5 minScore 0.3 Click to simulate different user queries 🔍 Vector Retrieval Process —”. Together, they show that the lesson is not just a conclusion to remember, but a claim with conditions.

Carry the judgment into the next situation

The same logic applies to retrieval: define what counts as relevant, check whether recall covers the question, and then inspect whether ranking, chunking, or freshness pushed useful evidence out.

  • “An Analogy”: 🗂 Short-term Memory = Your Desk The context window — limited space for information → 🗄 Long-term Memory = Your Filing Cabinet Vector database — retrieve relevant files onto the desk when needed The desk can't…
  • “Interactive Demo: Click a Question to Observe the Retrieval Process”: 🗄 记忆库 8 memory entries Embedding gemini-embedding-001 维度 768 存储 LanceDB topK 5 minScore 0.3 Click to simulate different user queries 🔍 Vector Retrieval Process —
  • “Three Design Decisions”: 📌 Design Decision ① What information should be stored in long-term memory? User preferences, project config, historical bugs, frequently used operations — these determine the Agent's level of personalization…

The final “Three Design Decisions” brings the discussion to “📌 Design Decision ① What information should be stored in long-term memory? User preferences, project config, historical bugs, frequently used operations — these determine the Agent's level of personalization…”. The useful thing to carry forward is knowing which judgments must be revisited when input, scale, or risk changes.

Mark as learned Your reading progress updates automatically
← PreviousNext →

Keep reading

The next useful article in the thread.

ARTICLE DISCUSSION

Leave one useful thought here.

Keep the idea that clicked, the question that stayed open, or a small note for the next learner.

Discussing Long-Term Memory: Vector Retrieval The Harness Around the Model
3discussionsArticle discussion · synced with the Circle
View in the learning circle
AM
Asha MorganContent editor
INSIGHTField note

I turned one judgment from this article into a small experiment I could run today. Knowing what to observe next is more useful than simply remembering the conclusion.

ARTICLE DISCUSSION7 helpful
LH
Lin HarperIndie developer
INSIGHTInsight

After reading this, I first looked for the conditions behind the idea instead of copying the method into a project. That order made the later trade-offs much clearer.

ARTICLE DISCUSSION5 helpful
KM
Kiki MooreProduct operations
QUESTIONQuestion

When this judgment reaches real work, which constraint should be added first? I am curious which step matters most between reading and the first practical attempt.

ARTICLE DISCUSSION4 helpful