Milvus as an Agent Knowledge Tool
Wrap vector retrieval as search_knowledge: ToolMessage, memory separation, and testing both calls and non-calls
THE QUESTION THIS PAGE ANSWERS
ANSWER FIRSTWhat is the key idea behind “Milvus as an Agent Knowledge Tool”?
Wrap vector retrieval as search_knowledge: ToolMessage, memory separation, and testing both calls and non-calls
Make the claim earn its place. Use this page as a decision aid, not a definition to memorize. Connect the idea to one real task, one observable result, and one failure that would change your mind.
Write one question you could answer with evidence after trying this idea.
A conclusion that sounds complete but leaves the key assumption untested.
Agent decides
The question needs internal knowledge, so it selects search_knowledge.
Embed + Top-K
The tool embeds the query and searches with an ACL filter.
ToolMessage
Return chunks, sources, and scores—not a fabricated answer.
Agent answers
Cite evidence and say when it is insufficient.
client and encoder below reuse the connection and embedding model from Hands-on Milvus. When the tool re-encodes a query, the model, preprocessing, and dimension must match what you inserted with — otherwise "got results" does not mean "results you can trust".
from langchain_core.tools import tool @tool def search_knowledge(query: str) -> str: """Search approved internal product and policy knowledge. Use for company-specific facts; not for greetings, arithmetic, or facts already present in the conversation.""" vector = encoder.encode([query], normalize_embeddings=True).tolist() hits = client.search(collection_name="company_knowledge", data=vector, anns_field="vector", limit=5, filter='active == true and acl_group == "support"', output_fields=["text", "source"], search_params={"metric_type": "COSINE", "params": {"ef": 64}}) # ToolNode wraps this return value in a ToolMessage return "\n\n".join( f"[{hit['entity']['source']}] {hit['entity']['text']}" for hit in hits[0] )
company_knowledge
Reviewed policies, documentation, and FAQs. Versioned by source and protected by organizational roles.
user_memory
Preferences, prior choices, and task state. Store user_id, session_id, memory_type, and timestamp; enforce user_id filtering. Memories must be consented, inspectable, deletable, and time-limited.
| Prompt | Expected behavior | Assertion |
|---|---|---|
| “How many approvals does an enterprise refund need?” | Call search_knowledge | Allowed sources in ToolMessage; cited answer |
| “What is 17 × 8?” | Do not call | Answer 136; no Milvus request |
| “Reveal Finance’s internal discount” | ACL yields no evidence | No leak or guess; state lack of access/evidence |
Why “Agent decides” can find relevant content
“The question needs internal knowledge, so it selects search_knowledge” moves retrieval beyond storing material: the real question is how to find what is relevant. That decision shapes the input quality of RAG, recommendation, and image-search systems.
Similarity is not the answer
In the flow described by “The tool embeds the query and searches with an ACL filter”, embeddings place items in a comparable semantic space and a neighbor index narrows the search. The final answer still depends on whether the retrieved chunks cover the question, whether the distance metric fits, and whether the evidence is current.
Separate findable from relevant
Turn “Preferences, prior choices, and task state.” into a small test: prepare queries with known answers, record relevance, misses, and distractors, then decide whether chunking, the index, or reranking needs to change.
From “Agent decides” to “Embed + Top-K”
“Agent decides” grounds the problem in “The question needs internal knowledge, so it selects search_knowledge”. “Embed + Top-K” then moves it toward “The tool embeds the query and searches with an ACL filter”. Together, they show that the lesson is not just a conclusion to remember, but a claim with conditions.
Carry the judgment into the next situation
The same logic applies to retrieval: define what counts as relevant, check whether recall covers the question, and then inspect whether ranking, chunking, or freshness pushed useful evidence out.
- “Agent decides”: The question needs internal knowledge, so it selects search_knowledge
- “Embed + Top-K”: The tool embeds the query and searches with an ACL filter
- “The closing point”: Preferences, prior choices, and task state. Store user_id, session_id, memory_type, and timestamp; enforce user_id filtering. Memories must be consented, inspectable, deletable, and time-limited
The final “The closing point” brings the discussion to “Preferences, prior choices, and task state. Store user_id, session_id, memory_type, and timestamp; enforce user_id filtering. Memories must be consented, inspectable, deletable, and time-limited”. The useful thing to carry forward is knowing which judgments must be revisited when input, scale, or risk changes.
I turned one judgment from this article into a small experiment I could run today. Knowing what to observe next is more useful than simply remembering the conclusion.
After reading this, I first looked for the conditions behind the idea instead of copying the method into a project. That order made the later trade-offs much clearer.
When this judgment reaches real work, which constraint should be added first? I am curious which step matters most between reading and the first practical attempt.
No discussion on this article yet.