RAG Is Not the Architecture. Evidence Is.
I kept looking at the normal data-agent pipeline and thinking we were optimizing the wrong thing. The breakthrough is not a better chunker, embedding model, or prompt. It is turning the entire system into a closed loop that plans, retrieves, verifies, and learns around claims.
Make claims the primitive.
Once the system tracks evidence for every factual claim, hallucination stops being a vague model-quality problem. It becomes an evidence-coverage problem that can be measured, blocked, and repaired.
We turned RAG into a ritual.
The mature pipeline is familiar: database, retrieval, chunking, cleaning, embeddings, indexing, prompting, orchestration, validation, answer. Then we monitor similarity scores, failed features, database errors, and confident-but-wrong output.
That is useful, but it still assumes retrieval is one predetermined operation. Every question gets pushed through roughly the same machinery even when the answer should have come from a direct SQL query, an aggregation, an API, a graph traversal, or a clarification from the user.
We keep trying to make the pipeline smarter. I think the pipeline itself is the limitation.
Intent + risk
What kind of answer is this, and how wrong can we afford to be?
Adaptive plan
What claims must be established, and what evidence does each one need?
Evidence graph
SQL, documents, APIs, lineage, and provenance stay attached to claims.
Verification
Claims pass, fail, contradict, or trigger another retrieval cycle.
Let the answer earn the right to exist.
Pick a question below. The agent does not start by searching for passages. It starts by identifying what must be true, then chooses the right evidence source for each claim.
Why did enterprise revenue decline last quarter?
A revenue question is not a document-search problem.
Take the question: “Why did enterprise revenue decline last quarter?”
Before writing anything, the system needs to establish whether revenue actually declined, by how much, where the change came from, whether the data is complete, and what supports the explanation.
The first answer might require SQL. The second needs an aggregation. The third needs cohort analysis. The fourth belongs to ingestion and quality metadata. Only the explanation might need semantic retrieval.
That removes a huge amount of unnecessary RAG.
Monitor the truth, not just the machinery.
Similarity scores and model confidence are not enough. I want to know how much of the answer is supported, what was contradicted, and whether the system repaired its own bad reasoning before a user saw it.
Evidence coverage
% of claims backed by authoritative data
Retrieval efficiency
Useful evidence divided by everything retrieved
Verification depth
Whether important claims were independently checked
Contradiction rate
How often authoritative sources disagree
Repair rate
How often the agent catches and fixes its own failure
Unsupported-claim escape
Unsupported claims that still reach the user
Unsupported-claim escape rate.
Not how confident the model sounded. Not whether retrieval returned something. How many claims without enough evidence actually made it into the final answer.
Every failure should improve the infrastructure.
A log file is not learning. When verification fails, the system should identify why: schema ambiguity, stale index, bad join, missing filter, weak chunk, permissions, data quality, user ambiguity, or reasoning failure.
Then it should turn that diagnosis into a reusable correction for the planner, semantic layer, metadata, or retrieval policy. The next agent should inherit the fix instead of rediscovering the same failure.
Verifier catches a weak claim
The answer is blocked before it reaches the user.
System classifies the failure
Wrong table, stale index, missing join, ambiguity, permissions, or reasoning.
Correction becomes policy
Planner rules, metadata, schemas, or retrieval instructions improve.
Future agents inherit the fix
The organization gets better at answering the next question correctly.
RAG becomes a tool, not the architecture.
Give the system contracts, schemas, lineage, permissions, quality signals, and evaluation objectives. Let it decide whether the question needs SQL, vectors, a graph, documents, an API, a clarification, or a refusal.
The moat is not “our LLM is better.” The moat is that every question makes the organization’s data-agent system slightly better at answering the next question correctly.