Ben Faircloth is Senior Director of AI Solutions at Seekr, where he leads the Solutions Architecture team for Enterprise Sales. He was previously a Delivery Solutions Architect and Embedded Product Lead for Public Sector GenAI at Databricks, and holds a Master of Science in Data Science from Northwestern University. In this written Q&A with AI Frontier Network, he discusses Seekr's research on why AI systems can retrieve the right document and still answer from the wrong one, and what that means for versioned enterprise records, context attribution, audit trails and evaluation.
In Seekr's subsequent 80-question evaluation on U.S. Treasury statements, Llama 3.3 70B and Qwen 2.5 72B scored 95% and 92.5% when given only the correct statement, then fell to 85% and 87.5% once statements from other years were added, even though the correct one was still there. When you looked at the failed answers, what did they look like? Did the model pick the wrong year, blend figures from two years, or fall back on what it already believed, and how did context attribution let you tell those apart?
One of the questions in our study asked for a financial figure from a 1903 Treasury statement. The model had the correct document available, but it answered using a figure from 1913.
The model hadn't invented a number. It extracted a real figure from a real government document. The problem was that it selected the wrong evidence, even though the correct evidence was present.
That was one of three failure patterns we identified. In our initial 40-question evaluation, seven errors occurred despite successful retrieval. Three involved extracting figures from the wrong source. Two produced values unsupported by the retrieved documents. The remaining two combined figures from incompatible sources. In one case, the model calculated a ratio using a numerator from one fiscal year and a denominator from another. The arithmetic was internally consistent, but the inputs were incompatible.
SeekrFlow's context attribution helped us distinguish those behaviors. Wrong-source extraction showed strong influence from competing documents, with attribution scores between 0.68 and 0.95. Unsupported answers showed little detected influence from the retrieved evidence. Mixed-source answers showed influence distributed across incompatible passages.
In our subsequent 80-question evaluation, Llama lost 10 percentage points of accuracy when distractors were introduced, while Qwen lost 5. Both models had the correct evidence available.
What concerns me is how plausible these errors can appear. An authentic source document doesn't necessarily establish that the model used the right evidence.
In the published 40-question study, full-RAG accuracy fell to 35%, with retrieval failures accounting for most errors. Seven of the 26 errors occurred even when the required evidence was retrieved. For a team with a fixed engineering budget, how should effort be split between fixing retrieval and fixing how the model uses what it gets, and what measurement shows which problem they actually have?
Our research identified two different engineering problems that can produce the same outcome: the retrieval system may fail to find the required evidence, or the model may receive that evidence and still fail to use it correctly.
In the initial 40-question study, 15 of the 26 errors were clean retrieval failures. The correct document never reached the model. Another seven occurred after retrieval had successfully supplied the required evidence.
Retrieval was the larger problem in that experiment, so I'd initially prioritize improving it. But I wouldn't translate those numbers directly into engineering allocation. Some failures are considerably harder to correct than others, and the cost of an intervention matters.
Even when retrieval successfully provides the correct evidence, the model still has to identify and use it appropriately. That becomes particularly challenging when documents from different periods or versions share nearly identical structures. Our research demonstrates why retrieval quality and evidence utilization need to be measured separately. Finding the right information is only part of producing a reliable answer.
In our controlled experiment, accuracy declined when competing documents were introduced, even though the correct evidence remained available.
One measurement I'd emphasize is accuracy conditional on successful retrieval. In that study, the model answered only nine of 17 full-recall questions correctly.
Combine that with retrieval recall and source attribution, and teams can distinguish where engineering effort belongs. The allocation should change as the bottleneck changes, rather than assuming that retrieval or generation is always responsible.
Contracts with amendments, quarterly filings and regulatory documents revised every year all have the same structure as your Treasury setup: near-identical documents that differ mainly by date or version. Before relying on the model to pick the right version, what should enterprises change in ingestion, indexing and retrieval (version metadata, date filtering, deduplication, scoping the query to one period)? Which of these have you seen make a measurable difference in a customer deployment?
Versioned documents expose a particular challenge for enterprise AI systems. Documents from different reporting periods can contain almost identical tables, terminology, and structures, making semantic relevance alone insufficient to establish which evidence is applicable.
In our controlled Treasury research, we deliberately supplied the correct evidence alongside competing documents from other periods. Accuracy declined even though the right source remained available. That demonstrated why source selection and evidence utilization need to be considered together.
For contracts and regulatory documents, I'd preserve effective dates, amendment relationships, and supersession status as structured metadata. Those attributes should survive chunking, and retrieval should use them to constrain searches before ranking candidates. The constraints also need to reflect the question. Comparing two versions requires retaining evidence from both.
I'd treat deduplication carefully as well. Two amendments may look almost identical, yet a single changed provision can determine the correct answer.
That's where we're taking the research next: using attribution evidence to guide targeted corrections, whether through retrieval optimization, prompt re-engineering, or retrieval-augmented fine-tuning. The important measure is whether those interventions improve evidence utilization and answer accuracy without introducing new failure modes.
Context attribution estimates which supplied sources shaped an answer. In production, what does it cost in latency and compute to run it on every answer versus a sample? How reliable is the attribution signal itself, and what rule decides when an answer is flagged or sent to a human, for example when the cited source and the attributed source disagree?
One challenge with explainability is making it useful without imposing so much overhead that organizations cannot apply it operationally.
We've invested in optimizing SeekrFlow's context attribution for enterprise use. There is additional computation involved, and the cost depends on the workload, context size, and deployment configuration. Running attribution on every response provides broader coverage, while selective analysis or sampling reduces the computational workload. The tradeoff is that some responses go unexamined, so the appropriate level of coverage should reflect the consequences of an undetected error.
Reliability matters just as much as performance. In our published study's 17-question full-recall subset, correct answers showed average maximum attribution to expected evidence of approximately 0.44, compared with 0.07 for incorrect answers. That separation provides a useful diagnostic signal, although it doesn't establish a universal threshold.
Consider a system preparing information for a regulatory filing. If an answer cites the current document but attribution indicates substantial influence from an older version, that discrepancy should trigger verification. The system could check the applicable reporting period, validate the supporting evidence, and escalate unresolved concerns according to their consequences.
We're working toward that. SeekrFlow already provides the underlying attribution evidence, while automated verification and escalation depend on additional controls and application-specific policies.
You have written that in regulated and government settings a plausible answer with a real citation can pass review unnoticed. Drawing on that work and your public sector GenAI experience at Databricks, who should own the decision when attribution flags an answer as likely wrong, and what does an audit trail need to contain to stand up to a regulator, auditor or inspector general?
In public-sector and regulated environments, the question is whether the organization can establish why an AI answer was accepted and demonstrate that appropriate controls were applied.
An attribution flag is evidence of a potential problem, not a verdict that the answer is wrong. Responsibility for investigating that signal belongs with the system and its operational processes. Accountability for consequential decisions remains with the designated domain or process owner.
That distinction matters because most people using AI aren't qualified to independently verify every result. Nor can an organization rely on experts manually investigating every response. The system needs to surface meaningful evidence, perform verification where possible, and escalate unresolved issues according to established policies and the consequences of the decision.
For an auditor or inspector general, I would expect an audit trail to establish what information was available, which sources influenced the answer, what the system produced, and what happened when a concern was detected. That includes source and model versions, applicable policies, verification results, corrective actions, and any human approvals or overrides.
Most importantly, the record should allow an independent reviewer to reconstruct and challenge the decision, rather than simply confirm that a process was followed.
That's the direction we're pursuing at Seekr: systems that accelerate the path from evidence to resolution while making organizational accountability more effective.
Most teams deploy AI precisely because they cannot check every answer, and many have no answer key. How should a team build an evaluation set from its own documents, including deliberately adding near-duplicate distractors? Roughly how many questions are needed before the results mean something, and which metrics beyond final-answer accuracy belong on the dashboard?
A system can perform extremely well on questions where the correct document is obvious, then struggle when several plausible sources compete for its attention.
That's why I'd build evaluation sets around an organization's actual documents and deliberately test those more difficult conditions.
Start with questions that have verifiable answers and identifiable source passages. Subject-matter experts can validate a relatively small set of reference answers, even when checking every production response would be impractical. Some questions might involve extracting dates or financial values. Others should require comparisons, calculations, or interpretation of a particular provision.
Then introduce near-duplicate distractors: earlier contract amendments, filings from adjacent reporting periods, or policies revised while retaining similar language. I'd also include questions where the available documents don't support a defensible answer.
Run the same questions under controlled conditions. First provide only the correct evidence, then introduce competing documents, and finally allow retrieval to select the context. The differences help isolate failure mechanisms.
An initial set of 100 to 200 carefully reviewed questions can be useful for identifying patterns and evaluating changes, although it won't establish reliable rates for every failure category, particularly rare ones.
I'd track retrieval recall, accuracy conditional on successful retrieval, source attribution, unsupported claims, wrong-version errors, and unanswered requests. Individual failure categories should remain visible alongside aggregate accuracy.
The published study examines 40 historical financial questions, while subsequent controlled evaluations extend the evidence-utilization findings across two model families. Neither yet shows how often these errors occur in live contract or filing workloads. What evidence would you need before generalizing? And as Seekr moves from diagnosing these errors to correcting them, which interventions are you testing (reranking, version-aware prompting, fine-tuning, abstention), and how will you measure whether a correction worked rather than just moved the error somewhere else?
Our research has shown that correct evidence can be retrieved without being used correctly, and our expanded controlled evaluation found the same accuracy degradation under competing context across two model families.
What we don't yet know is how frequently those failures occur across live enterprise workloads.
Our Treasury corpus was selected because documents from different historical periods share remarkably similar structures. That makes it useful for exposing source-selection failures, but we need to establish how those findings transfer to other environments.
I'd want to reproduce the analysis with contracts, regulatory filings, policies, and other enterprise collections. I'd also vary retrieval configurations, question complexity, models, and conversational context.
We're already extending the research beyond single-turn document questions to examine how evidence utilization changes across longer interactions and more complex AI workflows.
Based on the diagnoses, we're designing tests of reranking, version-aware retrieval and prompting, stronger evidence constraints, abstention, and potentially fine-tuning. We don't yet have validated before-and-after correction results.
For each intervention, we'll need matched baselines, held-out tests, and measurements of individual failure categories. Reducing one type of error means little if the change simply produces another.
Outside Seekr, whose work on evaluation, attribution or reliability of grounded AI should production teams be following right now, and why?
There's important work happening across the research community. The ARES research by Jon Saad-Falcon, Omar Khattab, and colleagues examines how to evaluate retrieval relevance, answer faithfulness, and answer quality while accounting for uncertainty in automated judgments. NIST's TREC RAG track is also doing valuable work on independent evaluation of grounded generation, particularly whether cited evidence genuinely supports generated claims.
At Seekr, we're contributing another dimension: understanding how the evidence supplied to an AI system actually influences its behavior. Our Treasury research demonstrated that correct information can be available to a model without being used correctly. Using context attribution, we distinguished failures involving competing sources, unsupported answers, and incompatible evidence. These distinctions matter because they point toward different corrective interventions.
We're extending that work toward evidence-informed optimization, where attribution and evaluation results help determine whether retrieval, prompt re-engineering, fine-tuning, or additional verification is the appropriate response.
The broader question guiding our research at Seekr is: Under what conditions can an AI system's behavior be considered reliable, and what evidence is required to support that judgment?
I think these research efforts are complementary. Establishing reliability requires more than measuring whether an answer is correct. We need to understand what supports that answer, how the system arrived at it, and whether we can reliably correct the behavior when something goes wrong.
Research findings and figures cited in this interview are reported by Seekr. The published study, "Retrieved Does Not Mean Used," is available on Seekr's website.
