Skip to content
Insights/Article

When the Knowledge Base Has No Answer, How Should AI Say It Does Not Know?

Key takeaway

Separate four cases: no material, insufficient evidence, no access permission, and expired content. The reply states what was searched, what is missing, and whom to ask or what to provide. Without evidence, say so plainly instead of filling in with general knowledge. Do not reveal whether restricted material exists.

Four abstention types when AI faces a knowledge gap: not covered, insufficient evidence, no permission, expired, with next steps

Suppose the following happens. A customer asks whether a special clause applies to their order. AI answers specifically: it applies, requires seven days' advance application, and lists two conditions. Sales forwards the answer to the customer. Legal later checks and finds no such rule in the knowledge base. AI stitched several similar clauses into a plausible-looking answer.

Inventing an answer costs more than saying "I do not know." With an abstention, the customer asks again or reaches a person. With an invented answer, the customer makes decisions on it, and by the time anyone discovers the problem, a promise, quote, or contract exists.

Four kinds of "no answer," handled differently

One: not in the material. The knowledge base does not cover the topic, or the question is out of scope, such as a service never launched.

Two: insufficient evidence. The material mentions the topic but does not answer the specific question. A manual covers warranty scope but omits a regional exclusion.

Three: no access. The material exists, but the current user lacks permission, such as a frontline agent asking about a management-only pricing policy.

Four: expired. The material is found but past its validity or replaced by a new version, such as last year's price list.

The replies differ: the first suggests adding material or handing over; the second names the missing part; the third withholds content and points to an access request; the fourth states the expiry and the path to the current version.

Three parts of a useful "I do not know"

What was searched: which documents and scope were checked. The user learns it was searched, not skipped.

What is missing: whether the topic is absent entirely, or a specific condition, region, or version is missing.

Next step: whom to ask, what to provide, which entrance to check, or whether to hand over.

For example: "I did not find a warranty exclusion for this model in high-altitude regions in the current product manual or after-sales policy. The available material covers standard regions only. Confirm with the after-sales policy owner, or submit a ticket for human review." That is far more useful than "unable to answer at this time."

Do not hide gaps with vague wording

NIST AI 600-1, section 2.2 addresses confidently presented erroneous output. The four abstention types here are a business-design classification.

"Possibly," "generally speaking," and "consult the relevant department" without evidence still produce an uncertain answer. Users read "possibly applies" as "probably applies."

Without evidence, say so. General background may be offered but must be labeled as background, not company policy or product commitment. Keep background knowledge and company facts separate.

Permission and existence

Without access, do not say "the material exists but you cannot see it" unless the permission rule explicitly allows revealing existence. A safer reply: "This account does not have permission to view this policy. Contact this role to request access." It neither reveals content nor hints at what exists.

If the user keeps asking, keep the same wording. Do not leak information just because the question is phrased differently.

Answering with expired material

When found material is expired, neither use it as the answer nor pretend it was not found. Say: "The 2025 price list was found; it was replaced in January 2026. The current version was not retrieved from the knowledge base. Confirm with this role."

If the system can locate the current version, cite it first and mark the old one as historical reference. When it cannot determine which is current, present both and let a person decide.

A hypothetical reply comparison

Question

Bad reply

Acceptable reply

Is the warranty excluded in high-altitude regions for this model?

"Generally high-altitude regions are not covered; apply in advance." (No basis)

"The current manual covers standard regions only and states no high-altitude exclusion. Confirm with the after-sales policy owner, or submit a ticket for human review."

Can last year's customer renew at the new price?

"Yes, existing customers usually get the new price." (Guess)

"The knowledge base contains only the 2026 price list and no transition rule for renewals. Provide the contract number and I will route it to contract management."

What is the management discount floor?

"About 30 percent off; negotiable." (No permission)

"This account does not have permission to view the pricing policy. Contact the sales director to request access."

All three acceptable replies share one trait: they do not invent, blur, or overreach. They may not satisfy the user, but they leave the user with a next step instead of a wrong answer.

If the user rephrases the same question, keep the same answer; do not produce an unverified answer just because the wording changed. Repeated unanswered questions are a signal to add source material or a new entry.

State it once, clearly.

Test How the Assistant Responds Without Evidence

[AI abstention checklist]

1. Classification
- Does the system distinguish not covered, insufficient evidence, no permission, and expired:
- Does each type have a reply pattern:
- Is the abstention reason recorded:

2. Reply
- Does it state the searched scope and missing content:
- Does it name the next contact or material:
- Does it avoid hiding gaps with "possibly" or "generally":

3. Permission
- Does it withhold content and existence when access is denied:
- Does it keep the same wording when asked again:
- Is the access request path clear:

4. Expiry
- Does it state the version and replacement date:
- Can it locate the current version:
- Does it hand over when it cannot determine the current version:

Take ten real questions the knowledge base cannot answer and write a reply for each of the four types. If half end with "consult the relevant department," the reply has not provided a real next step. The scope of enterprise knowledge Q&A and abstention design can be discussed within AI applications and workflows.

Why knowledge bases fail overall is in six common reasons an enterprise AI knowledge base fails; expiration governance is in what to do when knowledge expires; citation-ready content is in what content AI is more likely to cite.

When available documents disagree, use the conflicting-source answer structure. If a person must decide, preserve the handoff context.

Sources

Let's talk about your project

Websites / Business systems / AI applications

Chat on WhatsApp