Skip to content

When AI Is Convincingly Wrong

Put bluntly, a language model can explain very convincingly why 1 + 1 = 3. It can build a neat argument, lay out intermediate steps, and present the result with the confidence of a textbook.

In that example, the risk is limited. We know the correct answer and spot the error immediately.

The same mechanism becomes much more interesting where we ourselves do not have enough expertise to verify the underlying assumption. A false statement about a rarely used framework API can look like a correct technical explanation. A flawed statement about a transaction boundary can sound entirely plausible to someone without experience in distributed systems. In a security-sensitive question, a false assumption may only be visible to someone who has already analyzed exactly that kind of system.

So the decisive question is not whether we can detect obvious nonsense.

What happens when the human reader does not know that the system has just claimed 1 + 1 = 3?

In everyday language people quickly say that AI has “lied.” As a rhetorical shortcut that may work, but technically it describes the mechanism poorly. A lie normally implies that someone believes a statement to be false and still expresses it with the intent to deceive. We should not casually attribute that kind of intent to a large language model.

A more precise formulation is:

A model can be convincingly wrong.

And that is why one idea matters for the rest of this article:

Plausibility is not a criterion for truth.

Plausibility is not a criterion for truth: an answer can be phrased convincingly and still be either technically correct or based on a false or unsupported assumption.

The most dangerous hallucination does not look dangerous

Section titled “The most dangerous hallucination does not look dangerous”

Obvious nonsense is comparatively easy to handle. If an agent calls a method that is syntactically impossible, the compiler can object. If a generated test does not compile, we get immediate feedback. If a model claims that Berlin is in France, most people do not need another tool before becoming skeptical.

Errors that fit seamlessly into what we already know are much harder.

An answer can sound technically precise, use familiar patterns, be neatly structured, and contain plausible reasoning. Nine out of ten statements may even be correct. Only the tenth statement—the one on which the rest of the solution depends—is false.

That does not make it less problematic. Quite the opposite.

The most dangerous hallucination is the one that requires domain knowledge in order to be recognized as false.

People easily interpret linguistic confidence as a signal of factual confidence. With an LLM, however, we must not equate those two levels. An elegant explanation is, at first, only an elegant explanation. A logical structure does not prove that its premises are correct. A professional-looking code block does not prove that the API it uses actually exists.

One could initially dismiss this as a simple quality problem of individual models. The mechanism goes deeper than that and begins with the way language models learned to generate text.

The term hallucination is now used very broadly. Sometimes it means invented facts, sometimes any incorrect answer, and occasionally even any unwanted decision by an agent.

For a technical discussion, a narrower view is more useful. Not every error has the same cause.

A language model is not a registry of truth

Section titled “A language model is not a registry of truth”

A large language model does not simply contain an internal database like this:

statement
true / false

During pretraining, a model first learns statistical structures and relationships from large amounts of text. Put simply, it learns which continuations are likely under given conditions. Each individual sentence in the training data does not come with an explicit label saying “this is true” or “this is false.”

In a 2025 analysis, OpenAI describes part of the hallucination problem from precisely this perspective: predictable linguistic structures can be learned well from many examples, while rare or largely arbitrary facts do not necessarily provide a stable enough pattern from which their exact form can be reconstructed reliably. Common evaluation methods can also make guessing more attractive than uncertainty when a wrong answer costs less than abstaining.

This does not mean that an LLM merely guesses facts at random. Modern models contain enormous amounts of useful knowledge and can reproduce many relationships with remarkable precision. The important point is different: when reliable information is missing at a particular point, the ability to generate plausible language does not automatically disappear with it.

A knowledge gap does not have to look like a knowledge gap in the output.

A model can therefore know a great deal about a subject and still fail to represent the one piece of information that matters for the task at hand.

This becomes especially visible when knowledge ages quickly.

Frameworks gain new APIs. Libraries change their behavior. Standards evolve. Cloud services replace products. Internal company systems do not appear in public training data at all.

A model may know how Angular components are structured, how dependency injection works, what roles RxJS and change detection play, and which coding patterns were common for many years. Its knowledge of a newly released API may nevertheless be incomplete.

That creates an interesting situation: the model has enough context to produce an answer that looks like Angular. It may even know enough to use the name and basic idea of the new API correctly. But it does not yet know enough detail to assemble the individual parts correctly.

The result can look syntactically convincing and conceptually plausible—and still fail.

The same can happen with rare libraries, proprietary APIs, specialist knowledge, or very recent standards.

Knowing a lot does not protect a model from knowing too little at the decisive point.

Not every wrong answer results from a gap in the model’s knowledge.

An agent can use a highly capable model and still work from poor evidence.

Perhaps repository search finds an outdated implementation. Perhaps an old documentation page describes a concept that has since been replaced. Perhaps key requirements are missing. Perhaps two agent files contradict each other. Perhaps a tool returns an error that the agent misreads. Perhaps a memory contains a decision that no longer applies to the current version of the system.

In such cases, calling everything a hallucination would be imprecise.

The relevant information may even be present in the context and still be weighted incorrectly, combined incorrectly, or passed over in favor of less relevant evidence.

That is better described as a misinterpretation of available evidence.

The distinction matters in practice. Current documentation or search may address missing knowledge. More context does not automatically address poor evidence. More outdated documentation does not make a false conclusion more accurate.

There is a third variation.

The model may have the relevant facts right and still draw the wrong conclusion.

It may correctly identify two transaction boundaries while still making a false claim about atomicity. It may describe two APIs correctly even though their combination is invalid in the lifecycle situation at hand. It may explain a security mechanism accurately and still apply it at the wrong trust boundary.

We should therefore distinguish at least four classes of error:

  • missing or outdated knowledge,
  • genuinely invented or unsupported information,
  • misinterpretation of available evidence,
  • errors in reasoning or deduction.

These categories can overlap in everyday work. For verification, however, it still matters which one we are dealing with.

Knowledge gaps and linguistic confidence are not the same thing

Section titled “Knowledge gaps and linguistic confidence are not the same thing”

Humans are not perfectly calibrated either. We overestimate what we know, misremember things, or answer questions with more confidence than our actual evidence warrants.

A language model adds another difficulty: the tone of an answer does not reliably reveal the strength of the underlying evidence.

A model does not automatically have a perfect internal display that says:

I know this for certain
I know part of this
I do not know this

Research does show that models can contain information about the probability that their own answers are correct. As early as 2022, Anthropic studied whether models could estimate the probability that their own statements were accurate. On suitable tasks and in suitable formats, the results showed useful calibration, but they also showed problems generalizing to new tasks.

This distinction matters. Saying that models can never assess their uncertainty would be just as wrong as assuming that confidence expressed in language is automatically well calibrated.

More recent work therefore examines explicitly whether phrases such as “probably,” “I’m not sure,” or highly confident answers actually correspond to a model’s internal uncertainty. The 2025 EMNLP paper MetaFaith shows that, for the models and methods studied, this linguistic calibration can remain distinctly unreliable without targeted measures.

A knowledge gap and linguistic uncertainty are not the same thing in an LLM.

That is precisely why abstention—the ability not to make a specific claim when evidence is insufficient—has become an optimization goal in its own right.

OpenAI’s SimpleQA results provide an interesting example. Without web access, gpt-5-thinking-mini achieved 22 percent accuracy and a hallucination rate of 26 percent. o4-mini achieved a very similar 24 percent accuracy, but a hallucination rate of 75 percent. The decisive difference was that the newer model refrained from giving a specific answer much more often: 52 percent abstention versus one percent. These numbers do not say that either model has a particular hallucination rate in general. SimpleQA is a deliberately difficult benchmark for short factual questions. But they illustrate very clearly that more answers and better answers are not the same objective.

Anyone who has used LLMs extensively since the early ChatGPT years can easily make two opposing mistakes.

The first is carrying experiences from 2023 over to today’s models unchanged. The second is mistaking the models’ enormous improvement for a complete solution to the problem.

Neither does the development justice.

Early factuality benchmarks make clear why hallucinations so quickly became a dominant topic. In the HaluEval benchmark published in 2023, for example, the authors reported that ChatGPT generated unverifiable information for roughly 19.5 percent of the user questions studied in their specific test categories. That is not a general hallucination rate for ChatGPT in 2023, but a result produced under that benchmark’s methodology.

FActScore examined a very different case in the same year: longer generated biographies were broken down into atomic facts, then checked to determine what proportion was supported by reliable sources. In their human evaluation, the authors reported a FActScore of 58 percent for ChatGPT. This number, too, describes only that particular task and must not be combined or directly compared with HaluEval’s 19.5 percent.

TruthfulQA had already shown how difficult even seemingly simple truthfulness could be. In the original benchmark, the best model studied gave truthful answers 58 percent of the time, compared with 94 percent for humans. TruthfulQA was introduced in 2021, however, and tested older generations of models with questions designed around common misconceptions. The number is therefore historical context, not a comparison value for today’s systems.

More recent measurements, on the other hand, reflect a different level of quality.

The GPT-5 System Card examined open-ended factual questions from LongFact and FActScore both with and without browsing. With browsing, gpt-5-thinking recorded hallucination rates of 0.7 percent for LongFact Concepts, 0.8 percent for LongFact Objects, and 1.0 percent for FActScore under the claim-level metric used there. OpenAI o3 recorded 4.5, 5.1, and 5.7 percent in the same evaluation. Without browsing, the FActScore value for gpt-5-thinking rose to 3.7 percent. These figures apply only to OpenAI’s specific evaluation and grading pipeline, but they show two developments at once: better models reduce errors substantially, and external evidence such as search can improve factual accuracy further.

The development continued. In its August update for GPT-5.6, OpenAI reports further substantial improvements over GPT-5.5 Instant on three deliberately difficult factuality sets: production prompts with a strong factuality component, error cases previously flagged by users, and high-stakes prompts from medicine, law, and finance. Across the three sets, GPT-5.6 Sol is reported to have roughly 60 percent lower factual error rates. OpenAI explicitly emphasizes that these benchmarks select difficult, hallucination-prone cases and do not represent an average error rate for ordinary ChatGPT traffic.

Anthropic reaches a similar picture from a different evaluation direction. The Claude Opus 4.8 System Card examines several closed-book factuality benchmarks on which the model could not use web search or other tools. Among the six models compared, Opus 4.8 had the lowest incorrect rate on all four benchmarks. A substantial part of the improvement came not only from more correct answers, but also from abstaining more often under uncertainty. For the net score (correct minus incorrect), however, the differences from Opus 4.7 were not statistically significant. Anthropic also stresses that factual hallucinations without external tools are still not considered solved.

These benchmarks measure different things. Their absolute values cannot be compared directly; combining them into a single curve for some supposed global “AI hallucination rate” would be methodologically wrong.

What they show in common is the direction.

Modern models are substantially better at factual accuracy than early chat models. Better is not the same as solved.

My Angular experience from the early phase of LLM use

Section titled “My Angular experience from the early phase of LLM use”

One thing I remember particularly well from this period is my early use of LLMs for Angular.

The models generally knew Angular remarkably well. Components, services, dependency injection, RxJS, established change-detection patterns—the models available at the time could already generate very useful code in those areas.

Then Signals arrived.

Angular 16 introduced the new reactive model with Signals as a developer preview in 2023. The API and the patterns emerging around it really were very new.

My personal impression at the time was that, in some areas, the models lagged well behind the current Angular version I was using in practice. This is not a scientific measurement, nor is it a claim that every model of the period was exactly a certain number of years “behind.”

What mattered was the behavior: the model knew a great deal about Angular—just not enough at the decisive point.

I asked for modern Signals-based solutions. The model knew the term, recognized the intention, and combined it with established Angular patterns. The result looked like Angular code. It even looked like Signals code. And yet some APIs did not exist in that form, concepts were mixed together, or RxJS-era patterns were transferred in a way that simply could not work with the new API.

Precisely because so many other parts were correct, you needed to know Angular well enough yourself to spot the error.

My experience today is entirely different. Current models know far more recent framework versions. Agents can search repository code, load current documentation, look up APIs, and then run their implementation against TypeScript, the Angular compiler, and tests.

The underlying mechanism has not disappeared. But the opportunities to avoid a false assumption, or catch it in time, have improved substantially.

A modern coding agent is not merely a language model with a large prompt. Depending on the system, it can use repository search, current documentation, web search, compiler feedback, type checking, test runners, linters, architecture rules, evals, and other tools as part of its process. Missing or uncertain knowledge can therefore be checked against external evidence much more often than in the early chat-only days.

That changes the practical error situation substantially.

Suppose a model does not reliably remember the signature of a current API. A plain chat may have to answer from its parametric knowledge. An agent, by contrast, can read the installed version from package.json, inspect the type definitions, retrieve current documentation, produce an implementation, and then ask the compiler.

From:

I think the API works like this

we ideally move to:

hypothesis
documentation
repository
implementation
compiler
tests
feedback

It would therefore be misleading to judge a modern coding agent solely by the hallucination problems of early chat models.

Tools change the system.

LongFact itself is an interesting example. Its SAFE method breaks long answers into individual facts and checks them against external evidence through search queries. The work does more than introduce a new benchmark; it also demonstrates the underlying pattern: generation and verification can be technically separate steps.

One qualification remains important, however.

Tools do not guarantee truth. The agent can select the wrong documentation. Search results can be outdated. A test can check the wrong property. A compiler confirms type correctness, not correctness with respect to the domain. A green build says nothing about whether a security boundary was modeled correctly.

The mechanism has not disappeared, but both its likelihood and the opportunities for correction have changed dramatically.

At this point, a strange question often appears.

If models can generate more and more code, why do we still need people with decades of experience?

That question reduces software development to code production.

An experienced developer is not simply someone who can type the same method faster. Especially with coding agents, another capability becomes more visible: being able to judge whether a plausible solution actually fits the problem and the system.

An experienced engineer may recognize that an API does not exist in exactly this framework version. They may see that an alleged security fix protects the wrong trust boundary. They may recognize that two individually correct database operations do not together form an atomic operation. They may notice a race condition even though every local code fragment looks reasonable. They may see that a new dependency violates an architectural boundary, or that a locally elegant solution does not fit the system in the long term.

Those are not questions of typing speed.

They require a mental model of both the system and the domain.

The less I know about a domain, the harder it becomes for me to distinguish a convincing hallucination from a convincing correct answer.

That does not mean humans must have been able to generate every answer themselves. I do not need to memorize every detail of a library. But I need enough understanding to know which claims are critical, which assumptions must be verified, and which tool can provide evidence strong enough to trust.

Expertise does not become valuable because humans generate faster. Expertise becomes valuable because it enables better judgment.

This is not an argument against junior developers

Section titled “This is not an argument against junior developers”

None of this should be misread as an argument that only senior developers should work with coding agents.

Experience, after all, does not emerge by keeping people away from difficult problems.

Coding agents can even be excellent learning tools. They can explain unfamiliar code, show alternatives, generate tests, answer follow-up questions, and discuss the consequences of different designs.

The problem begins when generation replaces one’s own thinking process completely.

If someone simply accepts a solution because the tests are green and the code looks professional, they learn less about why the solution works. If that understanding never develops, the very judgment needed later to recognize a plausible but false solution may be missing.

So this is an argument for mentoring, reviews, and deliberate learning—not against junior developers. Experience does not appear overnight; it still needs room to develop.

From chat to agent: the error becomes operational

Section titled “From chat to agent: the error becomes operational”

Up to this point, we could still treat hallucinations largely as a classic chat problem.

A false answer appears in text. A human reads it, verifies it, or rejects it.

Coding agents change that situation.

Article 5 of this series described an agent’s solution path as a sequence of decisions, actions, observations, and changing context. That is exactly what can give a false assumption a new quality.

false assumption
plausible decision
code change
repository state
future context

A false assumption becomes operational inside a coding agent: it shapes a decision, leads to a code change, alters the repository state, and thereby becomes part of the future context.

Assume that an agent misinterprets an architectural rule.

It then creates a new dependency between two modules. The code compiles. The tests remain green. The change may even solve the current ticket.

At that point, the false assumption is no longer just text.

It has changed the repository.

On the next task, the same agent or another one sees this dependency. For it, this is simply existing code at first—local evidence for how this system apparently works.

That is the point at which a single deviation can become something longer-lived.

Not every false assumption gets that far. A compiler, type system, tests, reviews, architectural rules, or an attentive developer can stop the error immediately.

If that does not happen, however, the action changes the context of the next decision.

Repository code has a special meaning for a coding agent.

The agent does not initially know the history behind every line.

It does not automatically know that a particular class was written four years ago under deadline pressure. It cannot see that a strange access exists only because a migration once had to happen in two steps. It does not necessarily know about the review in which someone explicitly wrote: “Please do not copy this as a pattern.”

At first, it sees code.

That means a one-off workaround can become local evidence:

one-off workaround
committed
existing repository code
interpreted later as a pattern
reproduced again
pattern increasingly appears established

The phenomenon obviously exists without AI as well. Developers have copied existing code for decades. Legacy systems often grow consistently in the wrong direction precisely because a local decision is repeatedly used as a template.

Agents mainly change the speed and scale of this mechanism.

AI does not make bad architecture possible in the first place. But it can make bad architecture become consistent much faster.

The good news is that the same reinforcement effect works in the other direction.

If a codebase has clear boundaries, uses consistent patterns, keeps its examples maintained, and checks architecture rules mechanically, an agent finds that local evidence too.

Then the loop becomes:

good pattern
existing repository code
local evidence
reproduction
more consistency

Coding agents therefore do not inherently amplify bad architecture. They first amplify the signals presented to them as local evidence by the repository, documentation, tests, and rules.

The increasingly interesting architecture question is what signals we give them.

Broken windows and drift in a codebase: a one-off workaround can become local evidence after it is committed, be reproduced later, and gradually harden into what looks like an established pattern. The same reinforcement mechanism can also strengthen good patterns.

Hallucination, misinterpretation, and drift are not the same

Section titled “Hallucination, misinterpretation, and drift are not the same”

At this point it helps to separate three related but different concepts cleanly.

A hallucination, in the sense used in this article, is a false or unsupported statement or assumption. The model may claim, for example, that an API exists even though it does not.

A misinterpretation occurs when available information is weighted or combined incorrectly. The documentation exists, but the agent applies a rule from version 18 to version 21 or treats a legacy workaround as the current architectural rule.

Drift, by contrast, describes a process across multiple steps. A solution path or the resulting architecture increasingly moves away from the originally intended direction.

A hallucination can trigger drift.

But it does not have to.

hallucination
compiler error
correction

There is no relevant drift here.

Conversely, drift can emerge entirely without a classic hallucination.

A local decision may be legitimate in isolation. It is later reproduced several times. That changes the structure of the codebase. Other changes begin to follow it. An exception becomes a convention even though nobody ever invented a false fact.

Drift is therefore less a single model error than a feedback mechanism.

A simplified sequence looks like this:

deviation
accepted
becomes part of the codebase
becomes future context
reproduced again
new local convention

The important point is the feedback.

An agent does not only produce output from a context. Through its actions, it changes the very environment from which future context will later be retrieved.

Agent Decision
Repository State
Context
Agent Decision

This is not some mystical “AI drift.” It is a feedback loop between decisions and an environment that those decisions themselves reshape.

That is why early deviations are particularly interesting. One additional dependency may look harmless. Ten later changes that treat it as precedent can turn it into a new structure.

Article 5 showed that result variance initially means different plausible solution paths. On its own, that is not yet drift. But if an agent keeps working on a plausible false assumption and derives further decisions from it, deviations can reinforce each other over a longer chain of actions. The longer and more autonomous that chain becomes, the more important explicit boundaries, verifiable intermediate results, and feedback become.

Safety-critical questions need more than plausibility

Section titled “Safety-critical questions need more than plausibility”

For an ordinary UI refactoring, a false assumption is annoying.

For safety-critical decisions, “looks plausible” is fundamentally insufficient.

An agent with limited knowledge of a concrete security question does not necessarily have to answer blindly. A well-constructed agent system can load additional documentation, search internal policies, use tools, verify concrete facts, ask follow-up questions, or refuse to act and escalate to a human when the evidence is insufficient.

The decisive phrase, however, is: a well-constructed agent system can do that.

Those are properties of the overall system, its tools, its training objectives, and its guardrails. They do not follow automatically from the mere presence of a powerful LLM.

Even abstention is not trivial: the system must not only find information, but also recognize when the available evidence is insufficient for a reliable answer.

That is why one principle applies especially strongly in high-risk domains:

For safety-critical decisions, linguistic plausibility must never be the only proof of quality.

We need evidence, technical checks, and, where appropriate, a qualified human.

Generation and verification are different tasks

Section titled “Generation and verification are different tasks”

Coding agents substantially reduce the cost of additional generation. Another implementation variant, another test, or another hypothesis can often be produced today with far less effort than it would take manually.

That shifts part of the work.

The question is no longer exclusively:

How do we generate a solution?

Increasingly, it becomes:

How do we know that this solution is acceptable?

A compiler verifies one class of properties. A type system verifies another. Tests only check what was actually specified and implemented. Architecture tests can enforce dependency rules. Security scanners can find certain known classes of problems. Reviews contribute additional domain expertise.

None of these tools is a truth machine on its own.

Together, however, they can substantially reduce the space of solutions that are plausible and wrong at the same time.

The cost of generation can fall while the importance of verification rises.

We will return to this relationship in more detail later in the series. For now, one observation is enough: if a system can produce many plausible solutions very quickly, the ability to distinguish among them does not become less important.

It becomes more important.

Hallucinations were among the most visible weaknesses of early LLM systems. Anyone who has followed the development since 2023 should, however, also acknowledge how much their practical significance has changed.

Models have become better at factuality. Abstention is trained more explicitly. Context is used more effectively. Search and retrieval provide current information. Coding agents can use compilers, type systems, and tests as external feedback channels. Current system cards continue to show factuality improvements over their respective predecessors.

We can be impressed by that progress without minimizing the underlying problem. A model can know a great deal and still lack the one decisive piece of information. An answer can be excellently written and still rest on weak evidence. A locally plausible agent decision can change the repository, become evidence for later decisions, and turn isolated accepted deviations into longer-term drift.

Plausibility is not a criterion for truth.

Article 5 showed that an agent’s solution path is not deterministic.

Article 6 adds:

A plausible solution path can rest on a false assumption.

The answer cannot be to specify every individual decision an agent will make in advance. If we did that, we would not need the agent.

Instead, we need to describe more clearly which solutions are acceptable at all, which properties are non-negotiable, and which signals make a wrong path visible early.

That is where the next article begins.

The first of those boundaries is requirements.


OpenAI – Why Language Models Hallucinate, 2025. This paper examines statistical causes of hallucinations and the effect of evaluation mechanisms that can reward guessing over abstention. It is particularly relevant to explaining why a knowledge gap does not automatically appear as uncertainty in the output. Why Language Models Hallucinate – OpenAI

Li et al. – HaluEval, EMNLP 2023. HaluEval studies the generation and detection of hallucinated content. The roughly 19.5 percent cited in this article refers to its specific study of ChatGPT answers within the benchmark; it is not a general product-wide hallucination rate. HaluEval – ACL Anthology

Min et al. – FActScore, EMNLP 2023. FActScore breaks long answers into atomic facts and measures the proportion supported by a reliable knowledge source. The 58 percent cited for ChatGPT comes from the human evaluation of generated biographies. FActScore – ACL Anthology

Lin, Hilton, Evans – TruthfulQA. This benchmark focuses on questions for which common human misconceptions can encourage a false answer. Its historical results provide context for early truthfulness problems, not a metric that can be compared directly with current models. TruthfulQA – OpenAI

Wei et al. – Long-form factuality in large language models / LongFact & SAFE, 2024. LongFact measures factual accuracy in open-ended, long-form answers. SAFE breaks answers into facts and checks them with search assistance. The work is particularly relevant to the separation of generation from external verification. Long-form factuality – Google DeepMind

OpenAI – GPT-5 System Card, 2025. The LongFact, FActScore, and SimpleQA values cited here come from the specific evaluations in this system card. OpenAI uses different settings with and without browsing as well as its own grading methods, so the values must not be read as general hallucination rates. GPT-5 System Card

OpenAI – GPT-5.6 August Update, 2026. This update reports factuality evaluations on deliberately difficult prompt sets, including production prompts with a strong factuality component, error cases previously flagged by users, and high-stakes questions. The values explicitly do not represent an average production error rate. GPT-5.6 August Update

Anthropic – Claude Opus 4.8 System Card, 2026. Anthropic evaluates factual hallucinations using four closed-book benchmarks, among other tests, and considers correct, incorrect, and abstention behavior. Opus 4.8 has the lowest incorrect rate among the six models compared on all four benchmarks; frequent abstention under uncertainty accounts for a substantial part of the result. The net score does not differ statistically significantly from Opus 4.7. Claude Opus 4.8 System Card, section 6.3.3.1

Kadavath et al. – Language Models (Mostly) Know What They Know, 2022. This paper examines self-evaluation and calibration in language models. It shows that models can contain information about the probability of their own correctness, but that this ability depends on the task format and does not generalize perfectly. Anthropic Research – Language Models (Mostly) Know What They Know

Liu et al. – MetaFaith, EMNLP 2025. This paper examines whether natural-language expressions of uncertainty reliably reflect actual model uncertainty and documents substantial remaining calibration problems in the models studied. MetaFaith – ACL Anthology

Feng et al. – Don’t Hallucinate, Abstain, ACL 2024. This paper explicitly addresses gaps in knowledge and methods intended to make models abstain instead of answering when they lack the necessary knowledge. Don’t Hallucinate, Abstain – ACL Anthology