When an AI Says It Is Confident, What Does That Number Mean?

What happened

When a large language model says it is “80% confident,” the number may sound precise. But where did it come from? Is the model reporting something about its own internal state, repeating the style of confidence statements in its training data, or simply producing a persuasive-looking number?

A new Apple research paper, Verbalized and Internal Probabilities Are Coupled in Large Language Models, investigates that distinction. The authors—Sinead Williamson, Jiaxuan Li, Nick Foti, Russ Webb and Masha Fedzechkina—compare two probability readouts. An internal probability comes from the model's token distribution: how much probability it assigns to one answer rather than another. A verbalized probability is the confidence the model states when asked.

Williamson also shared the work publicly on X, where it became one of the research leads monitored by Haivian Discovery News. The paper was submitted to arXiv on 30 September 2026. It is a preprint, not a report of an independent Haivian audit.

The central result is more interesting than a simple claim that the two numbers correlate. The researchers report evidence that internal and verbalized probabilities respond to both observed frequencies and explicit probability statements, and remain aligned after the target probability is statistically controlled. Their interpretation is that the two readouts can draw on a shared representation of uncertainty.

What the evidence shows

The study combines observation with intervention. First, the researchers measured both readouts across 21 open-weight models from four model families, ranging from 0.5 billion to 32 billion parameters. On forced-choice versions of five question-answering datasets, most models larger than two billion parameters showed significant agreement between internal and verbalized confidence.

Correlation alone would not establish coupling. Two readouts might agree because both independently track the same easy and difficult questions. To probe causality, the team trained small transformers in a controlled synthetic environment. Some fictional events appeared only through repeated observations; others appeared only through explicit probability statements. The models could transfer between those two representations. In one reported experiment, verbalized probabilities for frequency-only events reached a concordance correlation coefficient of 0.91 with the target probabilities, while internal probabilities for assertion-only events reached 0.74.

The authors then introduced uncertainty about fictional people through fine-tuning or in-context examples. Both the model's stated confidence and its internal probability distribution shifted in response to observed frequencies. Both also shifted in response to written probability assertions.

That latter result is double-edged. When frequency evidence and explicit claims disagreed, both readouts moved toward the asserted probability. In the in-context condition, the paper reports that a 25% share of asserted statements was already enough for both readouts to align with the assertion. Under training, the frequency signal was more resistant at that mixture, but the assertion became dominant by 40%.

The paper therefore supports a bounded conclusion: asking a model for confidence can reveal something connected to its internal distribution. It does not establish that the stated number is calibrated to reality, nor that verbalized confidence is safe to treat as a truth score.

What remains uncertain

The experiments deliberately simplify the world. As the authors note in their limitations, much of the causal evidence comes from clean synthetic datasets in which frequency-based and assertion-based signals can be separated. Real training corpora are messier. Claims repeat, sources copy one another, and confidence can correlate with authority, popularity or writing style rather than accuracy.

The study also focuses on binary outcomes and uses token logits as the internal probability measure. Open-ended generation, multi-step reasoning and semantic uncertainty are more complicated. A model may assign probabilities across many phrasings of the same idea, and a single next-token distribution may not capture uncertainty over the final answer.

Distribution shift is another open test. Haivian's analysis is that coupling should be examined when prompts, domains and incentives change—especially when a model is rewarded for appearing decisive. The present paper does not show that verbalized confidence remains reliable under adversarial prompting or in safety-critical deployment.

There is also a warning inside the reported result. The authors found that confident statements about one attribute could sometimes increase confidence about unrelated attributes of the same fictional person. They describe the effect as reduced when richer contextual detail was provided. Even so, it suggests that confidence can spread through semantic association rather than evidence.

Why independent scrutiny matters

If confidence scores are used to rank answers, trigger escalation or decide whether a human should review a result, a technically plausible relationship between internal and verbalized uncertainty is only the start. Independent evaluation should test calibration, robustness across model families, sensitivity to prompt wording, failure under contradictory sources and performance outside binary question answering.

The paper gives researchers a useful experimental framework for separating “what the model says” from “what its output distribution does.” It also shows why neither quantity should be accepted without tracing the evidence that shaped it.

Haivian offers bounded independent research audits for mathematical, scientific and AI claims. An audit tests a defined claim and its evidence; it does not guarantee validation, certification or publication.