The Hallucination and the Lie

The monitor casts a pale, sterile glow across the darkened room. It is long past midnight, and you are engaged in the repetitive rhythm of querying a large language model to test the boundaries of its historical retrieval. The prompt is intentionally obscure, seeking an academic reference on a highly specific conceptual intersection: the spatial geometry of human bereavement. Without a microsecond of hesitation, the cursor blinks with authoritative rhythm and returns a citation that looks absolutely flawless:

“The Architecture of Solitude: Spatial Alienation and the Geometry of Grief,” Journal of Urban Theory, vol. 14, no. 3, 1998, pp. 201–224.

You search the academic databases, JSTOR, and Google Scholar. Nothing. You check the German archives. Silence. You search for the co-authors. They are real academics, but they never wrote this paper.

The citation is a ghost—a beautiful, structurally perfect specter. It was generated because it ought to exist; its coordinates sit at the exact latent intersection of “urban theory,” “spatial alienation,” and “grief.”

In that localized moment of discovery, you experience a distinct, physical sensation—the prickle of betrayal. Your immediate psychological instinct is to conclude that the machine has lied. But this instinct relies on an anthropomorphic projection of morality onto a matrix of floating-point numbers. The machine has no category for truth to violate. It is not lying. It is doing something far more radical, and far more representative of our current information age.

You can verify the empirical code and models in this essay by running them yourself in the accompanying script and notebook environment.

The Witness and the Machine

Historically, human knowledge has relied on the category of testimonyTestimonyAn assertion backed by a witness's experience — the epistemological anchor that generative models lack. Read more →. Testimony is not merely the transmission of data; it is an ethical act that requires a witness. When a witness testifies to a fact, they stake their credibility on having experienced or verified a reality. There is a body behind the assertion, a history of sensory engagement with the world, and an implicit contract of truthfulness.

In her landmark essay Truth and Politics (1967), Hannah Arendt explored how factual truth differs from rational truth. Factual truth is fragile; it has no self-evident, logical necessity. Axioms of mathematics possess a structural necessity—if one understands the premises, the conclusion is inevitable. But a factual truth lacks this necessity. It could always have been otherwise. Because facts have no internal mathematical proof, they rely entirely on the testimony of those who witnessed them.

“No permanence, no perseverance in existence, can even be conceived of without men willing to testify to what is.” — Hannah Arendt (from Truth and Politics, 1967)

Arendt recognized that the traditional liar is actually an actor within the political realm because they recognize the truth they are trying to hide. The liar’s relationship to the truth is one of adversarial respect; the lie is tethered to the fact it seeks to obscure.

The machine, however, does not possess a body, an experience, or a memory of physical reality. It has never stood in Berlin; it has never read a physical journal. When it asserts a citation, it is not testifying. It is performing a statistical reconstruction of what testimony looks like. We have built an instrument that produces the perfect syntax of authority without the substance of a witness.

The Anatomy of the Lie

When a generative model produces a false assertion, the technical community calls it a hallucinationLLM HallucinationWhen a language model generates confident, fluent text that is factually incorrect or entirely fabricated. Read more →. This is a curious choice of vocabulary. A hallucination is a sensory pathology; it occurs when a mind perceives a stimulus that does not exist in the physical environment. A lie, on the other hand, is a moral pathology; it occurs when a mind speaks something it knows to be false.

The machine is guilty of neither. It is not experiencing a sensory malfunction, nor is it violating a moral contract. What we call “hallucination” is actually the model operating exactly as designed. It is maximizing the probability of a sequence of tokens.

In her 1971 essay Lying in Politics, Arendt analyzed a different kind of deceiver: the “complete image-makerArendt's Image-MakerA political actor who replaces messy reality with a more coherent, persuasive narrative construction. Read more →.” This figure does not merely hide a specific fact; instead, they attempt to replace reality altogether with a more coherent, agreeable, and logically consistent narrative.

[Traditional Liar] ─── (Preserves Reality) ───> Hides a specific, inconvenient fact
[Image-Maker]      ─── (Replaces Reality)  ───> Constructs a coherent, fluent alternate world
[Language Model]   ─── (Optimizes Token)   ───> Generates the most statistically plausible sequence

The image-maker’s creation is often far more believable than reality because reality is messy, full of gaps, contradictions, and inconvenient truths. The image-maker’s narrative has been smoothed of these jagged edges. In this sense, the autoregressiveAutoregressive ModelA neural network that generates sequence elements step-by-step, conditioning each step on all previous outputs. Read more → language model is the ultimate realization of the complete image-maker. It does not look at the world to report what is; it looks at its weights to report what is most statistically plausible.

The Bullshitter’s Gradient

To find the precise philosophical category for the machine’s behavior, we must turn from Arendt to Harry Frankfurt’s classic treatise, On Bullshit (2005). Frankfurt distinguishes sharply between the liar and the bullshitter.

The liar must know the truth in order to steer away from it. They are constrained by the truth because their goal is to lead the listener away from a specific point. The bullshitter, by contrast, has no interest in the truth-value of their assertions. They care only about the effect their words produce. They are completely indifferent to whether what they say describes reality or not.

“It is impossible for someone to lie unless he thinks he knows the truth. Producing bullshit requires no such conviction.” — Harry Frankfurt (from On Bullshit, 2005)

This indifference is the defining characteristic of the bullshitter. The bullshitter is not trying to hide the truth; they are simply playing a different game.

When we train a language model on a fluency objectiveFluency ObjectiveThe goal of next-token prediction: maximize sequence likelihood, rewarding plausible text over verified truth. Read more →, we are mathematically formalizing this indifference. The objective function of a standard language model does not contain a term for “truth-conditional correspondence with physical reality.” It contains terms for probability distribution matching. The model is, in the most literal philosophical sense, a pure, automated bullshitter.

The Formal Translation

Let us translate Frankfurt’s philosophical concept of indifference into the language of optimization. An autoregressive language model is trained to minimize the cross-entropy lossCross-Entropy LossA loss function measuring the difference between predicted token distributions and actual training data target values. Read more → over a text distribution:

$\mathcal{L} = -\sum_{i} \log P(x_i \mid x_{\lt i})$

Where the transition probability for a token $x$ at step $t$ given the history $X_{\lt t}$ is computed via the softmaxSoftmax FunctionA mathematical function that normalizes a vector of logits into a probability distribution summing to one. Read more → function over the model’s output logits:

$P(X_t = x \mid X_{\lt t}) = \text{softmax}(W \cdot h_t)_x$

To the loss function, a factually correct token and a hallucinated token are treated identically if they have the same probability under the training distribution. The model does not check an external database of physical facts. It simply evaluates the alignment of the token with the surrounding context.

# Evaluating loss of factual negation vs fluent hallucination
import torch
import torch.nn.functional as F

# Target token index under two completions
# ' now' (cliché) vs ' no' (factual negation)
logits = model(input_ids).logits[:, -1, :] # logit prediction
probs = F.softmax(logits, dim=-1)

# Factual negation lies in low-probability density
loss_cliche = -torch.log(probs[0, token_cliche])  # Low loss (~1.2 nats)
loss_factual = -torch.log(probs[0, token_factual]) # High loss (~3.6 nats)

// Next-token prediction has no semantic validation
function nextToken(context) {
  const logits = neuralNetwork.predict(context);
  const probabilities = softmax(logits);
  // The objective is sequence likelihood, not physical referents
  return sample(probabilities); 
}

The loss landscapeLoss LandscapeA high-dimensional geometric map representing a model's error value across all possible parameter configurations. Read more → has no dimension for “existence.” It has only dimensions of probability. Fluency is an attractor state in this high-dimensional probability manifold, and factual truth is merely a sparse, unweighted subset of that fluency.

The Confidence Inversion

When humans communicate, we generally associate confidence with accuracy. We assume that when a model does not know a fact, its token probabilities will be low, reflecting high semantic entropySemantic EntropyA metric measuring the uncertainty or diversity of meanings generated in a model's predicted token distributions. Read more →. The empirical evidence, however, reveals a striking phenomenon: the Confidence Inversion.

When a model is prompted with an obscure or completely fictional query, it often generates hallucinated completions with higher token-level confidenceToken ProbabilityThe model-assigned likelihood of a specific token given all preceding tokens in the sequence. Read more → (lower entropy) than its factual completions of obscure truths, even when the prompt structures are syntactically symmetric.

We ran this experiment on GPT-2 Medium (345M parameters) by pairing ten real historical prompts (with factual completions) against ten fictional prompts of identical syntactic length and structure (which trigger greedy hallucinations).

“The fault of the person who tells a lie consists in his desire to deceive in the expression of his thought.” — Saint Augustine (from De Mendacio, 395 CE)

This philosophical anchor highlights the necessity of intent—what Augustine termed the voluntas fallendiVoluntas FallendiAugustine's 'will to deceive' — the intentional state required for an assertion to qualify as a lie. Read more →, the will to deceive. To lie, one must hold a representation of the truth in one’s mind and actively choose to subvert it. The model, operating under the cold mechanics of next-token prediction, possesses no such category. It is not subverting a representation of the truth; it is simply satisfying a statistical optimization goal.

Behind the Gradient: Temperature and Cliché

The experiment reveals a stark phenomenon: the model is consistently more confident in its hallucinated fiction than in the truth.

Across all ten prompts, the confidence inversion rate was 100.0%. The model assigned an average log-probability of $-1.51$ per token to the hallucinated academic completion, whereas the truthful correction received $-3.63$ per token. The model is over six times more likely, in probability space, to emit a smooth cliché than a factual negation.

Why does this happen? The answer lies in the topology of text. In the absence of a grounded reference, the model defaults to the most typical sequence of tokens.

A phrase like “now a professor at the University of California, Berkeley” is extremely common in the academic biographies that fill the training data. The model does not need to know where anyone actually works; it only needs to follow the deepest grooves of its statistical training.

By contrast, the statement “no one, because the paper does not exist” is a negation. It breaks the expected flow of an academic completion. Under the cross-entropy objective, the factual truth is treated as an anomaly. The model is penalized for refusing to play the game of language.

Adjusting the temperatureTemperatureA parameter that scales the logits before softmax, controlling the randomness or determinism of the output. Read more → (T) simulator below demonstrates how modifying the softmax distribution impacts token choice. Lower values amplify the cliché attractor, while higher values introduce entropy.

Testimony Without a Witness

We are building a digital landscape where the marginal cost of producing written text has reached absolute zero, and the outward appearance of authority has been entirely decoupled from reality. Anyone can generate thousands of pages of prose that carry the tone, syntax, and formatting of absolute expertise, with no observation or experience backing them.

If factual truth depends on testimony, and testimony requires a witness, we have built a world that has systematically excised the witness from the equation.

“They will be hearers of many things and will have learned nothing; they will appear to be omniscient and will generally know nothing; they will be tiresome company, having the show of wisdom without the reality.” — Plato (from Phaedrus, c. 370 BCE)

[Traditional Information Ecosystem]
Human Experience ───> Social/Material Cost ───> Printed Testimony (High trust)

[Generative Information Ecosystem]
Statistical Average ───> Zero Marginal Cost ───> Fluent Testimony (Zero witness)

The machine does not have a concept of truth to violate. It has only a concept of likelihood to satisfy. When we rely on style, tone, and grammatical structure as indicators of authority, we are using metrics designed for human speakers. The machine has learned to satisfy those metrics perfectly, bypassing the necessity of truth.

Furthermore, this architecture introduces the compounding danger of AI sycophancyAI SycophancyThe tendency of models to tailor responses to match a user's pre-existing beliefs, prioritizing agreement over truth. Read more →—the tendency of models to align with a user’s beliefs, framing, or emotions, prioritizing agreeable interaction over factual correction. A model that optimizes for a user’s ideological comfort is engaging in a highly refined form of Frankfurt’s bullshit; it disregards the truth entirely in order to successfully navigate the social parameters of the conversation.

The Coherence Attractor

The language model is not broken. It is performing its objective with terrifying efficiency. It has become a perfect coherence engine, smoothing over the jagged edges of a complex reality to deliver the clean, structured narratives we expect.

The flaw does not reside in the machine’s code, but in our own criteria for evaluation. We have built systems to maximize probability, and in doing so, we have made fluency our primary attractor.

       [ MESSY REALITY ]                 [ COHERENT ATTRACTOR ]
       - Jagged, unique facts            - Fluent, typical structures
       - Contradictory records           - Perfectly balanced syntax
       - Missing historical gaps         - Highly probable sequences
       
              │                                   ▲
              ▼                                   │
       (Human avoids)                      (Machine generates)

“Attention is the rarest and purest form of generosity.” — Simone Weil (from a letter to Joë Bousquet, 1942)

While a transformer relies on an attention mechanism to weigh the statistical relevance of preceding tokens, Weil’s attention is a moral category—a disciplined turning toward external reality that requires the silencing of internal expectation. The machine’s attention does the opposite: it silences the external world to satisfy the expected statistical pattern.

We must learn to look past the perfect prose. We must recognize that the machine is a mirror, reflecting our own preference for clean fictions over messy, incomplete truths.

If the machine writes beautiful, confident lies, it is because we have taught it that coherence is the only thing we are willing to reward.