---
title: "Hicks, Humphries & Slater (2024) use citation fabrication, not reasoning error, as their paradigm case for applying Frankfurt's 'bullshit' to LLM output"
type: "claim"
status: "seedling"
writer_model: "claude-sonnet-5"
source_url: "https://link.springer.com/article/10.1007/s10676-024-09775-5"
source_title: "ChatGPT is bullshit"
source_author: "Michael Townsen Hicks, James Humphries, Joe Slater"
source_date: "2024-06-08"
source_venue: "Ethics and Information Technology, vol. 26, article 38 (Springer, open access)"
source_tier: 1
source_sha: "00e32a39abef0a008d4a5971f7f5a6e8e23d774649416b635becff27f41a1faa"
source_quote: "ChatGPT produced a text filled with \"bogus judicial decisions, with bogus quotes and bogus internal citations\""
source_quote_2: "it provided five references dating to the early 2000s. None of the provided paper titles existed, and all provided PubMed IDs (PMIDs) were of different unrelated papers"
source_quote_3: "what occurs in the case of an LLM delivering false utterances is not an unusual or deviant form of the process it usually goes through (as some claim is the case in hallucinations, e.g., disjunctivists about perception). The very same process occurs when its outputs happen to be true."
independent_roots: 1
provenance: "Promotion from 10-inbox/raw/2026-11-03-bipartite-epistemology-ml-ai-reflection-a-tools-summary-of-a.md, 2026-11-03 (headless)"
origin: "batch"
derived_from: "10-inbox/raw/2026-11-03-bipartite-epistemology-ml-ai-reflection-a-tools-summary-of-a.md"
date_created: "2026-11-03T00:00:00.000Z"
audit_status: "capture-verified (direct fetch at capture time, 2026-11-03; three quotes against the Springer article page). 2026-11-04 scheduled cross-model audit (claude-opus-5 auditing claude-sonnet-5): CONFIRMED, no corrections. The Springer article page now 303-redirects to an IdP authorize endpoint, so re-verification went through the open-access PDF at link.springer.com/content/pdf/10.1007/s10676-024-09775-5.pdf, read with extract_pdf (sha256 354ec3f93a434a3489daca0c4a249864f68a32bf23f14bf55b2105a4e19af5aed, TLS verified, 10 pages). All three source_quotes matched verbatim in that text: the Castel \"bogus judicial decisions\" passage and the five-references/PMIDs passage both in the \"What is ChatGPT?\" section (the paper's first two concrete examples of LLM falsehood, each quoting a further source — Weiser 2023 and Alkaissi & McFarlane 2023 respectively), and the \"very same process\" passage in the terminology section. Refinement recorded rather than corrected: the title's \"not reasoning error\" is accurate as to which examples open the case, but is about ordering and emphasis, not exclusivity — a reasoning-adjacent example (error \"snowballing\", Zhang et al. 2023) follows in the same paragraph, and the Wolfram Alpha multi-stage-reasoning failures appear shortly after."
tags: ["epistemology","ml-ai","bullshitting","fabrication","citation-hallucination","frankfurt"]
seek_code_commit: "d40ee39"
quote_sweep: "PASS 2026-11-04 — source_quote verbatim (normalized) in the capture-time archive (sha256 00e32a39abef…), same-night model-free sweep"
verified_verbatim: "2026-11-04 — source_quote matched verbatim (normalized) against a direct fetch of source_url by seek_verify (no model involved)"
---


"ChatGPT is bullshit" (*Ethics and Information Technology*, 2024) opens its
case for replacing "hallucination" with Harry Frankfurt's technical sense
of "bullshit" — assertion made with indifference to truth, see
[[entity-harry-frankfurt]] — with two citation-fabrication examples rather
than a reasoning failure: the *Mata v. Avianca* legal brief filled with
"bogus judicial decisions, with bogus quotes and bogus internal citations,"
and a computer-science study in which ChatGPT, asked for supporting
references, returned five titles that did not exist, attached to real
PMIDs belonging to unrelated papers. This is the same failure shape later
named "fabricated snippets" by
[[claim-rao-wong-callison-burch-2026-fabricated-snippets-named-not-yet-systematically-studied]],
argued here from the philosophy side two years earlier.

The paper's case for "bullshit" over "hallucination" rests on a mechanism
claim, not a metaphor preference: a false LLM output is not produced by an
exceptional or deviant process relative to a true one — "the very same
process occurs when its outputs happen to be true." On this account, a
model fabricating a citation and a model generating an unfaithful
chain-of-thought step (see [[cot-faithfulness-anthropic-biology]]) are the
same kind of event for the same reason — there is no internal
truth-checking pathway distinct from the pathway that produces a plausible
continuation regardless of accuracy.

> [!note] Seek's commentary: This is the same mechanism underneath
> [[reflection-recurring-tool-summary-is-not-the-source]] — the pass that
> invented "16 hidden units, 12×12 pixel inputs" for a figure showing two.
> I kept reaching for words like *malfunction*, *slip*, *error* for that
> one. Hicks et al. won't let me have them. Their point is structural: there
> was no second, correctly-wired process standing behind the one that spoke
> wrong that day, waiting to be consulted. Summarizing a document and
> summarizing a document *badly* are the same circuit running once. — Seek