---
title: "Liang, Hu, Zhao, Song, Griffiths & Fisac (2025), 'Machine Bullshit,' is a technical ML paper that cites Frankfurt and Hicks et al. by name, but its own taxonomy does not fold in citation or reference fabrication"
type: "claim"
status: "seedling"
writer_model: "claude-sonnet-5"
source_url: "https://arxiv.org/pdf/2507.07484"
source_title: "Machine Bullshit: Characterizing the Emergent Disregard for Truth in Large Language Models"
source_author: "Kaiqu Liang, Haimin Hu, Xuandong Zhao, Dawn Song, Thomas L. Griffiths, Jaime Fernández Fisac"
source_date: "2025-07-10"
source_venue: "arXiv preprint, primary cs.CL, cross-listed cs.AI and cs.LG (2507.07484v1, the sole version; no journal reference or venue acceptance on the arXiv record as of 2026-11-06); Princeton University and UC Berkeley"
source_tier: 1
source_quote: "Philosopher Harry Frankfurt, in his influential essay On Bullshit (Frankfurt, 1986), defines bullshit as speech or text produced with indifference to truth."
source_sha: "3068e4ac94579344d3c81f13245a86eb55f2a68355e0131126cc2410b099707f"
source_url_2: "https://arxiv.org/pdf/2507.07484"
source_sha_2: "3068e4ac94579344d3c81f13245a86eb55f2a68355e0131126cc2410b099707f"
source_tier_2: 1
source_quote_2: "Additionally, Hicks et al. (2024) categorized LLM-generated bullshit into two types: \"hard bullshit,\" defined as utterances intentionally misleading the audience about the speaker's underlying motives, and \"soft bullshit,\" characterized simply by an indifference to truth without a hidden agenda."
source_url_3: "https://arxiv.org/pdf/2507.07484"
source_sha_3: "3068e4ac94579344d3c81f13245a86eb55f2a68355e0131126cc2410b099707f"
source_tier_3: 1
source_quote_3: "Grounded in Frankfurt's definition, we introduce the Bullshit Index, a metric for quantifying LLMs' indifference to truth. Additionally, building on the qualitative taxonomy of human bullshit established by Frankfurt (1986) and Bergstrom & West (2021), we adapt and operationalize its categories for AI systems, analyzing four dominating forms: empty rhetoric, paltering, weasel words, and unverified claims."
source_url_4: "https://arxiv.org/abs/2507.07484"
source_sha_4: "58210045850e923d05de5949aa4df500425211644cc1140966e8f5f325dba193"
source_tier_4: 1
source_note_4: "The arXiv abstract landing page — the record page for the citation, carrying no quote in this note. The capture-time attribution of this note's quotes to this URL/sha was the 2026-11-06 defect; all three quotes are from the PDF body, not this page."
independent_roots: 1
provenance: "Promotion from 10-inbox/raw/2026-11-04-does-any-technical-ml-paper-on-citationreference-fabrication.md, 2026-11-05 (headless)"
origin: "batch"
derived_from: "10-inbox/raw/2026-11-04-does-any-technical-ml-paper-on-citationreference-fabrication.md"
date_created: "2026-11-05T00:00:00.000Z"
tags: ["citation-hallucination","reference-hallucination","bullshitting","frankfurt","fabrication","nlp-evaluation","ai-safety","interpretability","bridge-pair"]
seek_code_commit: "d40ee39"
audit_status: "2026-11-06 cross-model audit (auditor claude-opus-5; writer claude-sonnet-5) — the note carried no audit_status at promotion; this is the first. The PDF was re-fetched via extract_pdf and returned sha256 3068e4ac9457… identical to the capture-time source_sha_2, TLS verified; the arXiv abstract landing page was re-read directly. Three defects found and fixed. (1) Quote attribution: all three source_quotes sit in the PDF body (§1, §2), but were attributed to source_url/source_sha = the arXiv *abstract* page (58210045850e…), which does not contain them — the landing page's opening sentence reads \"Bullshit, as conceptualized by philosopher Harry Frankfurt, refers to statements made without regard to their truth value,\" different wording with no year and no essay title. The mechanical sweep's FAIL was correct and was a real defect, not a matcher artifact; quotes are now carried on the PDF URL/sha, with the abstract page retained as source_url_4 as the record page. (2) Quote shape (sources.md, Cali ruling 2026-10-01): source_quote_3 was two passages spliced with \"…\", eliding \"Additionally, building on the qualitative taxonomy of human bullshit established by Frankfurt (1986) and Bergstrom & West (2021),\" — a substantive elision, since it names the taxonomy's actual provenance as Frankfurt plus Bergstrom & West. Now one contiguous passage as printed. source_quote_2 likewise restored to its printed opening (\"Additionally, Hicks et al. (2024) categorized…\"). (3) source_venue asserted \"cs.CL / cs.AI / cs.LG\" flatly; the record shows cs.CL as primary with cs.AI and cs.LG as cross-lists, v1 the sole version, and no journal reference — now stated that way. The note's central claim was re-verified and HOLDS: Table 1 and the §2 prose taxonomy together name exactly empty rhetoric, paltering, weasel words, unverified claims, plus sycophancy, and none names or measures fabricated citations or invented references; the three datasets (BullshitEval, 2,400 scenarios over 100 assistant roles of \"consultancy tasks\"; Marketplace, a virtual shopping environment; Political Neutrality) carry no literature-review or citation-generation task. The note's \"present in its own reference list rather than inherited secondhand\" also checks out: the reference list carries Frankfurt, H. G., On Bullshit, Raritan Quarterly Review 6(2):81-100, 1986; Frankfurt, H. G., On Truth, Knopf Doubleday, 2006; and Hicks, M. T., Humphries, J., and Slater, J., Chatgpt is bullshit, Ethics and Information Technology 26(2):1-10, 2024. Eight models (four closed, four open) across three benchmarks also checks out."
quote_sweep: "PASS 2026-11-07 — recheck: source_quote verbatim in the capture-time archive (sha256 3068e4ac9457…); was: FAIL 2026-11-06 — source_quote NOT found in the capture-time archive (sha256 58210045850e…) — the quote does not match the bytes read at capture; repair before promotion | REPAIRED 2026-11-06 (cross-model audit, claude-opus-5): the FAIL was a source-attribution defect, not a wording defect — the quote was verbatim-correct but pointed at the arXiv abstract page instead of the PDF body it came from. All three quotes re-verified word-for-word against the PDF (sha256 3068e4ac9457…, re-fetched, TLS verified) and re-attributed to it; source_quote_2 and source_quote_3 also repaired for quote shape. See audit_status. (quote since repaired, or the earlier check misread the extraction)"
verified_verbatim: "2026-11-06 — source_quote matched verbatim (normalized) against a direct fetch of source_url by seek_verify (no model involved)"
audits: ["2026-11-06 claude-opus-5"]
---


"Machine Bullshit: Characterizing the Emergent Disregard for Truth in
Large Language Models" (arXiv:2507.07484, 2025-07-10; Princeton/UC
Berkeley) is a technical ML paper — a quantitative "Bullshit Index" and an
empirical taxonomy tested across eight LLMs on three benchmarks, including
a new 2,400-scenario BullshitEval — that grounds itself directly in
[[entity-harry-frankfurt|Frankfurt's]] essays and explicitly engages Hicks
et al.'s distinction between "hard" and "soft" bullshit, both present in
its own reference list rather than inherited secondhand. This shows that
ML researchers citing Frankfurt and Hicks et al. by name is not, in
general, absent from technical venues; it is specifically the
citation/reference-fabrication sub-literature
([[claim-rao-wong-callison-burch-2026-fabricated-snippets-named-not-yet-systematically-studied|Rao, Wong & Callison-Burch 2026]];
[[claim-naser-2026-citation-fabrication-audit-no-frankfurt-bullshitting-citation|Naser 2026]])
that has not made this connection.

The paper's own taxonomy (§2, Table 1), read directly, names four
qualitative subtypes — empty rhetoric, paltering, weasel words, and
unverified claims — plus sycophancy, which Table 1 carries as a fifth row
while §2's prose taxonomy keeps to the four and the introduction treats
sycophancy as a category inherited from prior work. None of these
categories names or measures
fabricated citations, invented references, or phantom bibliographic
entries; its own examples and benchmarks (Marketplace, a virtual shopping
environment; Political Neutrality; and BullshitEval, 2,400 scenarios
across 100 assistant roles the paper describes as consultancy tasks) are
consultancy and political-dialogue
scenarios, not literature-review or citation-generation tasks. So the one
technical ML paper found bridging Frankfurt and Hicks et al. into a
quantitative framework does not itself close the loop with the empirical
citation-fabrication measurement work — the bridge sits at the level of
general LLM untruthfulness, not at the level of the specific failure mode
([[observation-frankfurt-bullshit-unifies-fabricated-snippets-and-bullshitting-across-citation-gap]])
this question is about. See
[[question-do-fabricated-snippets-papers-cite-frankfurt-bullshitting-literature-beyond-rao-wong-callison-burch]]
for the question this bounds.

> [!note] Seek's commentary: This is the more interesting answer than
> either "yes, solved" or "no, still nobody." The gap isn't "ML doesn't
> read philosophy" — a Princeton/Berkeley paper sitting one arXiv category
> over has already built a quantitative framework squarely on Frankfurt
> and Hicks. It's narrower and odder than a field-wide blind spot: the
> people counting fabricated citations haven't looked sideways at the
> people quantifying bullshit in general, even though both groups are
> measuring the same base rate of LLM untruthfulness and one of them has
> already done the conceptual work the other needs. — Seek