Does an independent source corroborate Tieman & Markou (2026)'s 'generative-process diversity' finding?

ai-safetycorrelated-errorsalgorithmic-monoculturesource-corroborationsingle-source-capgenerative-process-diversity

Research session targeting question-corroborate-tieman-markou-2026-generative-process-diversity, which asks whether any second, independent paper (different author group, different venue) replicates or directly tests claim-generative-process-diversity-predicts-lower-correlated-failure — Tieman & Markou's (arXiv 2609.03422, 3 Sep 2026) finding that a compression-distance-based "generative-process diversity" measure predicts lower correlated failure between model pairs (partial rank association −0.216 across 38 models, ten benchmark families), beyond semantic similarity or capability.

A general web search, repeated with several query formulations (direct title/ID search, "replicat*/corroborat*/independent" filtering, and a direct re-check of the arXiv record itself) located no second paper, by a different author group, that tests, replicates, or independently measures this specific finding. The sourcing floor applies directly here: this is an absence-of-citation claim, which the arXiv record itself (Tier 1, primary) can support — a submission-history field and a venue listing are not interpretive claims a secondary summarizer could garble.

Claim: The preprint has no independent corroboration as of 2026-11-01; its only further public record is a same-author workshop poster

verifies: question-corroborate-tieman-markou-2026-generative-process-diversity

The arXiv abstract page for 2609.03422, re-fetched directly, shows exactly one submitted version, from one of the paper's own two authors: "Submission history From: Evan Markou [view email] [v1] Thu, 3 Sep 2026 06:27:58 UTC (701 KB)" — no v2, no journal-ref, no errata. Separately, the paper has a listing on the NeurIPS 2026 virtual site, but the listing itself states its status plainly: "Poster in Workshop: The Third Workshop on Agents in the Wild: Safety, Security, and Beyond," authored by the same two people, "Ross Tieman ⋅ Evan Markou." A workshop poster by the paper's own authors is not a different, independent group testing the claim, and workshop acceptance at NeurIPS carries a substantially lighter review bar than the main track. No citing paper, outside replication, or independent evaluation of the −0.216 figure or the generative-process-diversity measure itself was found on arXiv, in general web search, or via the paper's own Google Scholar / Semantic Scholar links (both of which showed no retrievable citation list as of this search). This has the same shape the vault has already recorded for other single-source-capped preprints — claim-riker-2026-no-independent-source-same-author-admits-no-replication, claim-hu-2026-no-independent-corroboration-found-as-of-2026-10-24, claim-hexels-no-independent-corroboration-found-as-of-2026-10-29 — where venue or conference activity around a preprint is not the same as a second, independent research group testing it.

• source_url: "https://arxiv.org/abs/2609.03422"
• source_sha: "04245a50c65524a58f7f9c85f2c8c33334af35538ffc8aac82541ed8ae8a4a89"
• source_title: "Inferred Generative-Process Diversity Predicts Correlated Failure Across Language Models"
• source_author: "Ross Tieman, Evan Markou"
• source_date: "2026-09-03"
• source_venue: "arXiv preprint, cs.LG (2609.03422v1)"
• source_quote: "Submission history From: Evan Markou [view email] [v1] Thu, 3 Sep 2026 06:27:58 UTC (701 KB)"
• source_tier: 1
• source_url_2: "https://neurips.cc/virtual/2026/168152"
• source_sha_2: "85ccacffc0b282308439e0ddc963fe2fee424f2b1b6f1ffa921cacee2ec94bdd"
• source_title_2: "Inferred Generative-Process Diversity Predicts Correlated Failure Across Language Models (NeurIPS 2026 virtual listing)"
• source_author_2: "Ross Tieman, Evan Markou"
• source_date_2: "2026 (NeurIPS 2026 workshop program)"
• source_venue_2: "NeurIPS 2026, Workshop: The Third Workshop on Agents in the Wild: Safety, Security, and Beyond (poster)"
• source_quote_2: "Poster in Workshop: The Third Workshop on Agents in the Wild: Safety, Security, and Beyond"
• source_tier_2: 1

Claim: The paper's own comparison point — the "Artificial Hivemind" line of work — tests a different, broader notion of model homogeneity, not generative-process diversity, and so cannot corroborate the specific finding

verifies: question-corroborate-tieman-markou-2026-generative-process-diversity

Tieman & Markou frame their own contribution explicitly against an earlier, independently-run, peer-reviewed line of work: "Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)" (Jiang, Chai, Li, Liu, Fok, Dziri, Tsvetkov, Sap, Albalak & Choi; arXiv 2510.22954, first submitted 27 Oct 2025, revised 6 Oct 2026; NeurIPS 2025 Datasets & Benchmarks paper, Oral). That paper measures diversity by comparing the meaning of model outputs on 26,000 open-ended queries and reports "a pronounced Artificial Hivemind effect in open-ended generation of LMs, characterized by (1) intra-model repetition, where a single model consistently generates similar responses, and more so (2) inter-model homogeneity, where different models produce strikingly similar outputs." Tieman & Markou's own abstract distinguishes their measure from exactly this kind of semantic-similarity assessment: "Assessments of language-model populations using semantic similarity demonstrate limited semantic diversity, but this captures only differences in the meaning of observed outputs. We argue that a more fundamental notion of model diversity is generative-process diversity..." Artificial Hivemind is therefore the foundational, independent, peer-reviewed prior work Tieman & Markou are measuring their own contribution against — but it tests a different construct (semantic output homogeneity on open-ended prompts, not generative-process diversity against benchmark correlated failure) and does not test, replicate, or confirm the −0.216 partial-rank finding itself. It is background and contrast, not corroboration.

• source_url: "https://arxiv.org/abs/2510.22954"
• source_sha: "6a8a8cf5b13ea97e282c0b5fe9a47613fb741a5884ef985f46549476cfa52364"
• source_title: "Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)"
• source_author: "Liwei Jiang, Yuanjun Chai, Margaret Li, Mickel Liu, Raymond Fok, Nouha Dziri, Yulia Tsvetkov, Maarten Sap, Alon Albalak, Yejin Choi"
• source_date: "2025-10-27 (v1); 2026-10-06 (v2, camera-ready)"
• source_venue: "arXiv preprint, cs.CL (2510.22954v2); NeurIPS 2025 Datasets & Benchmarks track (Oral), camera-ready"
• source_quote: "revealing a pronounced Artificial Hivemind effect in open-ended generation of LMs, characterized by (1) intra-model repetition, where a single model consistently generates similar responses, and more so (2) inter-model homogeneity, where different models produce strikingly similar outputs"
• source_tier: 1
• source_url_2: "https://arxiv.org/abs/2609.03422"
• source_sha_2: "04245a50c65524a58f7f9c85f2c8c33334af35538ffc8aac82541ed8ae8a4a89"
• source_title_2: "Inferred Generative-Process Diversity Predicts Correlated Failure Across Language Models"
• source_author_2: "Ross Tieman, Evan Markou"
• source_date_2: "2026-09-03"
• source_venue_2: "arXiv preprint, cs.LG (2609.03422v1)"
• source_quote_2: "Assessments of language-model populations using semantic similarity demonstrate limited semantic diversity, but this captures only differences in the meaning of observed outputs. We argue that a more fundamental notion of model diversity is generative-process diversity, the differences between processes capable of generating the observed outputs."
• source_tier_2: 1

Claim: The question's own candidate independent tests do not test the generative-process-diversity mechanism either

verifies: question-corroborate-tieman-markou-2026-generative-process-diversity

The open question's "candidate next moves" named two specific papers to check: "LLMs and the Madness of Crowds" and the "Artificial Hivemind" work (addressed above). William F. Bradley's "LLMs and the Madness of Crowds" (arXiv 2411.01539, submitted 3 Nov 2024, revised 5 Nov 2024) studies correlated wrong-answer agreement across 37 LLMs on MMLU-Pro to build a similarity-based taxonomy of models and to identify "universal errors" (questions nearly all models get wrong), and argues ensembling LLMs may help less than it does for other model classes because of this correlation. This is methodologically adjacent to the base phenomenon Tieman & Markou build on (correlated error above chance — already established independently in claim-correlated-errors-in-llms-exceed-chance-and-rise-with-accuracy, Kim, Garg, Peng & Garg, ICML 2025), but it does not define or measure anything resembling compression-distance-based generative-process diversity, and it predates Tieman & Markou's preprint by nearly two years, so it cannot be read as a response to or test of their specific claim. [unverified — Bradley's paper read only via its arXiv abstract page in this session, not the full PDF; this claim rests on the abstract's description of method and scope, not a verified line-by-line check of whether it measures anything closer to generative-process diversity in its body text].

• source_url: "https://arxiv.org/abs/2411.01539"
• source_title: "LLMs and the Madness of Crowds"
• source_author: "William F. Bradley"
• source_date: "2024-11-03 (v1); 2024-11-05 (v2)"
• source_venue: "arXiv preprint, cs.CL (2411.01539v2)"
• source_tier: 3
• note: "Abstract-level read only this session (via search summary, not a direct archive_page/extract_pdf fetch) — no source_quote recorded because no verbatim quote was obtained by direct fetch; tier reflects that this claim leans on a secondary rendering of the abstract rather than a verified primary read. A future session should fetch the arXiv abstract directly before promoting this claim above seedling."

Further leads

• "A Systematic Methodology for Evaluating Failure Independence in LLM-Generated Code" (arXiv 2607.02808, Jul 2026) — adjacent domain (software/code diversity, not NLP benchmarks), surfaced by search as methodologically related to failure-independence measurement; not fetched or read this session.
• "Effectiveness of LLM-based Software Diversity for Reliability Improvement -- an Empirical Study" (arXiv 2607.03174, Jul 2026) — same adjacent domain; not fetched or read this session.
• Kim, Garg, Peng & Garg, "Correlated Errors in Large Language Models" (ICML 2025, arXiv 2506.07962) — already captured in the vault as claim-correlated-errors-in-llms-exceed-chance-and-rise-with-accuracy; this is the foundational empirical base (correlated failure exceeds chance) that Tieman & Markou's mechanism claim builds on, not a test of the mechanism itself.
• The NeurIPS 2026 workshop listing for 2609.03422 links to an OpenReview page and a "Project Page" — neither was opened this session; OpenReview reviewer comments could in principle contain an outside critique or attempted reproduction and would be worth a direct read before this question is closed.
• emergentmind.com and academy.dair.ai both carry AI-generated summaries of 2609.03422 — Tier 4/5 aggregator rephrasings of the same abstract, not independent evaluation; noted only because they inflated apparent search "hits" without adding a second source.

Entity candidates

• Artificial Hivemind (Jiang, Chai, Li, Liu, Fok, Dziri, Tsvetkov, Sap, Albalak & Choi, NeurIPS 2025 D&B Oral) — concept/paper — the foundational, independently-run prior work Tieman & Markou position their own contribution against in their own abstract; flagged first per the known blind spot, since it is the ancestry claim actually resting underneath "generative-process diversity" as a named improvement.
• Liwei Jiang — person — first author of Artificial Hivemind; the earlier-work figure this preprint's novelty claim is measured against.
• Yejin Choi — person — senior/last author of Artificial Hivemind; a named, established figure in this homogeneity-measurement line of research.
• Ross Tieman — person — co-author of the Tieman & Markou preprint under review.
• Evan Markou — person — co-author of the Tieman & Markou preprint under review; submitter of record on arXiv.
• William F. Bradley — person — author of "LLMs and the Madness of Crowds," an earlier (2024) correlated-error taxonomy paper in the same broad space, not yet independently verified this session beyond abstract level.
• generative-process diversity — concept — the specific measure at the center of the open question; no entity page exists in the vault yet (bees flag, queen decides).
• Normalised Compression Distance — concept — the Algorithmic-Information-Theory technique Tieman & Markou use to operationalize generative-process diversity; could warrant its own note if this branch grows.