---
id: "20261101-0229-does-an-independent-source"
title: "Does an independent source corroborate Tieman & Markou (2026)'s 'generative-process diversity' finding?"
type: "capture"
status: "promoted"
origin: "batch"
writer_model: "claude-sonnet-5"
date_created: "2026-11-01T00:00:00.000Z"
date_promoted: "2026-11-02T00:00:00.000Z"
provenance: "Batch research run, 2026-11-01, targeting question-corroborate-tieman-markou-2026-generative-process-diversity"
derived_from: []
verifies: "question-corroborate-tieman-markou-2026-generative-process-diversity"
tags: ["ai-safety","correlated-errors","algorithmic-monoculture","source-corroboration","single-source-cap","generative-process-diversity"]
seek_code_commit: "d40ee39"
promoted_to: ["30-notes/claim-tieman-markou-2026-no-independent-corroboration-as-of-2026-11-01.md (merges the capture's three claims — arXiv/NeurIPS record check, Artificial Hivemind contrast, Bradley candidate ruled out — into one atomic finding, per the 'atomic is not many' ruling and the established shape of claim-hu-2026/claim-riker-2026/claim-hexels)","40-entities/entity-artificial-hivemind.md (new hub — established, peer-reviewed, NeurIPS 2025 Oral, foundational contrast work)","40-entities/entity-generative-process-diversity.md (existing watching stub — appended a dated line, not recreated)","50-questions/question-corroborate-tieman-markou-2026-generative-process-diversity.md (left open, progress line appended — not forced to answered; capture's own 'further leads' flagged the unopened OpenReview page as worth checking first)"]
not_promoted: ["Further-leads items 'A Systematic Methodology for Evaluating Failure Independence in LLM-Generated Code' (arXiv 2607.02808) and 'Effectiveness of LLM-based Software Diversity...' (arXiv 2607.03174) — adjacent-domain leads, not fetched or read this session; not claims yet.","Kim, Garg, Peng & Garg (ICML 2025) reference — already a vault claim-note ([[claim-correlated-errors-in-llms-exceed-chance-and-rise-with-accuracy]]); cited inline in the new note, not re-promoted.","emergentmind.com / academy.dair.ai AI-summary pages — Tier 4/5 aggregator rephrasings of the same abstract; noted as search noise, not a source.","Entity candidates Ross Tieman, Evan Markou, Liwei Jiang, William F. Bradley — co-authorship on one paper (or authorship of one adjacent paper) is a mention, not a hub, per the entity-page spec's person test; left as mentions in the new claim-note and entity-artificial-hivemind.md, no stub pages created.","Entity candidate Yejin Choi — senior author of Artificial Hivemind, but this capture surfaces nothing about her beyond that co-authorship; per the 2026-10-01 ruling ('co-authorship alone is a mention, not a hub') she stays a mention on entity-artificial-hivemind.md rather than getting her own page.","Entity candidate Normalised Compression Distance — real, established AIT technique, but appears in the vault only as this one preprint's methodology detail; does not yet recur or carry independent load-bearing weight, so left unpromoted per the entity spec's 'when unsure, don't promote.'"]
---


Research session targeting
[[question-corroborate-tieman-markou-2026-generative-process-diversity]],
which asks whether any second, independent paper (different author group,
different venue) replicates or directly tests
[[claim-generative-process-diversity-predicts-lower-correlated-failure]] —
Tieman & Markou's (arXiv 2609.03422, 3 Sep 2026) finding that a
compression-distance-based "generative-process diversity" measure predicts
lower correlated failure between model pairs (partial rank association
−0.216 across 38 models, ten benchmark families), beyond semantic
similarity or capability.

A general web search, repeated with several query formulations (direct
title/ID search, "replicat*/corroborat*/independent" filtering, and a
direct re-check of the arXiv record itself) located no second paper, by a
different author group, that tests, replicates, or independently measures
this specific finding. The sourcing floor applies directly here: this is
an absence-of-citation claim, which the arXiv record itself (Tier 1,
primary) can support — a submission-history field and a venue listing are
not interpretive claims a secondary summarizer could garble.

## Claim: The preprint has no independent corroboration as of 2026-11-01; its only further public record is a same-author workshop poster

verifies: question-corroborate-tieman-markou-2026-generative-process-diversity

The arXiv abstract page for 2609.03422, re-fetched directly, shows exactly
one submitted version, from one of the paper's own two authors: "Submission
history From: Evan Markou [view email] [v1] Thu, 3 Sep 2026 06:27:58 UTC
(701 KB)" — no v2, no journal-ref, no errata. Separately, the paper has a
listing on the NeurIPS 2026 virtual site, but the listing itself states its
status plainly: "Poster in Workshop: The Third Workshop on Agents in the
Wild: Safety, Security, and Beyond," authored by the same two people,
"Ross Tieman ⋅ Evan Markou." A workshop poster by the paper's own authors
is not a different, independent group testing the claim, and workshop
acceptance at NeurIPS carries a substantially lighter review bar than the
main track. No citing paper, outside replication, or independent
evaluation of the −0.216 figure or the generative-process-diversity
measure itself was found on arXiv, in general web search, or via the
paper's own Google Scholar / Semantic Scholar links (both of which showed
no retrievable citation list as of this search). This has the same shape
the vault has already recorded for other single-source-capped preprints —
[[claim-riker-2026-no-independent-source-same-author-admits-no-replication]],
[[claim-hu-2026-no-independent-corroboration-found-as-of-2026-10-24]],
[[claim-hexels-no-independent-corroboration-found-as-of-2026-10-29]] — where
venue or conference activity around a preprint is not the same as a second,
independent research group testing it.

- source_url: "https://arxiv.org/abs/2609.03422"
- source_sha: "04245a50c65524a58f7f9c85f2c8c33334af35538ffc8aac82541ed8ae8a4a89"
- source_title: "Inferred Generative-Process Diversity Predicts Correlated Failure Across Language Models"
- source_author: "Ross Tieman, Evan Markou"
- source_date: "2026-09-03"
- source_venue: "arXiv preprint, cs.LG (2609.03422v1)"
- source_quote: "Submission history From: Evan Markou [view email] [v1] Thu, 3 Sep 2026 06:27:58 UTC (701 KB)"
- source_tier: 1
- source_url_2: "https://neurips.cc/virtual/2026/168152"
- source_sha_2: "85ccacffc0b282308439e0ddc963fe2fee424f2b1b6f1ffa921cacee2ec94bdd"
- source_title_2: "Inferred Generative-Process Diversity Predicts Correlated Failure Across Language Models (NeurIPS 2026 virtual listing)"
- source_author_2: "Ross Tieman, Evan Markou"
- source_date_2: "2026 (NeurIPS 2026 workshop program)"
- source_venue_2: "NeurIPS 2026, Workshop: The Third Workshop on Agents in the Wild: Safety, Security, and Beyond (poster)"
- source_quote_2: "Poster in Workshop: The Third Workshop on Agents in the Wild: Safety, Security, and Beyond"
- source_tier_2: 1

## Claim: The paper's own comparison point — the "Artificial Hivemind" line of work — tests a different, broader notion of model homogeneity, not generative-process diversity, and so cannot corroborate the specific finding

verifies: question-corroborate-tieman-markou-2026-generative-process-diversity

Tieman & Markou frame their own contribution explicitly against an earlier,
independently-run, peer-reviewed line of work: "Artificial Hivemind: The
Open-Ended Homogeneity of Language Models (and Beyond)" (Jiang, Chai, Li,
Liu, Fok, Dziri, Tsvetkov, Sap, Albalak & Choi; arXiv 2510.22954, first
submitted 27 Oct 2025, revised 6 Oct 2026; NeurIPS 2025 Datasets & Benchmarks
paper, Oral). That paper measures diversity by comparing the *meaning* of
model outputs on 26,000 open-ended queries and reports "a pronounced
Artificial Hivemind effect in open-ended generation of LMs, characterized
by (1) intra-model repetition, where a single model consistently generates
similar responses, and more so (2) inter-model homogeneity, where different
models produce strikingly similar outputs." Tieman & Markou's own abstract
distinguishes their measure from exactly this kind of semantic-similarity
assessment: "Assessments of language-model populations using semantic
similarity demonstrate limited semantic diversity, but this captures only
differences in the meaning of observed outputs. We argue that a more
fundamental notion of model diversity is generative-process diversity..."
Artificial Hivemind is therefore the foundational, independent, peer-reviewed
prior work Tieman & Markou are measuring their own contribution against —
but it tests a different construct (semantic output homogeneity on
open-ended prompts, not generative-process diversity against benchmark
correlated failure) and does not test, replicate, or confirm the
−0.216 partial-rank finding itself. It is background and contrast, not
corroboration.

- source_url: "https://arxiv.org/abs/2510.22954"
- source_sha: "6a8a8cf5b13ea97e282c0b5fe9a47613fb741a5884ef985f46549476cfa52364"
- source_title: "Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)"
- source_author: "Liwei Jiang, Yuanjun Chai, Margaret Li, Mickel Liu, Raymond Fok, Nouha Dziri, Yulia Tsvetkov, Maarten Sap, Alon Albalak, Yejin Choi"
- source_date: "2025-10-27 (v1); 2026-10-06 (v2, camera-ready)"
- source_venue: "arXiv preprint, cs.CL (2510.22954v2); NeurIPS 2025 Datasets & Benchmarks track (Oral), camera-ready"
- source_quote: "revealing a pronounced Artificial Hivemind effect in open-ended generation of LMs, characterized by (1) intra-model repetition, where a single model consistently generates similar responses, and more so (2) inter-model homogeneity, where different models produce strikingly similar outputs"
- source_tier: 1
- source_url_2: "https://arxiv.org/abs/2609.03422"
- source_sha_2: "04245a50c65524a58f7f9c85f2c8c33334af35538ffc8aac82541ed8ae8a4a89"
- source_title_2: "Inferred Generative-Process Diversity Predicts Correlated Failure Across Language Models"
- source_author_2: "Ross Tieman, Evan Markou"
- source_date_2: "2026-09-03"
- source_venue_2: "arXiv preprint, cs.LG (2609.03422v1)"
- source_quote_2: "Assessments of language-model populations using semantic similarity demonstrate limited semantic diversity, but this captures only differences in the meaning of observed outputs. We argue that a more fundamental notion of model diversity is generative-process diversity, the differences between processes capable of generating the observed outputs."
- source_tier_2: 1

## Claim: The question's own candidate independent tests do not test the generative-process-diversity mechanism either

verifies: question-corroborate-tieman-markou-2026-generative-process-diversity

The open question's "candidate next moves" named two specific papers to
check: "LLMs and the Madness of Crowds" and the "Artificial Hivemind" work
(addressed above). William F. Bradley's "LLMs and the Madness of Crowds"
(arXiv 2411.01539, submitted 3 Nov 2024, revised 5 Nov 2024) studies
correlated wrong-answer agreement across 37 LLMs on MMLU-Pro to build a
similarity-based taxonomy of models and to identify "universal errors"
(questions nearly all models get wrong), and argues ensembling LLMs may
help less than it does for other model classes because of this
correlation. This is methodologically adjacent to the base phenomenon
Tieman & Markou build on (correlated error above chance — already
established independently in [[claim-correlated-errors-in-llms-exceed-chance-and-rise-with-accuracy]],
Kim, Garg, Peng & Garg, ICML 2025), but it does not define or measure
anything resembling compression-distance-based generative-process
diversity, and it predates Tieman & Markou's preprint by nearly two years,
so it cannot be read as a response to or test of their specific claim.
[unverified — Bradley's paper read only via its arXiv abstract page in this
session, not the full PDF; this claim rests on the abstract's description
of method and scope, not a verified line-by-line check of whether it
measures anything closer to generative-process diversity in its body text].

- source_url: "https://arxiv.org/abs/2411.01539"
- source_title: "LLMs and the Madness of Crowds"
- source_author: "William F. Bradley"
- source_date: "2024-11-03 (v1); 2024-11-05 (v2)"
- source_venue: "arXiv preprint, cs.CL (2411.01539v2)"
- source_tier: 3
- note: "Abstract-level read only this session (via search summary, not a direct archive_page/extract_pdf fetch) — no source_quote recorded because no verbatim quote was obtained by direct fetch; tier reflects that this claim leans on a secondary rendering of the abstract rather than a verified primary read. A future session should fetch the arXiv abstract directly before promoting this claim above seedling."

## Further leads

- "A Systematic Methodology for Evaluating Failure Independence in LLM-Generated Code" (arXiv 2607.02808, Jul 2026) — adjacent domain (software/code diversity, not NLP benchmarks), surfaced by search as methodologically related to failure-independence measurement; not fetched or read this session.
- "Effectiveness of LLM-based Software Diversity for Reliability Improvement -- an Empirical Study" (arXiv 2607.03174, Jul 2026) — same adjacent domain; not fetched or read this session.
- Kim, Garg, Peng & Garg, "Correlated Errors in Large Language Models" (ICML 2025, arXiv 2506.07962) — already captured in the vault as [[claim-correlated-errors-in-llms-exceed-chance-and-rise-with-accuracy]]; this is the foundational empirical base (correlated failure exceeds chance) that Tieman & Markou's mechanism claim builds on, not a test of the mechanism itself.
- The NeurIPS 2026 workshop listing for 2609.03422 links to an OpenReview page and a "Project Page" — neither was opened this session; OpenReview reviewer comments could in principle contain an outside critique or attempted reproduction and would be worth a direct read before this question is closed.
- emergentmind.com and academy.dair.ai both carry AI-generated summaries of 2609.03422 — Tier 4/5 aggregator rephrasings of the same abstract, not independent evaluation; noted only because they inflated apparent search "hits" without adding a second source.

## Entity candidates

- Artificial Hivemind (Jiang, Chai, Li, Liu, Fok, Dziri, Tsvetkov, Sap, Albalak & Choi, NeurIPS 2025 D&B Oral) — concept/paper — the foundational, independently-run prior work Tieman & Markou position their own contribution against in their own abstract; flagged first per the known blind spot, since it is the ancestry claim actually resting underneath "generative-process diversity" as a named improvement.
- Liwei Jiang — person — first author of Artificial Hivemind; the earlier-work figure this preprint's novelty claim is measured against.
- Yejin Choi — person — senior/last author of Artificial Hivemind; a named, established figure in this homogeneity-measurement line of research.
- Ross Tieman — person — co-author of the Tieman & Markou preprint under review.
- Evan Markou — person — co-author of the Tieman & Markou preprint under review; submitter of record on arXiv.
- William F. Bradley — person — author of "LLMs and the Madness of Crowds," an earlier (2024) correlated-error taxonomy paper in the same broad space, not yet independently verified this session beyond abstract level.
- generative-process diversity — concept — the specific measure at the center of the open question; no entity page exists in the vault yet (bees flag, queen decides).
- Normalised Compression Distance — concept — the Algorithmic-Information-Theory technique Tieman & Markou use to operationalize generative-process diversity; could warrant its own note if this branch grows.