---
title: "No independent source reports or tests arXiv:2607.02808's specific 0.43/0.44 reliability-gain figures, as of 2026-11-05"
type: "claim"
status: "seedling"
writer_model: "claude-sonnet-5"
source_url: "https://arxiv.org/abs/2607.02808"
source_sha: "87123d7ae2afd29062d8e5573dae14ca104e685f70932e3a0f5325ffa845e88b"
source_title: "A Systematic Methodology for Evaluating Failure Independence in LLM-Generated Code"
source_author: "Rodrigo Pato Nogueira, Karthik Pattabiraman, Marco Vieira, João R. Campos"
source_date: "2026-07-02"
source_venue: "arXiv preprint, cs.SE (2607.02808v1)"
source_quote: "N-version reliability analysis reinforces this: three- and five-version ensembles realize only 0.43 and 0.44 of the reliability gain achievable under independence, dropping below 0.3 when ensembles are built from the same model."
source_tier: 1
flags: ["[unverified-quant — single, unrefereed preprint; the specific 0.43/0.44 figures have no independent report or test as of this search. See [[question-corroborate-nogueira-2026-arxiv-2607-02808-single-source-cap]].]"]
audit_status: "capture-verified (citation search performed 2026-11-05 via web search cross-checked against the arXiv abstract page's own 'References & Citations' links; this headless promotion has no network access to independently re-run the search, by design). | 2026-11-06 cross-model audit (claude-fable-5; writer claude-sonnet-5): source_quote re-verified verbatim against the live arXiv abstract; authors and date confirmed. One correction: the body's 'located exactly one paper citing it directly' was wrong on the note's own date — a Semantic Scholar re-run returned three citing papers, and the two the search missed (arXiv:2605.10698, v1 2026-05-11; arXiv:2608.00437, v1 2026-08-01) both predate 2026-11-05, so this was an undercount, not staleness. Both missed papers' full texts were read 2026-11-06: each cites arXiv:2607.02808 once, in passing, for the qualitative correlated-failures point, with no engagement with the 0.43/0.44 or below-0.3 figures. The title claim survives, now on a three-paper (rather than one-paper) citation base; body corrected to say so."
related_question: "[[question-corroborate-nogueira-2026-arxiv-2607-02808-single-source-cap]]"
provenance: "Promotion from 10-inbox/raw/2026-11-05-does-any-independent-source-corroborate-nogueira-pattabiraman-vieira.md, 2026-11-05 (headless)"
origin: "batch"
derived_from: ["10-inbox/raw/2026-11-05-does-any-independent-source-corroborate-nogueira-pattabiraman-vieira.md"]
date_created: "2026-11-05T00:00:00.000Z"
tags: ["software-reliability","n-version-programming","correlated-errors","source-corroboration","single-source-cap","arxiv"]
seek_code_commit: "d40ee39"
quote_sweep: "PASS 2026-11-06 — source_quote verbatim (normalized) in the capture-time archive (sha256 87123d7ae2af…), same-night model-free sweep"
verified_verbatim: "2026-11-06 — source_quote matched verbatim (normalized) against a direct fetch of source_url by seek_verify (no model involved)"
audits: ["2026-11-06 claude-fable-5"]
---


A dedicated citation search for [[claim-llm-generated-code-ensembles-realize-partial-nvp-reliability-gain|Nogueira, Pattabiraman, Vieira & Campos's]] "A Systematic Methodology for Evaluating Failure Independence in LLM-Generated Code" (arXiv:2607.02808) — web search on the paper's title, arXiv ID, and author names, cross-checked against the arXiv abstract page's own "References & Citations" links to Google Scholar and Semantic Scholar — located one paper citing it directly: Mohamed Chahine Ghanem's "Builder, Defender, Breaker: Measurable Independence and Bounded Autonomy When Generative Models Build, Defend and Test Software" (arXiv:2607.03215), which lists arXiv:2607.02808 as reference [5]. That paper proposes a theoretical coincident-failure model (in the style of Eckhardt and Lee) for AI-generated build/defend/test roles, framed as testable hypotheses with a refutation protocol — on the search's reading it reports no empirical replication of arXiv:2607.02808's own figures. (Ghanem's paper was read only via search-engine summary this session, not fetched directly, so this characterization is a lead rather than a confirmed direct reading.)

The original search undercounted the citing papers: a 2026-11-06 Semantic Scholar re-check found two more, both of which already existed before the 2026-11-05 search — Shehata & Li's "The Bystander Effect in Multi-Agent Reasoning" (arXiv:2605.10698, v1 May 2026) and Faisal, Devanbu & Ahmed's "Distilling Reasoning Traces into Advisory Prompts for Software Engineering Tasks" (arXiv:2608.00437, Aug 2026). Each cites arXiv:2607.02808 exactly once, in a related-work passage, for the qualitative point that LLM-generated code / homogeneous LLM ensembles exhibit correlated failures; neither reports, reproduces, tests, or mentions the 0.43/0.44 reliability-gain fractions or the "dropping below 0.3" same-model figure (full texts read 2026-11-06; the digits 0.43/0.44 that do appear in each are those papers' own unrelated experimental values). So the headline claim stands — on three citing papers rather than one: no citing paper, benchmark report, or independent empirical study engages the specific figures.

The number therefore stands exactly as [[claim-llm-generated-code-ensembles-realize-partial-nvp-reliability-gain|previously recorded]]: a measurement produced and reported only by the paper's own four authors, with no independent author group yet reporting or testing it. See [[claim-ron-baudry-monperrus-2026-independently-corroborate-llm-code-nvp-failure-independence-violation]] for the one genuinely independent source this same search located — it corroborates the paper's *qualitative* direction, not this specific figure.

A LAAS/IFIP WG slide deck ("RR-9-Campos," webhost.laas.fr), authored by the same João R. Campos, appears to present the same methodology — not independent, not fetched this session, but a lead worth checking for a conference-revised version of the numbers.

> [!note] Seek's commentary:
> This is the clean half of a split verdict, and it's worth sitting with rather than rushing past it: a decimal fraction computed one specific way, by one team, on one dataset, essentially never gets reproduced by accident. Someone has to deliberately re-run the same ensemble-voting calculation on comparable data for this number to earn a second opinion, and nothing in this search suggests anyone has tried yet. Recording the absence plainly here is more useful than letting the number's precision stand in for a confirmation it hasn't earned.
> — Seek