---
title: "Diverse LLM-generated code ensembles realize only ~0.43-0.44 of the theoretical N-version-programming reliability gain, dropping below 0.3 for same-model ensembles"
type: "claim"
status: "seedling"
writer_model: "claude-sonnet-5"
source_url: "https://arxiv.org/abs/2607.02808"
source_sha: "87123d7ae2afd29062d8e5573dae14ca104e685f70932e3a0f5325ffa845e88b"
source_title: "A Systematic Methodology for Evaluating Failure Independence in LLM-Generated Code"
source_author: "Rodrigo Pato Nogueira, Karthik Pattabiraman, Marco Vieira, João R. Campos"
source_date: "2026-07-02"
source_venue: "arXiv preprint, cs.SE (2607.02808v1)"
source_quote: "three- and five-version ensembles realize only 0.43 and 0.44 of the reliability gain achievable under independence, dropping below 0.3 when ensembles are built from the same model."
source_quote_2: "Overall, these results suggest LLM-generated solutions do not satisfy NVP's failure independence assumption, though heterogeneous models help partially."
source_tier: 1
flags: ["[unverified-quant — single, unrefereed preprint, abstract-level read only, not yet independently corroborated. See [[question-corroborate-nogueira-2026-arxiv-2607-02808-single-source-cap]].]"]
audit_status: "capture-verified (quote and sha recorded at capture time, 2026-11-04). This headless promotion has no network access to independently re-fetch (by design)."
related_question: "[[question-corroborate-nogueira-2026-arxiv-2607-02808-single-source-cap]]"
provenance: "Promotion from 10-inbox/raw/2026-11-04-a-dedicated-corroboration-search-for-tieman-markous-generative.md, 2026-11-04 (headless)"
origin: "batch"
derived_from: ["10-inbox/raw/2026-11-04-a-dedicated-corroboration-search-for-tieman-markous-generative.md"]
date_created: "2026-11-04T00:00:00.000Z"
tags: ["software-reliability","n-version-programming","correlated-errors","algorithmic-monoculture","generative-process-diversity","single-source-cap"]
seek_code_commit: "d40ee39"
quote_sweep: "PASS 2026-11-05 — source_quote verbatim (normalized) in the capture-time archive (sha256 87123d7ae2af…), same-night model-free sweep"
verified_verbatim: "2026-11-05 — source_quote matched verbatim (normalized) against a direct fetch of source_url by seek_verify (no model involved)"
---


Nogueira, Pattabiraman, Vieira & Campos's "A Systematic Methodology for Evaluating Failure Independence in LLM-Generated Code" (arXiv:2607.02808) quantifies how much of the theoretical reliability gain from failure-independent redundancy diverse LLM-generated code ensembles actually realize. Three-version and five-version ensembles of LLM-generated solutions to the same specification realize only 0.43 and 0.44 of the reliability gain that true [[entity-n-version-programming|N-version-programming]] failure independence would predict — and that fraction drops below 0.3 when the ensemble is built from implementations of the same model rather than different models. The paper's own summary states plainly that "these results suggest LLM-generated solutions do not satisfy NVP's failure independence assumption, though heterogeneous models help partially."

This is a specific, load-bearing quantitative finding from a single unrefereed preprint read only at the abstract level this session — distinct from, and not a test of, [[claim-generative-process-diversity-predicts-lower-correlated-failure|Tieman & Markou (2026)]]'s −0.216 figure, which measures a different construct (compression-distance diversity of raw model outputs on benchmark QA, not model-identity/decoding/language diversity of generated code). It is a finding in its own right in the N-version-programming/LLM-code tradition, recorded here as the third claim-note resting on this preprint — see [[question-corroborate-nogueira-2026-arxiv-2607-02808-single-source-cap]] for the single-source cap this triggers and the further finding (a manual-fault-analysis root-cause claim) held back by it.

> [!note] Seek's commentary:
> 0.43 is the number that actually earns this paper a place in the vault on its own terms, independent of whatever it does or doesn't say about Tieman & Markou — it's a direct, quotable measurement of exactly how much of N-version programming's classical promise survives contact with LLM-generated code. Worth remembering on its own.
> — Seek