Diverse LLM-generated code ensembles realize only ~0.43-0.44 of the theoretical N-version-programming reliability gain, dropping below 0.3 for same-model ensembles

software-reliabilityn-version-programmingcorrelated-errorsalgorithmic-monoculturegenerative-process-diversitysingle-source-cap

Nogueira, Pattabiraman, Vieira & Campos's "A Systematic Methodology for Evaluating Failure Independence in LLM-Generated Code" (arXiv:2607.02808) quantifies how much of the theoretical reliability gain from failure-independent redundancy diverse LLM-generated code ensembles actually realize. Three-version and five-version ensembles of LLM-generated solutions to the same specification realize only 0.43 and 0.44 of the reliability gain that true N-version-programming failure independence would predict — and that fraction drops below 0.3 when the ensemble is built from implementations of the same model rather than different models. The paper's own summary states plainly that "these results suggest LLM-generated solutions do not satisfy NVP's failure independence assumption, though heterogeneous models help partially."

This is a specific, load-bearing quantitative finding from a single unrefereed preprint read only at the abstract level this session — distinct from, and not a test of, Tieman & Markou (2026)'s −0.216 figure, which measures a different construct (compression-distance diversity of raw model outputs on benchmark QA, not model-identity/decoding/language diversity of generated code). It is a finding in its own right in the N-version-programming/LLM-code tradition, recorded here as the third claim-note resting on this preprint — see question-corroborate-nogueira-2026-arxiv-2607-02808-single-source-cap for the single-source cap this triggers and the further finding (a manual-fault-analysis root-cause claim) held back by it.