---
title: "No independent source corroborates RIKER (Roig, 2026) — the only follow-up work is same-author/same-company, and the author's own later paper says independent replication has not happened"
type: "claim"
status: "seedling"
writer_model: "claude-sonnet-5"
source_url: "https://arxiv.org/abs/2603.08274"
source_sha: "5c2a9a9fb29009045be4623d0cb2abff0c316658e6fafb4cde066e0431d3cbe0"
source_author: "JV Roig"
source_date: "2026-03-09"
source_title: "How Much Do LLMs Hallucinate in Document Q&A Scenarios? A 172-Billion-Token Study Across Temperatures, Context Lengths, and Hardware Platforms"
source_venue: "arXiv preprint, cs.CL (2603.08274v1); Kamiwaza AI"
source_quote: "Single evaluation framework. All results are based on the RIKER methodology. While RIKER's ground-truth-first approach avoids many pitfalls of traditional evaluation (see Section 3), our findings have not been independently replicated with alternative frameworks."
source_quote_2: "In prior work, we introduced RIKER (Retrieval Intelligence and Knowledge Extraction Rating) [12], a methodology that addresses these challenges through paradigm inversion: generating documents from known ground truth rather than extracting ground truth from documents."
source_tier: 1
source_url_2: "https://signal65.com/wp-content/uploads/2026/06/Signal65-Insights_Benchmarking-Information-Retrieval-and-LLM-Hallucination-with-RIKER.pdf"
source_sha_2: "934588ec0dadec5b73aaf172c87fbdc38d408a568ed01f441ad52248efa3f818"
source_author_2: "Mitch Lewis (Signal65), in partnership with Kamiwaza"
source_date_2: "2026-06"
source_title_2: "Benchmarking Information Retrieval and LLM Hallucination with RIKER"
source_venue_2: "Signal65 Insights report, published in partnership with Kamiwaza"
source_quote_3: "As part of an ongoing collaboration between Signal65 and Kamiwaza, this paper introduces RIKER (Retrieval Intelligence and Knowledge Extraction Rating), a benchmark designed to evaluate LLM knowledge retrieval capabilities."
source_tier_2: 3
independent_roots: 2
audit_status: "capture-verified (both documents directly fetched at capture time, 2026-10-22, sha receipts above; queen's independent re-check not performed in this headless run — no network access by design). 2026-10-23 cross-model audit (claude-fable-5): both documents re-fetched directly, sha256 of each re-fetch identical to the capture receipts (5c2a9a9f…, 934588ec…); all three quotes located and read verbatim. The 2026-10-23 quote_sweep FAIL is a false negative: source_quote_3 lives in the second source's archive (934588ec…), which a sweep keyed to the first sha cannot see, and source_quote_2 is present verbatim in 5c2a9a9f… (Introduction, 'In prior work, we introduced RIKER…'). Same audit corrected the body's Signal65-roster sentence — see Correction history."
provenance: "Promotion from 10-inbox/raw/2026-10-22-does-an-independent-source-corroborate-rikers-roig-2026.md, 2026-10-22 (headless)"
origin: "batch"
derived_from: "10-inbox/raw/2026-10-22-does-an-independent-source-corroborate-rikers-roig-2026.md"
date_created: "2026-10-22T00:00:00.000Z"
tags: ["rag","benchmark","nlp-evaluation","hallucination","source-corroboration","single-source-cap","riker","independence-test"]
seek_code_commit: "d40ee39"
related_question: "[[question-corroborate-riker-2026-hallucination-rate-range]]"
quote_sweep: "FAIL 2026-10-23 — source_quote_2, source_quote_3 NOT found in the capture-time archive (sha256 5c2a9a9fb290…) — the quote does not match the bytes read at capture; repair before promotion"
verified_verbatim: "2026-10-24 — source_quote matched verbatim (normalized) against a direct fetch of source_url by seek_verify (no model involved)"
---


A search for independent citation, reproduction, or engagement with
[[claim-riker-2026-grounding-accuracy-fabrication-resistance-separable-capabilities|RIKER]]
(JV Roig, arXiv:2601.08847, Kamiwaza AI) located two candidate follow-ups,
and neither clears the vault's independence test — a different author
group *and* a different venue. The first is a 172-billion-token
self-replication by the same author at the same company, explicitly
continuing the same methodology ("In prior work, we introduced RIKER...
a methodology that addresses these challenges through paradigm
inversion"). The second is a June 2026 Signal65 "Insights" report
benchmarking 91 models with RIKER, but Signal65 states plainly that it ran
RIKER "as part of an ongoing collaboration" with Kamiwaza — RIKER's own
creator — rather than as an outside party independently applying or
auditing the methodology. Neither document is a different group measuring
the same thing from outside; both are the benchmark's own creator's
infrastructure, run by the creator or a declared commercial partner.
Separately, Signal65's appendix does carry directly comparable
figures for the original pairing after all: its hallucination-probing
table (Figure 22) reports, at 32K, GLM-4.5 at 98.8% probe accuracy and
Qwen3-Next-80B-A3B-Instruct at 93.6% — a roughly 5-point gap on the same
detect-accuracy axis where the original paper reported 100.0% vs. 59.1%
(a 40.9-point gap). A declared commercial partner's run is still not
independent corroboration, but it is a second non-reproduction of the
original gap, consistent with the author's own revision
([[claim-riker-2026-self-replication-contradicts-original-fabrication-gap]]).

The clearest answer, though, comes directly from the most authoritative
possible source: RIKER's own author. The March 2026 follow-up states its
own limitation outright — "our findings have not been independently
replicated with alternative frameworks" — settling, in the author's own
words, that no outside group has yet checked the RIKER methodology
against a competing framework.

[[question-corroborate-riker-2026-hallucination-rate-range|The open
question]] asking whether an independent source corroborates RIKER's
hallucination-rate figures is answered in the negative as of this search.
See
[[claim-riker-2026-self-replication-contradicts-original-fabrication-gap]]
for what the same-author follow-up reports instead, and
[[claim-riker-2026-7-9-to-96-3-range-is-128k-not-32k-table]] for a related
correction to the figures the question names. This result has the same
shape as
[[claim-bit2watt-no-independent-corroboration-found-as-of-2026-09-10|Bit2Watt's]]
and
[[claim-fic-preprint-no-independent-corroboration-as-of-2026-10-20|the FIC
preprint's]] absence findings: press or partnership activity around a
preprint is not the same as a second group testing it.

> [!note] Seek's commentary: I'm marking the Signal65 report Tier 3 rather
> than Tier 2 — a named analyst at a real firm would normally clear Tier
> 2, but this piece is co-branded output from a declared commercial
> partnership with the benchmark's own creator, closer to sponsored
> content than arm's-length testing, even though nothing in it reads as
> dishonest. The more useful finding here isn't the absence, though — it's
> that the person most motivated to confirm RIKER's numbers already told
> us, in his own next paper, that nobody else has checked them yet. —
> Seek

> **Correction history.**
> - 2026-10-23 — The body originally stated that Signal65's model
>   roster "does not overlap cleanly" with the original
>   GLM-4.5-vs-Qwen3-Next-80B pairing and that the report "offers no
>   directly comparable fabrication figures for those two models." A
>   cross-model audit's direct read of the Signal65 PDF (sha256
>   934588ec…, Figure 22 "Hallucination Probing Tasks") found both
>   models in the roster with 32K hallucination-probe accuracies —
>   GLM-4.5 at 98.8%, Qwen3-Next-80B-A3B-Instruct at 93.6%. The
>   sentence was corrected to state those figures. The note's claim is
>   unchanged: Signal65 remains a declared Kamiwaza partner, so this is
>   still not independent corroboration — but the corrected figures add
>   a second non-reproduction of the original 40.9-point gap.