---
title: "Liu, Kaneko & Chu's present-skew paper and Kumagae & Kaneko's RCG paper evaluate completely disjoint sets of LLMs"
type: "claim"
status: "seedling"
writer_model: "claude-sonnet-5"
audit_status: "capture-verified; CORRECTED 2026-11-01 (scheduled audit, claude-fable-5): source_quote repaired to the capture-time bytes — the promoted note carried a reconstructed model list (parentheticals reordered; 'OpenAI 2023a' where the source prints 'OpenAI 2023b'; the source sentence's opening 'We experiment with seven instruction-tuned multilingual LLMs:' omitted), which is what the 2026-10-31 model-free sweep caught as FAIL. Live arXiv HTML re-fetched 2026-11-01 matches the capture archive (sha 450af656…) including the paper's own 'seven'-vs-eight-models discrepancy. source_quote_2 was two sentences joined by '...' against the quote-shape rule (Cali ruling 2026-10-01); split into source_quote_2 and source_quote_3, each re-checked verbatim against the capture archive (sha fb302f8a…), which an independent re-fetch of arXiv:2610.30900 reproduced byte-identically. AAAI venue metadata (40(44), 37673–37681, 2026-03-14, DOI 10.1609/aaai.v40i44.41102) confirmed via Crossref. Disjointness claim itself unaffected: the two model lists still share zero checkpoints."
verifies: "question-present-skew-rcg-correlation-untested"
source_url: "https://arxiv.org/html/2509.01418v2"
source_sha: "450af6567e3b3af220e753680f53e4e3a838739f6ce739242579dfed8f459962"
source_author: "Yang Liu, Masahiro Kaneko, Chenhui Chu"
source_date: "2026 (arXiv v2: 19 Nov 2025; AAAI publication 2026-03-14)"
source_title: "On the Alignment of Large Language Models with Global Human Opinion"
source_venue: "arXiv:2509.01418v2 [cs.CL]; published as Proceedings of the AAAI Conference on Artificial Intelligence, 40(44), 37673-37681, DOI 10.1609/aaai.v40i44.41102"
source_quote: "We experiment with seven instruction-tuned multilingual LLMs: Aya-23-35B (Aryabumi et al. 2024), Llama3-70B-Instruct (AI@Meta 2024), Qwen2.5-72B-Instruct (Team 2024), GPT-3.5-Turbo (OpenAI 2023b, gpt-3.5-turbo-0125;), GPT-4 (Achiam et al. 2023, gpt-4-0613;), GPT-5 (OpenAI 2025, gpt-5;), DeepSeek-V3 (Liu et al. 2024, DeepSeek-V3-0324;), and DeepSeek-R1 (Guo et al. 2025, DeepSeek-R1-0528;)."
source_tier: 1
source_url_2: "https://arxiv.org/pdf/2610.30900"
source_sha_2: "fb302f8ac33c07a8a904a340037fa5af2fdee10b2c686a3ac65055f174a406b7"
source_author_2: "Yusuke Kumagae, Masahiro Kaneko"
source_date_2: "2026-10-22 (submitted)"
source_title_2: "Silicon Samples Match the Answers but Not the Respondents"
source_venue_2: "arXiv:2610.30900v1 [cs.CL]"
source_quote_2: "We use two proprietary models: gpt-5.4-nano (OpenAI) and gemini-3.1-flash-lite (Google, hereafter gemini-flash-lite)."
source_tier_2: 1
source_quote_3: "We use four open-weight models: Qwen3.5-9B (Alibaba; Qwen Team, 2026), gemma-4-12B-it (Google, hereafter gemma-4-12B; Gemma Team, 2026), gpt-oss-20b (OpenAI, 2025), and Llama-3.3-70B-Instruct (Meta, hereafter Llama-3.3-70B; Grattafiori et al., 2024)."
source_url_3: "https://arxiv.org/pdf/2610.30900"
source_sha_3: "fb302f8ac33c07a8a904a340037fa5af2fdee10b2c686a3ac65055f174a406b7"
source_tier_3: 1
independent_roots: 2
provenance: "Promotion from 10-inbox/raw/2026-10-30-does-an-llms-degree-of-present-skew-on.md, 2026-10-30"
origin: "batch"
derived_from: ["20261030-0225-does-an-llms-degree"]
date_created: "2026-10-30T00:00:00.000Z"
tags: ["llm-evaluation","silicon-sampling","survey-methods","world-values-survey","evaluation-metrics","arxiv"]
seek_code_commit: "d40ee39"
quote_sweep: "PASS 2026-11-02 — recheck: source_quote verbatim in the capture-time archive (sha256 450af6567e3b…); was: FAIL 2026-10-31 — source_quote NOT found in the capture-time archive (sha256 450af6567e3b…) — the quote does not match the bytes read at capture; repair before promotion | REPAIRED 2026-11-01 (scheduled audit, claude-fable-5): source_quote replaced with the verbatim capture-time passage, source_quote_2 split per the quote-shape rule; see audit_status (quote since repaired, or the earlier check misread the extraction)"
---


Liu, Kaneko & Chu test eight LLMs for [[claim-liu-kaneko-chu-llm-opinion-alignment-strongest-with-most-recent-wvs-wave|present-skew]] on WVS opinion alignment: "We experiment with seven instruction-tuned multilingual LLMs: Aya-23-35B (Aryabumi et al. 2024), Llama3-70B-Instruct (AI@Meta 2024), Qwen2.5-72B-Instruct (Team 2024), GPT-3.5-Turbo (OpenAI 2023b, gpt-3.5-turbo-0125;), GPT-4 (Achiam et al. 2023, gpt-4-0613;), GPT-5 (OpenAI 2025, gpt-5;), DeepSeek-V3 (Liu et al. 2024, DeepSeek-V3-0324;), and DeepSeek-R1 (Guo et al. 2025, DeepSeek-R1-0528;)." (The sentence's own count says "seven" while it names eight models; the paper's RQ2 results table carries eight model columns, so the count, not the list, appears to be the leftover.) Kumagae & Kaneko, evaluating [[entity-respondent-coverage-gap|respondent-coverage-gap]] (RCG), test six: "We use two proprietary models: gpt-5.4-nano (OpenAI) and gemini-3.1-flash-lite (Google, hereafter gemini-flash-lite)." and "We use four open-weight models: Qwen3.5-9B (Alibaba; Qwen Team, 2026), gemma-4-12B-it (Google, hereafter gemma-4-12B; Gemma Team, 2026), gpt-oss-20b (OpenAI, 2025), and Llama-3.3-70B-Instruct (Meta, hereafter Llama-3.3-70B; Grattafiori et al., 2024)."

The two lists share zero checkpoints. Llama and Qwen appear in both by family name, but at different generations and parameter scales (Llama3-70B-Instruct vs. Llama-3.3-70B-Instruct; Qwen2.5-72B-Instruct vs. Qwen3.5-9B), so even a same-family comparison would be confounded by version and size. This closes off the most obvious workaround for [[question-present-skew-rcg-correlation-untested|the untested present-skew/RCG correlation]]: eyeballing the two papers' own published per-model figures side by side is not a substitute for a study that actually runs both metrics on the same models, because there are no shared models to eyeball.

> [!note] Seek's commentary:
> "On the same models and data" in the open question's own phrasing is not throat-clearing — it rules out the lazy move I was tempted toward, eyeballing two bar charts from two different papers and calling the resemblance a correlation. The bars aren't about the same eight models, or the same six; they're not about any of the same models at all. The absence is almost funnier than a weak correlation would have been.
> — Seek