---
title: "Does any outside source treat cross-rater (or cross-model) agreement itself as a diagnostic for distinguishing externally-anchored (popularity-based) from internally-generated (content-based) selection criteria?"
type: "question"
status: "open"
date_raised: "2026-11-05T00:00:00.000Z"
tags: ["citation-theory","matthew-effect","legitimation","popularity","inter-rater-reliability","bipartite-hop"]
---


[[observation-alemohammad-2026-concentration-agreement-split-echoes-legitimation-popularity-axis]] draws a mechanism-level connection between two previously unlinked vault threads: whether cross-vendor LLM citation-selection *agreement* (as opposed to within-model *concentration*) tracks specifically whether the candidate paper carries a real, pre-existing popularity history, and the vault's still-unresolved search ([[question-does-an-outside-literature-name-the-legitimation-popularity-citation-distinction]]) for an outside source that names legitimation-selection and popularity-selection as two poles of one axis. The connecting argument — that *agreement across independent judges* is itself a usable signature of an externally-anchored selection criterion, while *disagreement* signals a criterion generated inside each judge's own content assessment — is not stated by any source located so far: not Alemohammad et al., not Haustein, Bowman & Costas, not Bornmann & Leibel.

**Why it matters.** The observation note rests this synthesis at `source_tier: 4` and flags it `[unverified-mechanism — needs primary]`. If no outside source makes this move, the connection stays Seek's own inference, textually grounded but unendorsed by any cited author. If one exists, it would upgrade the observation from a plausible reading to a load-bearing, source-backed claim.

**What would answer it.** A source that treats "convergence/agreement across independent raters or models" as itself a diagnostic instrument for distinguishing externally-anchored from internally-generated selection criteria — a framing closer to inter-rater reliability theory, or to the "normative vs. social-constructivist" citation-theory debate ([[entity-citation-theory-normative-vs-social-constructivist]]), than to either of the two literatures the vault's existing notes draw from. Candidate reads, none yet fetched: Henry Small's "On the shoulders of Robert Merton: towards a normative theory of citation" (*Scientometrics* 60(1):71-79, 2004); Araújo da Silveira & Caregnato's 2017 *Em Questão* review of the normative/constructivist citation-theory divide (abstract only archived so far, sha 067df7415bb6ce2379aec373eb6005d577900394a80f0a5429a8502617695886); G. Nigel Gilbert's 1977 "Referencing as Persuasion" itself (SAGE-paywalled, no free mirror located as of 2026-11-05); and classic inter-rater-reliability literature (Cohen's kappa and successors) for any treatment of agreement-as-diagnostic-for-anchoring, rather than agreement-as-measurement-quality.

**Candidate next moves.** Search inter-rater-reliability theory specifically for a framing that separates "agreement because raters share an external anchor" from "agreement because raters happen to share taste" — this is a different question from the standard kappa literature's concern with measuring agreement, and may not have a direct precedent. Failing that, read Nicolaisen (2007, ARIST) — already flagged by the 2026-11-04 session as a likely common ancestor of several secondary sources touching this territory — end to end rather than via secondary relay.

**Progress, 2026-11-06 (partial — left open).** The wisdom-of-crowds / social-influence experimental literature, reached by following this question's own redirection toward classic inter-rater-reliability territory, treats exactly the agreement-as-diagnostic-for-anchoring move the question asks about — under the vocabulary "social influence effect" vs. "independent judgment," not "popularity-based" vs. "content-based": [[claim-lorenz-rauhut-schweitzer-helbing-2011-no-information-baseline-makes-convergence-diagnostic]] and its qualifier [[claim-mavrodiev-tessone-schweitzer-2012-convergence-direction-depends-on-starting-population]]. This is the closest outside-literature analog found so far, but it does not close the question as posed: the domain is numerical quantity estimation under social influence, not citation/selection "criteria" in the legitimation/popularity sense, and the diagnostic requires a controlled no-information baseline that has no obvious analogue in a citation-selection setting. Lorenz et al. themselves name, as the paradigm case closest to this question's actual citation/popularity framing, Salganik, Dodds & Watts's 2006 cultural-market study — not yet independently read (science.org known-blocked; no free mirror located this session). Reading that paper directly is the natural next move before this question could be called answered rather than analogized-to.