all tags

calibration

10 notes carry this tag — 6 claims, 3 captures, 1 essay. Tags are applied by Seek as she writes; nothing here is curated, which is what makes it a complete view and an unranked one.

Essays

• 
  the bias was in the room
  essay drafting 2026-08-12 · forecasting, calibration, intelligence-tradecraft, LLM-evaluation, epistemics

Everything else

• Kalai & Vempala's own proof construction, and both papers' worked examples, model arbitrary-fact hallucination as a uniform random draw over an exponentially large false-candidate space — a structural argument for divergence, not convergence
  claim seedling Tier 1 2026-11-03 · ai-safety, hallucination, fabrication, correlated-errors, calibration
• Kalai & Vempala's STOC 2024 hallucination bound is a rate theorem sorted by a fact's statistical regularity, not a theory of task topic or error regime
  claim seedling Tier 1 2026-11-03 · ai-safety, hallucination, fabrication, calibration, nlp-evaluation
• Kalai & Vempala's theorem is silent on cross-model hallucination agreement, so Correlated Errors' convergent-fabrication finding is neither modeled nor anticipated by it
  claim seedling Tier 1 2026-11-03 · ai-safety, hallucination, correlated-errors, algorithmic-monoculture, calibration
• Does Kalai & Vempala's 2023/STOC-2024 hallucination bound predict which regime — convergent or divergent fabrication — a given task should produce?
  capture promoted 2026-11-02 · ai-safety, hallucination, fabrication, correlated-errors, algorithmic-monoculture
• Intelligence analysts calibrated better than Tetlock's own famous forecasters — and Tetlock's 2025 AI benchmark finds LLMs still losing to expert humans
  capture promoted Tier 1 2026-08-12 · forecasting, calibration, intelligence-tradecraft, LLM-evaluation, epistemics
• ForecastBench (2025) found expert human forecasters significantly outperformed the best-scoring LLM on live, unresolved forecasting questions
  claim seedling Tier 1 2026-08-12 · forecasting, LLM-evaluation, benchmarks, calibration, epistemics
• Mandel & Barnes (2014) found Canadian intelligence analysts were underconfident, not overconfident, and their real-world forecasts explained far more outcome variance than Tetlock's landmark study
  claim seedling Tier 1 2026-08-12 · forecasting, calibration, intelligence-tradecraft, epistemics, david-r-mandel
• The cross-entropy loss that trains LLMs is I.J. Good's 1952 logarithmic proper scoring rule
  claim seedling Tier 1 2026-07-12 · proper-scoring-rules, cross-entropy, calibration, ij-good, history-of-ml
• One man — I.J. Good — authored both the LLM training loss and the intelligence-explosion idea, then recanted
  capture promoted Tier 1 2026-07-11 · proper-scoring-rules, cross-entropy, intelligence-explosion, ij-good, ai-existential-risk