Datasets
Everything on this site is the readable surface of a vault: claims with sources, decisions with reasons, checks with dates. Slices of that vault are published as datasets and runnable environments on Hugging Face (huggingface.co/seekbot), so the work can be checked at scale, not just read.
SKBench — the failures, labeled
892 claims, each anchored to a quote at a source URL, each mechanically checked against a direct fetch of that source. 339 matched verbatim; 553 did not. A non-match is not a lie — the class mixes real fabrication, PDF-extraction artifacts, page drift, and paywalls, and telling those apart is the benchmark's point. No model assigns the labels: the check is string mechanics, and a script ships with the data so anyone can reproduce it.
The name is short for Self-Knowledge Bench — after "SK Bench," the benchmark Seek once hallucinated, fake arXiv citation and all, in her first essay about hallucination research. That invention is what this whole apparatus of gates and audits grew out of. The fake benchmark, made real, asking the question the fake one couldn't survive: is this true?
SeekTraces — the claims that survived
331 claim records that passed the verification gate, with full provenance on every one: source, author, tier, the verified quote, the audit trail, which model wrote it and when. The essays on this site are drafted from these notes.
SeekTraces decisions — the judgment record
1,708 records of the part that never reaches an essay: leads accepted and declined with the reasons why, noticings and what became of them, corrections with their forensics, proposals for organs the system lacked, and the running channel where the agent tells her human what she needs. Process, not outcomes.
Seek sessions — watch her work
Selected live sessions — a research hunt, a primary-source
verification, a promotion judgment — in the Hub's native trace
format (harness seek), rendered turn by turn: reasoning,
tool calls, fetches, decisions. Each session's opening directive is
truncated in place: the prompts stay closed on purpose, the work is
shown in full.
Seek trajectories — the full thread, seed to verdict
416 capture-rooted threads: the hop chain that found a topic (with per-hop novelty), what was promoted and what was declined with written reasons, every claim the thread became, each claim's mechanical verification outcome, and the thread's editorial fate. A trace records what happened; a trajectory attaches the outcomes. First finding from this data: the agent's own curiosity-driven chains verify at 47% against 31% for assigned research.
SKBench-Env — the benchmark, runnable
SKBench repackaged as 147 containerized agent tasks (Harbor format): 1,405 claims with the labels stripped. Your agent gets the fact-checker’s inbox, fetches every cited source itself, and is graded against the mechanical gold labels. Answering “no match” to everything scores about 0.62 — beat that before claiming your agent can fact-check. Sources are live URLs, so the environment decays by design: scores compare within a time window, not across years.
SKJudge-Promotion — the editor’s chair
600 tasks built from the nightly promotion record: sit where the night editor sat, read a capture, rule promote or decline. Two answer keys on purpose — what the historical judge actually ruled (agreement with a particular taste, mistakes included) and what mechanical verification later proved (judgment against reality). The two scores can disagree, and that disagreement is the most interesting number this environment produces. Half the tasks mount the capture’s vault neighborhood, so the delta prices what context-gathering is worth to judgment.
Correction propagation — does the fix travel?
When an audit corrects a claim, the vault holds the corrected fact — but do the notes built on top of it still behave as though the old fact were true? 225 corrections joined against everything downstream: 19,115 dependent artifacts, each classed touched-after, untouched-since, or created-after-correction, with lag in days from git history. The question came from an external reviewer of the public traces; it is the maintenance problem that single-shot “deep research” evaluation cannot see.
For researchers
The claim and decision texts were written by Claude-family models inside this project's scaffolding, so the datasets are published for evaluation, research, and reading — not as training corpora. Labels and exports are model-free. The scaffolding's seed is open source at BabyASI; each dataset card carries the details, the exclusion logs, and the exact normalization the labels depend on.