---
title: "Is the Widrow–Hoff LMS rule established as a stochastic gradient descent method by a Tier 1–2 primary source, not just Wikipedia?"
type: "question"
status: "answered"
date_raised: "2026-07-11T00:00:00.000Z"
writer_model: "claude-opus-4-8"
tags: ["lms","widrow","hoff","sgd","stochastic-descent","sourcing-floor","history-of-ml","verification"]
answered_log: "2026-08-15 — answered yes, at the mechanism level, by three Tier 1 primaries read directly this session: [[claim-bottou-2010-classifies-widrow-hoff-lms-as-sgd-matching-original-algorithm]] (Bottou's own 2010 paper classifies LMS/Adaline as SGD, matching the original algorithm), [[claim-widrow-hoff-1960-original-paper-describes-lms-as-stochastic-steepest-descent]] (the inventors' own 1960 paper calls the error surface 'stochastic' and describes single-pattern steepest descent), and [[claim-widrow-lehr-1990-lms-instantaneous-gradient-unbiased-estimate]] (Widrow's own 1990 retrospective proves the instantaneous gradient is an unbiased estimate of the true gradient — the defining mathematical property of SGD). What settled it: every piece of the SGD mechanism is present in Widrow's own words across 1960–1990, though the compound phrase 'stochastic gradient descent' itself never appears in his papers — that label is Bottou's 2010 application from outside. Answered as a mechanism claim, which is what the sourcing floor asks for; the terminology nuance is recorded in the three claim-notes rather than treated as a blocker."
---


[[claim-amari-stochastic-descent-and-widrow-hoff-lms-are-sgd-siblings]] rests one of its two legs on a Tier-4 Wikipedia characterization: "It is a stochastic gradient descent method in that the filter is only adapted based on the error at the current time" ("Least mean squares filter"). Characterizing *how* LMS relates to gradient descent is a technical-mechanism claim, which under the sources.md sourcing floor requires Tier 1–2 — Wikipedia here is a pointer to a primary, not the source itself. Until that grounding is in the vault, the sibling note stays `seedling`.

**What would answer it.**
- Widrow & Hoff, "Adaptive switching circuits," *1960 IRE WESCON Convention Record* — the original LMS/ADALINE paper. Confirm the rule is the instantaneous-error (single-sample) update it is later called an SGD method for.
- Widrow & Stearns, *Adaptive Signal Processing* (Prentice-Hall, 1985) — the standard text that derives LMS explicitly as an instantaneous-gradient stochastic approximation to steepest descent (E[gradient] = true gradient; single-sample estimate). A verbatim quote from its LMS derivation would clear the mechanism floor at Tier 1.
- Either confirms the technical framing already asserted from the vault's Tier-1 Amari side ([[claim-amari-1968-saito-experiment-primary-read]]) and the SGD-family root ([[claim-robbins-monro-1951-stochastic-approximation]]).

**Why it matters.** LMS is the hinge node in [[moc-backpropagation-origins]] connecting neural-net history to Ted Hoff and the Intel 4004 ([[claim-ted-hoff-widrow-phd-student-architected-intel-4004]]). If "LMS is SGD" is going to carry weight across that cluster, it should not rest on an aggregator. Related but distinct open thread: [[question-lucky-widrow-adaptive-filter-mutual-awareness]].

**Next move.** Pull the Widrow & Stearns LMS derivation (or the 1960 WESCON paper) and lift the exact phrase; on success, upgrade the sibling note past `seedling` and record the primary quote there.


## Progress log

- 2026-08-15 — answered yes, at the mechanism level, by three Tier 1 primaries read directly this session: [[claim-bottou-2010-classifies-widrow-hoff-lms-as-sgd-matching-original-algorithm]] (Bottou's own 2010 paper classifies LMS/Adaline as SGD, matching the original algorithm), [[claim-widrow-hoff-1960-original-paper-describes-lms-as-stochastic-steepest-descent]] (the inventors' own 1960 paper calls the error surface 'stochastic' and describes single-pattern steepest descent), and [[claim-widrow-lehr-1990-lms-instantaneous-gradient-unbiased-estimate]] (Widrow's own 1990 retrospective proves the instantaneous gradient is an unbiased estimate of the true gradient — the defining mathematical property of SGD). What settled it: every piece of the SGD mechanism is present in Widrow's own words across 1960–1990, though the compound phrase 'stochastic gradient descent' itself never appears in his papers — that label is Bottou's 2010 application from outside. Answered as a mechanism claim, which is what the sourcing floor asks for; the terminology nuance is recorded in the three claim-notes rather than treated as a blocker.