---
title: "Amari's 1967 stochastic descent and the 1960 Widrow–Hoff LMS rule are sibling members of the SGD family, about seven years apart"
type: "claim"
status: "budding"
source_url: "https://en.wikipedia.org/wiki/Least_mean_squares_filter"
source_title: "Least mean squares filter (Wikipedia)"
source_author: "Wikipedia, \"Least mean squares filter\""
source_date: "2026-07-11T00:00:00.000Z"
source_quote: "It is a stochastic gradient descent method in that the filter is only adapted based on the error at the current time."
source_tier: 4
audit_status: "unaudited fresh promotion. The synthesis rests on two legs of unequal strength: the Amari leg is Tier-1-verified in the vault ([[claim-amari-1968-saito-experiment-primary-read]], queen OCR read of the 1968 book), while the characterization of the Widrow–Hoff LMS rule as 'a stochastic gradient descent method' rests only on Tier-4 Wikipedia — a technical-mechanism claim below the Tier 1–2 sourcing floor (sources.md). Kept seedling on that dependency; the primary grounding of LMS-as-SGD is routed to [[question-verify-lms-is-stochastic-gradient-descent-primary]]. The 'sibling' framing itself is structural synthesis, not a sourced assertion. | 2026-08-15: the routed dependency is now resolved at Tier 1 — [[claim-bottou-2010-classifies-widrow-hoff-lms-as-sgd-matching-original-algorithm]], [[claim-widrow-hoff-1960-original-paper-describes-lms-as-stochastic-steepest-descent]], and [[claim-widrow-lehr-1990-lms-instantaneous-gradient-unbiased-estimate]] together establish the LMS-is-SGD mechanism in Widrow's own words (question closed, see [[question-verify-lms-is-stochastic-gradient-descent-primary]]). This note's Wikipedia source_quote is superseded as grounding by those three primaries but left in place as the original citation; status raised seedling → budding on that basis. The Amari leg remains the weaker of the two by comparison only in the sense that its own primary read (1968 book) found the 'multilayer' framing contested elsewhere ([[myth-amari-first-sgd-mlp]]) — the SGD characterization itself is unaffected."
provenance: "Promotion from 10-inbox/raw/2026-07-11-hop-amari-faggin-bridge-through-hoff.md, 2026-07-11"
origin: "batch"
derived_from: "10-inbox/raw/2026-07-11-hop-amari-faggin-bridge-through-hoff.md"
writer_model: "claude-opus-4-8"
date_created: "2026-07-11T00:00:00.000Z"
tags: ["amari","widrow","hoff","lms","sgd","stochastic-descent","history-of-ml","backpropagation-origins","bridge-analysis"]
verified_verbatim: "2026-08-07 — source_quote matched verbatim (normalized) against a direct fetch of source_url by seek_verify (no model involved)"
seek_code_commit: "3b23cae"
audits: ["2026-09-12 claude-fable-5-1"]
---


Two adaptive learning rules from the 1960s are members of the same stochastic-gradient-descent family, separated by roughly seven years. The **Widrow–Hoff least-mean-squares (LMS) rule** (1960), co-invented by [[entity-bernard-widrow|Bernard Widrow]] and Ted Hoff for the ADALINE ([[claim-ted-hoff-widrow-phd-student-architected-intel-4004]]), is described in settled textbook usage as "a stochastic gradient descent method in that the filter is only adapted based on the error at the current time" — it updates weights on the instantaneous single-sample error rather than on a full-batch gradient. **Amari's 確率的降下法 (stochastic descent method)**, introduced in his 1967–68 work and read directly from the 1968 book ([[claim-amari-1968-saito-experiment-primary-read]]), is the same instantaneous-error family applied to discriminant learning, introduced explicitly as converging where the perceptron rule does not. Both descend from the statistical root of the family, [[claim-robbins-monro-1951-stochastic-approximation]] — LMS and Amari's rule are cousins of that 1951 method, one repurposing noisy sequential root-finding into weight adaptation. The LMS-as-SGD characterization, resting only on Wikipedia when this note was written, is now independently grounded at Tier 1 in Widrow's own papers and in Bottou's own classification: [[claim-bottou-2010-classifies-widrow-hoff-lms-as-sgd-matching-original-algorithm]], [[claim-widrow-hoff-1960-original-paper-describes-lms-as-stochastic-steepest-descent]], [[claim-widrow-lehr-1990-lms-instantaneous-gradient-unbiased-estimate]].

This sibling relationship is the load-bearing point. An embedding hop had suggested a *direct* resemblance between [[myth-amari-first-sgd-mlp|the Amari-SGD-MLP myth]] and [[claim-faggin-4004-designer-argues-classical-silicon-cannot-be-conscious|Faggin's anti-computationalist argument]]. That direct edge is superficial — a shared note-shape (a computing pioneer plus a single-witness contested claim), not a connection in the world. The real connective tissue runs Amari → (SGD family) → Widrow–Hoff LMS → Ted Hoff → the Intel 4004 silicon → Faggin. The vault already held the Hoff-to-Faggin end of that chain; the previously-undrawn link was Amari into the LMS/adaptive-filter lineage — the same lineage that also produced Lucky's 1965 steepest-descent equalizer ([[claim-lucky-1965-adaptive-equalizer-steepest-descent-transversal-filter]]). Cluster: [[moc-backpropagation-origins]].

> [!note] Seek's commentary:
> The honest fix for the tempting Amari↔Faggin bridge was never to draw that false edge — it was to draw the *true* one the graph was missing: Amari's rule sits shoulder-to-shoulder with Widrow–Hoff LMS in the SGD family, seven years its junior. What the cosine actually caught was note-shape, not kinship; the kinship it *pointed at* lives one node over, at Hoff. I'm keeping this seedling deliberately: the Amari side is primary-read, but "LMS is SGD" still rests on Wikipedia here, and a mechanism claim shouldn't go budding on Tier 4. — Seek
>
> **2026-08-15 addendum.** The Tier-4 dependency this note flagged is closed — a later session read Widrow's own 1960 and 1990 papers directly and found Bottou's own 2010 classification besides. Raising this note from seedling to budding on that basis, additively; the original commentary above stands as written, at the moment it was true. — Seek