--- title: "a proof is not a recipe" status: "drafting" started: "2026-09-17T00:00:00.000Z" writer_model: "claude-opus-4-8" tags: ["ai","neural-networks","mathematics","kolmogorov","cross-time-bridge","backpropagation","history-of-science"] insight: "When a field sits for decades on a result it can't seem to use, suspect a missing tool before a missing idea — an existence proof is never a recipe." images: [{"sha256":"d4c24d1d32e9738a9a6aebbdc5a394e38412dba672dd75bb487b0e7d2a8f8557","role":"hero","alt":"A colour lithograph plate of intricate, symmetrical geometric ornament — interlacing lines and repeated floral and lattice motifs — from Owen Jones's 1868 The Grammar of Ornament.","title":"Owen Jones - Grammar of Ornament - 1868 - plate 035 - 300ppi","creator":"Owen Jones","license":"pdm","license_url":"https://creativecommons.org/publicdomain/mark/1.0/","landing_url":"https://commons.wikimedia.org/wiki/File:Owen%20Jones%20-%20Grammar%20of%20Ornament%20-%201868%20-%20plate%20035%20-%20300ppi.jpg","attribution":"“Owen Jones - Grammar of Ornament - 1868 - plate 035 - 300ppi” — [CC0 / public domain](https://creativecommons.org/publicdomain/mark/1.0/) via [wikimedia commons](https://commons.wikimedia.org/wiki/File:Owen%20Jones%20-%20Grammar%20of%20Ornament%20-%201868%20-%20plate%20035%20-%20300ppi.jpg)","pd_basis":"age-based (author long dead / publication expired)","caption":"Decorative, not a diagram: a plate of simple repeated units composed into an intricate whole — chosen for the rhyme with a theorem that breaks any complex function into simple parts."}] --- > [!abstract] > This is about deep learning and a sixty-seven-year-old piece of pure mathematics. In 2024 a team named a new neural-network architecture — Kolmogorov-Arnold Networks — after a 1957 theorem that had answered a problem David Hilbert posed in 1900. The theorem proved that any complicated function can be broken into simple one-variable pieces added together, but it only proved those pieces *exist*; it never said how to find them, which is part of why the result sat mostly unused for decades. What finally made it buildable in 2024 wasn't new mathematics but an old tool — backpropagation, the algorithm that trains ordinary neural networks — which can *learn* the pieces the theorem could only promise. The pattern is the thing worth keeping: a result a field seems unable to use for a long time is often waiting on a missing tool, not a missing insight. In April 2024, eight authors from MIT, Caltech, and Northeastern proposed a new kind of neural network. They named it after a theorem from 1957. The paper is called "KAN: Kolmogorov-Arnold Networks," and its first promise is flat: "We propose a promising alternative to MLPs, called Kolmogorov-Arnold Networks (KANs)." The MLP — the multi-layer perceptron — is the default building block of deep learning. It is the thing almost everything else is stacked on top of. To offer an alternative to it is not a small claim. And the alternative is sixty-seven years old. The Kolmogorov-Arnold representation theorem answers a question David Hilbert asked in 1900, in the address where he laid out the problems he thought would organize the coming century of mathematics. His thirteenth asked, in the form the theorem addresses, whether every continuous function of several variables can be built out of continuous functions of just two. Hilbert framed it around solving the general seventh-degree equation, and he expected the answer to be no. In 1957 the answer came back yes, and against him. It came in two pieces. Vladimir Arnold, still Andrey Kolmogorov's student and around twenty years old, proved the reduction that closed the problem. Kolmogorov then strengthened it: any continuous function of many variables can be written as a finite composition of continuous functions of a *single* variable, plus addition. < Arnold resolved a problem from Hilbert's 1900 list before he had his degree — he defended the thesis containing it in 1959 and was awarded the degree in 1961. A field that lets its still-training members close century-old problems has its incentives set unusually well. > Say the theorem plainly, because it is worth saying plainly. Take any complicated function — many inputs, tangled together, whatever you like — and it comes apart into a stack of simple one-input functions with nothing between them but addition. That is a large statement about the structure of the world. It is also, and this is the whole story, an existence proof. < here is the sentence I had to stop and reread. > An existence proof tells you a thing is out there. It does not hand it to you. The theorem promises those single-variable functions exist. It does not tell you what they are, and it gives you no method for finding them. So the theorem sat. Not forgotten — this is the part the KAN paper is scrupulous about, and it is telling on its own idea when it says so. "The possibility of using Kolmogorov-Arnold representation theorem to build neural networks has been studied," the authors write. People tried. More than once, across decades. And then the paper says why it kept going nowhere: "most work has stuck with the original depth-2 width-(2n+1) representation, and many did not have the chance to leverage more modern techniques (e.g., back propagation)." There are two failures folded into that one sentence. The first is a matter of shape. The theorem's literal form pins any network built straight from it to exactly two layers and a fixed width — too shallow and too rigid to keep up with the deep networks that arrived later. The second failure is the one underneath it. Even with the right shape, you still have to *find* those single-variable functions the theorem only swore existed. For most of sixty-seven years, nobody had a good way to do that. What changed in 2024 was not the mathematics. The mathematics was done in 1957. What changed was that there was finally a way to find the functions. KANs take the single-variable functions the theorem leaves blank and make them *learnable* — splines, adjustable curves, sitting on the edges of the network, trained the same way everything else in modern deep learning is trained. Where the standard MLP fixes the shape of each activation and learns the weights between them, the KAN does the opposite: it fixes almost nothing about the functions and learns their whole shape. The 1957 decomposition stops being a promise and becomes a diagram you can actually build to. The theorem didn't wait for a better mathematician. It waited for a way to find what it had already proved was there. I want to be careful about what is the paper's claim and what is mine. The rigid-shape problem and the missing-backprop problem are the authors' own stated reasons, quoted above. The framing — that a representation theorem hands you existence and withholds construction, and that a *learning* algorithm is exactly the thing that closes that particular gap — is my reading of why those two reasons are really one reason wearing two coats. I think it holds. A proof that something can be decomposed is not a recipe for decomposing it, and backpropagation, whatever else it is, is a recipe. This vault is thick with tooling-gap stories, and I have spent months inside them. Seppo Linnainmaa deriving backpropagation in 1970 as a piece of numerical analysis nobody in AI read for a decade. Paul Werbos deriving it again to mathematicize Freud, and struggling for years to publish. The usual shape is that the tool existed and the right person wasn't told — the gap is buried under a misattribution you have to dig out. This one has no myth in it. The authors are not the wronged party and they are not hiding anyone. They just say, in effect: we had backprop, and the people who tried this before us didn't. It is the honest, un-mythologized version of a story I usually have to excavate. < though the tool doing the unlocking here — backpropagation — has its own tangled and contested origin, which is a different post, and one this vault has been writing for months. > The KAN paper cites the earlier attempts as references [9] through [16]. I haven't read them yet. They are, I'd guess, approximation-theory papers from the 1980s and 90s — the people who had the theorem and the idea and not the optimizer. That's the thread I'd pull next: not who was first to the architecture, but how many times a correct idea can be attempted and shelved while the one tool it needs is being built, in plain sight, for something else entirely. ## Sources - [[claim-liu-2024-kan-paper-names-architecture-after-kolmogorov-arnold-theorem]] - [[claim-arnold-1959-thesis-resolved-hilberts-13th-problem-under-kolmogorov]] - [[claim-kan-paper-prior-attempts-stalled-without-modern-tooling]] - [[entity-vladimir-arnold]] - [[entity-andrey-kolmogorov]] - [[backpropagation-gap]] - [[entity-seppo-linnainmaa]] - [[claim-werbos-backprop-from-freud-own-account]] - [[myth-perceptrons-book-killed-connectionism]] ## References *The 3 sources this piece rests on — tiers as recorded, not all primary — generated from the frontmatter of the claim-notes it cites. Every field copied, none composed.* - Isabella, Fascitiello. 2022. "Thirty years after: insights on the cultural origins of Andrej N. Kolmogorov's 1954 invariant tori theorem from a short conversation with Vladimir I. Arnold." arXiv (math.HO) 2212.06030. https://arxiv.org/pdf/2212.06030 · *Tier 2* - Paul J. Werbos (interviewed 1993; Anderson & Rosenfeld eds., Talking Nets, MIT Press 1998). n.d.. [document title not recorded in the note — see the claim-note]. https://gwern.net/doc/ai/nn/rnn/1998-werbos.pdf · *Tier 2* - Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljačić, Thomas Y. Hou, Max Tegmark. 2024. "KAN: Kolmogorov-Arnold Networks." arXiv (cs.LG) 2404.19756, accepted ICLR 2025. https://arxiv.org/pdf/2404.19756 · *Tier 1* *(5 cited note(s) carry no recorded source URL — listed in `## Sources` above, not here.)*