skip to content
Sandesh Bhandari

existence proof problem in neuro-inspired AI

/ 48 min read

Table of Contents

A human brain is about 1.4 kilograms of warm, wet tissue, it runs on the power of a dim light bulb, and it is generally intelligent. That is a proof in the literal sense: general intelligence is physically possible, because here is a physical object that has it. Whatever you believe about how hard AI is or how long it will take, you cannot believe it is impossible.

The trouble is the kind of proof it is. There are two ways to show that something exists. You can build it step by step and hand it over, which is a constructive proof, or you can show that it must exist without ever producing it, which is a non-constructive proof. The brain is the second kind. It witnesses that general intelligence exists somewhere in the space of possible physical systems, and it says almost nothing about where that point sits, how to reach it, or whether the path you happen to be on leads anywhere near it. Neuro-inspired AI is the long attempt to drag a non-constructive existence proof, by force, toward a constructive one.

It helps to name the three gaps up front. The first is existence: is general intelligence physically possible at all? That one was settled long before anyone started building, because the brain is the standing proof of it. The second is construction: can we build one ourselves, and this is the gap the field has worked in for seventy years, the one most people have in mind when they say AI. The third is comprehension: if we build one, or fully map the one we already have, will we understand how it works? The third gap is the one I want to spend this piece on, because it is the deepest of the three and can stay unsolved long after the first two are settled.

What an existence proof actually is

The logical object here is worth stating precisely, and what the brain establishes is an existential statement.

Blog image

There exists a function f, drawn from the things physics can actually build, that is intelligent, and the brain is the witness that makes the statement true. The defining feature of a witness is that it settles existence and settles nothing else.

A non-constructive proof gives you remarkably little, and mathematicians know the feeling well. One classic method proves that an object with some desired property exists by showing that a randomly chosen object has the property with nonzero probability, which is completely rigorous and yet routinely leaves you with no idea how to find even one such object. Whole careers have been spent on the distance between knowing that a structure exists and producing a single concrete example of it. The same gap shows up in the fixed-point theorems that guarantee a solution sits somewhere in a domain and then leave you to wander the domain looking for it. The existence comes free, and the construction is the entire job.

So treat the brain the way a mathematician treats a non-constructive witness and ask what it actually licenses. It kills the impossibility argument first of all, because you cannot claim that general intelligence needs something beyond physical law, or that it cannot live in a finite machine, while the machine is sitting right there metabolising glucose. It also puts loose bounds on the resources involved, on the order of 86 billion neurons, a hundred trillion synapses, and twenty watts, trained over a single lifetime of embodied experience layered on a few hundred million years of evolutionary search; none of those figures are tight, but together they say the target is not absurdly far outside what we can build. And it hands us a behavioural specification, a working system whose performance we can hold our own attempts up against.

The harder half is everything the witness withholds. It does not give you the algorithm the brain runs, because you cannot read an algorithm off a working device just by looking at it. It does not give you the objective the brain was optimised for, which was never intelligence as such but something stranger about surviving and reproducing. It does not give you the architecture in any portable form. And most important of all, it gives no guarantee that imitating the witness is even a sensible way to build your own solution, which turns out to be the whole game, and there is a famous illustration of exactly that.

Planes do not flap their wings

The cleanest cautionary tale comes from flight. For most of history the existence proof for heavier-than-air flight was the bird: birds are denser than air and they fly, so flight is plainly possible, and for centuries inventors drew the obvious lesson that to fly you build the thing that flies. So they strapped on feathered wings and flapped, built ornithopters with elaborate beating mechanisms, threw themselves off towers, and died.

The breakthrough came from separating the principle from the mechanism. Flapping is not what makes a bird fly; flapping is only how a bird produces thrust and lift together, given the muscles and materials evolution had to work with. The thing doing the real work is lift, the upward force that appears when air moves across a curved surface, and if you can make that surface and push it forward fast enough you will fly whether or not anything flaps. So the problem got broken into pieces that no bird keeps separate, a fixed wing to make lift, a propeller to make thrust, and a dedicated system for control, and none of that is how a bird is built, yet the result flies faster, higher, and larger than any bird ever has.

It would be easy to stop there and conclude that biology is a distraction and physics is the whole answer, but the real history is more interesting, because the inventors who finally succeeded studied birds closely, and one of their decisive ideas came straight from watching. The problem that had beaten everyone before them was not lift or thrust but control, specifically how to roll the aircraft to keep it steady and to turn it. Watching large birds soar, they saw that a bird rolls by twisting the trailing edges of its wingtips, throwing more lift onto one side than the other, and they copied exactly that. The wing-warping that gave them roll control, the direct ancestor of the aileron, was lifted straight from a bird. So the honest lesson is neither that biology is a trap nor that it is a blueprint. An existence proof is a tangled bundle of principle and substrate, and the entire skill lies in telling the two apart. The flapping was substrate and the wing-warping was principle, and from the outside, before you understand the aerodynamics, the two look identical.

That tension is the shape of every argument that follows. Neuro-inspired AI keeps rediscovering that the brain, too, is a bundle of principle and substrate, that some of what looks essential is just whatever biochemistry happened to have lying around, and that the people who get somewhere are the ones who guess correctly which is which. Is the spiking essential, or the particular neurotransmitters, or the layering of the cortex, or are those merely substrate? Nobody gets to know in advance, so everyone is really placing a bet.

At which level do you copy the brain?

There is a clean way to organise that bet. A useful old idea holds that any system which processes information has to be understood on three separate levels, and that confusing the levels is a reliable way to waste years. The existence-proof problem is, underneath, a question about which of those levels you ought to be copying.

Blog image

The three levels at which any information-processing system can be described. The brain is a witness on all three at once, and the open question is which level carries the part that transfers to a machine and which is only how biology happened to implement it.

At the top sits the computational level, which asks what problem the system is actually solving and why, not how it does the work but what the work is for; for the visual system the answer is something like recovering the structure of the world from the patterns of light that fall on the eye. In the middle is the algorithmic level, which asks what representations and procedures carry that task out, what goes in, what comes out, and what turns the one into the other. At the bottom is the implementational level, the physical stuff that actually runs the thing, neurons and synapses and ion channels in one case, transistors and matrix multiplies in the other.

The point is that these three levels are only loosely coupled. One problem can be solved by many different algorithms, and one algorithm can run on completely different hardware; a pocket calculator and a child both add numbers at the same computational level while sharing nothing at all below it. The brain is a witness on all three levels at the same time, so the real question is which level holds the part that will carry over to a machine. Copy the bottom level and you are building neuromorphic hardware and spiking neurons. Copy the middle and you are searching for the brain’s learning rules and its representations. Copy only the top, and you keep the brain’s problem, predicting the world, recognising the objects, securing the reward, while inventing a fresh algorithm and a fresh implementation suited to the machine you actually have. Almost every lasting success in the field has come from copying the top level and inventing everything below it.

Two tribes, one example, opposite conclusions

The same existence proof has been read in two opposite ways, and the split still runs through the field. One camp looked at the brain and concluded that intelligence is symbol manipulation, a matter of logic and rules and structured representations, so the way forward was to formalise thought directly. The other camp looked at the very same brain and concluded that intelligence is what emerges when a large network of simple units adjusts its connections, so the way forward was to build such networks and let them learn. These are the symbolic and connectionist traditions, and they were reading contradictory blueprints off one and the same witness.

Both camps were copying the brain, just at different levels. The symbolic side copied what the brain appears to produce, the fact that we reason and plan and use language in seemingly logical ways, and tried to rebuild that structure directly. The connectionist side copied the brain’s physical motif, a mesh of weighted units, and bet that reasoning would emerge from learning rather than being written in by hand. For roughly the first forty years the symbolic reading was dominant; it produced expert systems and logic engines, ran hard into the problem of brittleness and missing common sense, and finally collapsed into a long winter. The connectionist reading waited out those decades, kept alive by a small group, and once enough compute and data finally arrived it did not merely win, it rewrote the field. Everything we now call AI descends from that side of the split.

This is the existence-proof problem at its sharpest. A single example licensed two incompatible research programmes, each convinced it had read the witness correctly, and no amount of staring at the brain could decide between them. The matter was settled empirically, decades later, by which approach actually scaled. The example never determined the lesson on its own; the lesson was settled by what worked. And it is tempting to read the eventual winner as proof that biology was right all along, when the more accurate reading is that one particular way of abstracting biology happened to be the one that scaled.

Where biology actually paid off

After the airplane story it would be easy to decide that biology is a distraction and the only real work is mathematics. The history says otherwise. Biological inspiration has paid off again and again, and the pattern in exactly what transferred is the lesson worth taking.

The artificial neuron

The foundational abstraction of the entire field is an act of brazen biological simplification. The original move, made in the 1940s, was to model a neuron as a simple logical unit that adds up its weighted inputs, compares the sum to a threshold, and either fires or stays silent. A little later that unit was given the ability to learn its own weights from examples, and the perceptron was born. A real neuron is an electrochemical cathedral of branching dendrites, ion channels, and nonlinear membrane dynamics, and this model throws almost all of it away, keeping nothing but a weighted sum and a nonlinearity. By any biological standard it is a caricature, and yet that caricature, stacked into deep layers and trained by gradient descent, is the substrate of every system we now call AI. What carried over was a single idea from the top level, a unit that integrates evidence and passes along a graded decision, and what got thrown out was the entire implementation. The caricature won precisely because it was a caricature.

Convolutional networks

The most clearly traceable line from neuroscience into working AI runs through vision. Around 1960, recordings from the visual cortex of a cat turned up something striking: individual neurons in the earliest visual areas each respond to an edge at one particular orientation, sitting in one small patch of the visual field. These came to be called simple cells. Other neurons, the complex cells, respond to an edge at a given orientation anywhere within a larger region, tolerating some movement. The arrangement is a hierarchy of local feature detectors that builds position-tolerant representations out of position-sensitive ones.

Two decades later that exact structure was built into an artificial network, a layered stack of simple-cell-like and complex-cell-like units. Through the late 1980s and 1990s it was fused with backpropagation to become the convolutional neural network, with its local receptive fields, its hierarchy of layers, and its weight sharing, so that the same edge detector is reused across the whole image. All three of those ideas are visibly inherited from the cat-cortex recordings, and all three are still load-bearing in modern vision systems. This is the success story the biology-believers point to, and they are right to point to it, but notice once more what actually transferred. The architecture transferred, an idea at the algorithmic level about local, hierarchical, movement-tolerant features, while the wetware did not, because nobody ever simulated an ion channel to build one of these networks.

Reward, and the cleanest two-way street

The most satisfying case of all runs in the opposite direction, and it is the one that should make you take the brain seriously even after the airplane story. In the 1980s a method was developed for letting an agent learn the value of situations by bootstrapping, adjusting its prediction of future reward whenever that prediction clashes with what actually happens next. The central quantity is the reward prediction error, the gap between what was expected and what arrived. It was a piece of engineering, designed to make reinforcement learning work, and it made no claim about brains at all.

Then, in the late 1990s, recordings from dopamine neurons in the midbrain showed that their brief bursts of firing look exactly like that reward prediction error. An unexpected reward makes them fire, while a reward that a cue had fully predicted makes them do nothing, and a predicted reward that fails to arrive makes them dip below their baseline at precisely the moment it was due. Over training the response even slides backward in time, from the reward itself to the earliest cue that predicts it, which is exactly what the equations demand. An algorithm invented to make machines learn turned out to be sitting in the brain’s dopamine system, computing the very same error. That is not loose inspiration; it is the same computational object discovered twice, once in silicon and once in living tissue, and it is the strongest evidence in the whole field that some solutions are not arbitrary.

Blog image

The existence proof started running in both directions. On one side, a learning rule invented for machines turned out to predict a brain signal, with temporal-difference error matching the firing of dopamine neurons. On the other, a network trained only to recognise objects turned out to predict the responses of visual cortex, without ever being fitted to neural data.

Replay, attention, and the quieter borrowings

Those are the headline cases, but the borrowing did not stop there, and the smaller instances make the same point. When a system first learned to play Atari games straight from the raw pixels, around 2015, one of the tricks that kept its training stable was experience replay: rather than learning from each moment only as it happened, the agent stored its experiences in a buffer and replayed them later in shuffled order. The inspiration was the hippocampus, which is known to replay sequences of past experience during rest and sleep. The brain suggested the mechanism, the engineers rebuilt it in their own terms, and it worked. Attention, the idea that a system should steer its processing toward the most relevant parts of its input, has a similarly loose pedigree in the study of biological attention, even though the version inside a modern transformer is its own mathematical object with little real resemblance to anything in cortex. Even the practice of training on easy examples before hard ones echoes the way animals and children are taught.

The move is the same every time. Someone notices a principle in the brain, some fact about what computation is being done or how learning is arranged, lifts that principle out, and rebuilds it from scratch in the machine’s own materials. The principle makes the trip while the biology stays behind, and not once on this list did anyone get anywhere by faithfully reproducing neural tissue.

The pattern holds across all of them. What transferred each time was content from the top level, a unit that weighs evidence, a hierarchy of local features, a prediction-error signal, a buffer of replayed experience, a movable focus of processing, and what never transferred was the implementation underneath. Biology paid off every single time someone abstracted a principle and rebuilt it in the machine’s own terms, and it paid off not once for anyone who tried to copy the wetware directly. That second pattern, the failures, is worth looking at on its own.

Where biology led people into the weeds

The failures are quieter than the successes, because nobody writes triumphant histories of a decade spent on a dead end, but they are just as instructive. The common thread is the mirror image of the successes: trouble shows up whenever people copy the brain too low, at the implementation level, on the theory that the biology simply must be load-bearing.

Consider spiking neural networks. Real neurons communicate in discrete spikes rather than the smooth real numbers that flow through a standard deep network, and for years there has been a strong hunch that this must matter, that the spikes are doing something computationally essential and that networks built from them are the road to real intelligence. The energy argument behind that hunch is genuine, since the brain’s twenty watts shames any datacenter and its spike-based, event-driven style of computation is part of the reason. But on the measure neuro-inspired AI actually cares about, raw capability, spiking networks have mostly failed to pull ahead. Year after year the ordinary rate-based network trained by backpropagation keeps winning, and the spikes start to look more like substrate than principle, more like flapping than like lift. They may yet earn their place on energy grounds when it comes to deployment, but as a theory of why intelligence works they have underdelivered.

Or consider the maximalist version of the same bet, which is to copy the whole thing. Efforts to simulate cortical columns in biophysical detail, or to emulate an entire region of brain neuron by neuron, are serious science and have taught us real things about the tissue. They have not produced anything like competitive general intelligence, and the reason traces straight back to the three levels. Simulating the implementation faithfully does not hand you the algorithm, any more than perfectly simulating every transistor in a calculator hands you the concept of addition. You can hold the implementation in complete detail and still not possess the thing that makes it work.

A smaller version of the same confusion plays out in industry, where the existence proof gets used as a marketing line rather than a scientific argument. A team decides its product ought to be brain-inspired, and the inspiration arrives as a thin coat of paint, the word neuromorphic in a pitch deck, a slide with neurons drawn on it, an assumption that because the brain does something, doing it the brain’s way must be better. Usually the question that actually decides whether the product works is a plain engineering question with a plain engineering answer, and the biology is decoration. The opposite mistake happens too, when a team waves away an approach on the grounds that the brain does not work that way, as though that settled anything. Both are the same error in miniature, reading a proof of possibility as an instruction manual. The useful question is never whether something matches the brain, but whether the brain’s way of doing it is the part that transfers or merely the part that was convenient for biology.

The most revealing dead end of all is the long argument over whether backpropagation is biologically plausible. Backpropagation, which trains essentially every modern network, updates each connection using information about weights elsewhere in the network, in a way that real synapses do not appear to have any access to. The objection was raised decades ago and has a precise technical form, the weight transport problem: the backward pass needs a perfect copy of the forward weights, and biological neurons do not obviously keep one. For a great many researchers this came close to a refutation. If the brain cannot be running backpropagation, the reasoning went, then backpropagation must be the wrong theory, and a great deal of careful and genuinely elegant work went into finding biologically plausible replacements, from schemes that use random feedback weights to predictive coding, target propagation, and equilibrium propagation.

And backpropagation kept winning anyway, which forces a lesson that is easy to state and hard to accept: biological implausibility is not an argument that an algorithm is wrong. At most it is an argument that evolution could not have reached it. Evolution cannot perform global weight transport, cannot make a whole organism differentiable on command, and cannot freeze the animal to run a clean backward pass, and those are limits on evolution’s search, not on the space of correct algorithms. Backpropagation may simply be a good solution that biology could never get to, in the same way the wheel is a good solution that biology, with no way to grow a freely spinning axle, almost never found. The brain’s constraints are not our constraints, and treating its limitations as laws of intelligence is a category error.

The witness was found by a different optimiser

Here is the airplane argument in its rigorous form. The brain is not just any witness; it is the output of one specific optimisation process, evolution, run under one specific and rather alien set of constraints, aimed at an objective that is not the one we have. To copy the brain is to copy the solution to a problem you are not actually trying to solve, and writing the two problems down side by side makes the mismatch plain.

Blog image

Evolution searched for the function that minimises something close to negative reproductive success, averaged over the environments a lineage actually encountered, and it searched only among functions that biochemistry can build and that can grow from a single fertilised cell, all under a punishing set of side constraints: a tight energy budget, robustness to damage and noise, and the demand that every intermediate form along the way be viable, since evolution cannot cross a valley of broken organisms to reach a better peak. It is a wildly constrained search, and ours looks almost nothing like it.

Blog image

We minimise a loss on a dataset we chose, searching over functions that run on silicon and, crucially, that are differentiable so gradient descent can train them, under our own constraints of memory, compute cost, and trainability. The objectives differ, the data differs, and the spaces of allowed solutions barely overlap. The functions that biochemistry can build and grow and learn locally, and the functions that run on silicon and stay globally differentiable, intersect only weakly, and when two optimisation problems have almost disjoint feasible sets there is no reason for their best solutions to land anywhere near each other. The table below makes the gap concrete.

evolution's searchour search
search spacewhatever biochemistry can build and growwhatever runs on silicon and is differentiable
objectivesurvive and reproduce in the natural worldminimise a loss on a chosen dataset
constraintsenergy budget, robustness, must develop from a single cell, must be reachable by small mutationsmemory, compute cost, must be trainable by gradient descent
dataembodied lifetime plus hundreds of millions of years of evolutiona fixed corpus, seen many times
outputa braina model
Blog image

The same task, two different searches. Evolution and gradient descent minimise over almost non-overlapping sets of allowed solutions, under almost non-overlapping constraints. The brain sits at the bottom of evolution’s valley, and there is no reason it should also sit at the bottom of ours.

This is why copying the implementation is suspect in principle, not merely in practice. The implementation is exactly the part of the solution most tightly shaped by evolution’s peculiar constraints, the spikes, the chemistry, the developmental tricks, all of them answers to problems we do not share. The problem statement at the top, predicting the world, finding the objects, securing the reward, is shared, because it comes from the structure of the environment rather than from the substrate. That is the deep reason the successes all worked at the top level and the failures all worked at the bottom, since the top is where the two optimisation problems actually agree.

The brain is a prior, not an algorithm

A second formal idea sharpens what the existence proof is really worth, and it comes from learning theory. There is a theorem, proved in the 1990s, that sounds at first like a death sentence for the whole enterprise: averaged over every possible problem, all learning algorithms perform exactly the same, which is to say no better than random guessing. There is no universal learner. Any algorithm that does well on some problems has to do correspondingly badly on others, because the average is fixed.

Blog image

If that were the whole story, intelligence would be impossible, and it plainly is not, so the resolution has to be that we never actually face all possible problems. We face a tiny, structured, deeply non-random slice of them, the problems thrown up by a physical world full of objects and causes and gravity and other agents. A learner can be excellent on that slice precisely by being hopeless on the random nonsense it will never meet. What makes a learner good, then, is not some general-purpose cleverness, because there is no such thing; it is a set of built-in assumptions that happen to match the structure of the actual world, and the proper name for that set of assumptions is a prior.

Read in that light, the existence proof changes character entirely. The brain is not a witness to a powerful general-purpose algorithm, because the theorem says no such algorithm exists. It is a witness to a very good prior over natural problems, a set of assumptions about how the world is built, accumulated by evolution across hundreds of millions of years and handed to every animal at birth. There is a precise way to say what that prior is for. A system that has to persist in a changing world cannot afford to be surprised too often, because sustained surprise, finding yourself far outside the narrow band of physical states you can survive in, is just a slow synonym for dying, so the deep imperative is to keep your incoming signals close to what your internal model already predicts. The most developed version of this idea casts the brain as a hierarchical prediction machine, perpetually guessing its next input and correcting itself by the error, and it makes perception and action two routes to the same end: either you update the model to match the world, or you act on the world to make it match the model. The prior is the model you start from, and a good prior is the genuinely expensive thing, the part you cannot derive from first principles and cannot pick up for free. So the most valuable thing the brain may be telling us is not how to think but what to assume about the world before any data arrives, which turns it from a blueprint for an algorithm into something closer to an inherited model, a humbler description and a far more useful one.

A child is better evidence than a supercomputer

The cleanest way to make this concrete is the single most underrated number in the whole debate, which is how little data a human actually needs. A large language model trains on something like trillions of words, far more text than any person could read in a thousand lifetimes. A child arrives at a working command of its native language, grammar and meaning and the ability to handle sentences it has never heard, after exposure to tens of millions of words, perhaps a hundred million by the time it is a teenager. That is a gap of four or five orders of magnitude. The child is not seeing more data and learning more slowly; it is seeing drastically less and learning a comparably hard thing, and that gap is the prior made visible.

The theorem already told us that a learner can only be this sample-efficient on a problem if it shows up with assumptions matched to that problem. So the child’s efficiency is direct evidence of an enormous amount of built-in structure, assumptions about language and about the world that the child never had to learn because evolution had already paid for them. Developmental psychology has spent decades cataloguing some of this inheritance, the core knowledge an infant seems to arrive with very early: that objects are solid, persist when they pass out of sight, and travel on continuous paths; that agents have goals; together with early senses of number and of space. None of it is learned from scratch, and these are the priors the brain ships with.

Now the existence-proof problem grows teeth, because here is something the witness clearly possesses that we do not know how to build. We cannot yet write the brain’s priors down, and we have no way to install them in a network. So we compensate in the only way available to us, with sheer scale: if you cannot supply the right assumptions, you supply a thousand times the data and let the model rediscover some of the structure the hard way. That, in large part, is what the bitter lesson is cashing in. It works, and it produces capable systems, but it pays in data and compute for something the brain is handed for free at birth. You can watch the cheaper strategy at work wherever people actually have to decode brains under real conditions, because the brain is a non-stationary source, a moving target whose statistics drift from one session to the next and even from one electrode to its neighbour. A good decoder answers that drift not by relearning from scratch but by carrying the model it built last time forward as the prior for this time, so that a warm-up of as few as ten or twenty labelled trials is often enough to re-anchor it. It also weights each new observation by how far it trusts it, leaning on the incoming signal when the recording is clean and falling back on the prior when the signal is noisy, which is the same move attention makes in the brain. The brain’s sample efficiency is the loudest signal we have that it knows something we do not, and that something is not an algorithm but a prior we have not yet learned to read off the witness.

Convergence, and the reason to look at the brain at all

After all that skepticism, you might fairly ask why anyone should look at the brain at all. If the implementation is a trap, the algorithm is out of reach, and the witness was really solving a different problem, why not ignore biology altogether and let gradient descent and scale do the work? There is one strong answer, and it is the most hopeful idea in the field. It is called universality, and it surfaces the moment you look closely at what trained networks actually learn.

Open up the first layer of almost any vision network trained on natural images and you find oriented edge detectors, small filters that light up for a line at one particular angle. Those same orientation-tuned filters are what early recordings found in the first stages of biological vision more than sixty years ago. They turn up in the retina and the primary visual cortex, in a convolutional network trained on photographs, and increasingly in the early layers of vision transformers, which share almost none of the brain’s architecture. Three very different substrates, shaped by three very different processes, all converge on the same opening move. Researchers who study the insides of these networks have documented the same convergence across many models and named it universality: the same features, and even the same little circuits joining them, keep recurring across architectures and training runs.

Blog image Blog image

The same oriented-edge detector keeps turning up, in the retina and primary visual cortex, in the first layer of a convolutional network, and in a vision transformer. When a feature is universal across substrates, it is telling you about the structure of the data and the task, not about the quirks of any one system.

This is the answer to why the brain is worth studying at all. When a feature is universal, it is neither an accident of wetware nor an accident of silicon; it is forced by the structure of the problem and the data. Natural images are built out of edges and textures and surfaces, so anything that learns to see them well, by any means and on any substrate, will discover edge detectors early, because that is simply what the data demands. Where the brain and our machines converge, then, the brain is telling us something true about the problem rather than something parochial about itself. Convergence becomes a filter. Where the brain and a machine independently arrive at the same solution you have probably found a principle, and where they part ways you are probably looking at substrate. The old skill of telling lift from flapping has a modern, operational form: trust what is universal across substrates, and be suspicious of everything else.

The honest caveat is that this universality is only partial, and it weakens as you climb. The first-layer edge detectors are very nearly universal, the middle layers merely rhyme, and the top layers, where the abstract, task-specific representations live, diverge much more and are far harder to compare, so how deep the universality really goes is still an open and contested question. But even partial universality is enough to rescue the brain from irrelevance, because it means the witness is not pure noise. Some of what it found is forced by the problem, and that forced part is exactly the part worth copying.

The proof started pointing back

Something genuinely new has happened in the past decade, and it deepens the problem in a way the older debates never saw coming. The proof started running backward. For most of the history the brain was the example and the machine was the imitator, copying upward at whatever level it could manage. Now the machines we built by copying the brain’s problem have circled back and become the best scientific models we have of the brain itself.

The cleanest case is the one drawn just above. A deep convolutional network was trained to recognise objects and then asked a question nobody had been able to answer well before: how accurately does it predict the firing of individual neurons in a monkey’s visual cortex? The result was startling. The network, which had never been shown a single neural recording and had been optimised purely to label images, turned out to be the best predictor anyone had of the responses in the higher stages of the brain’s object-recognition pathway. And the better a network was at recognising objects, the better it predicted the brain, with task performance and cortical fidelity climbing together. A system built by copying only the brain’s problem, carrying none of its biology, came back around and explained the brain’s biology better than decades of carefully hand-built models had.

It was not a one-off. The same approach now predicts responses in auditory cortex using networks trained to recognise speech and music. Models trained only to predict the next word turn out to predict the brain activity of people who are reading and listening, closely enough that the correspondence has grown into a research programme of its own, one that treats artificial networks as working hypotheses for how the brain computes and tests them against neural data the way you would test any model. The summary is simple: a learning rule from AI predicted a brain signal, and a vision model from AI became the leading model of visual cortex. The witness and the imitation have begun to constrain each other.

This is the strongest form of the convergence argument, and it cuts against the airplane story in a surprising way. Airplanes never became the best model of birds; aerodynamics did, and a jumbo jet is not how you would study a sparrow. Here, by contrast, the artefact we built by abstracting the principle turned around and became the best available account of the original. That can only happen if the principle we abstracted was a deep one, if object recognition really is the problem the visual pathway is solving and a hierarchy of learned features really is how both systems solve it. The reverse direction is evidence that for sensory processing, at least, we abstracted the right thing. It is the most optimistic moment in the whole argument, and it is worth holding in mind next to the most pessimistic one, which is coming.

The bitter lesson, and the rival view

Set against all of this is a blunt observation that has embarrassed neuro-inspired AI more than once, and honesty requires stating it in its strongest form. A well-known argument distilled seventy years of AI history into a single uncomfortable pattern. Time and again, researchers tried to build in human knowledge, hand-crafted features, linguistic rules, cognitive scaffolding, biological detail, and time and again those systems were beaten, decisively, by general methods that simply threw more computation at the problem in the form of search and learning at scale. It happened in chess, in Go, in speech, in vision, and in language. In every case the elegant, knowledge-rich, brain-inspired system lost to the brute one that learned from data at scale. The lesson stings because the hand-crafted structure always felt like the real insight, and it kept turning out to be a crutch that scale could throw away.

Blog image

The recurring shape of it. Hand-built structure, biological structure included, buys an early lead and then flattens out, while general learning plus scale starts slower and keeps climbing, until the two curves cross. Neuro-inspired AI has found itself on the wrong side of that crossing more than once.

Through the existence-proof lens, this lesson is the claim that the specific structure of the witness is mostly a distraction. The brain’s particular wiring, its modules, its developmental tricks, the very things people keep trying to copy, are exactly the hand-built structure that scale renders unnecessary. What survives is only the most general lesson the brain teaches, that the world is learnable from data, that one learning process applied at scale can produce competence across an enormous range of tasks, and that the smart bet is on learning and computation rather than on cleverness. The two grandest bets in AI today line up precisely along this axis. The large language model is the maximal version of ignoring the implementation, copying the function, prediction, and scaling it until something happens. Whole-brain simulation is the maximal version of treating the implementation as the secret and copying it faithfully. On raw capability, the first is winning so lopsidedly that it is almost unfair.

But this is only a claim about how to get capability, and it concedes the other half of the question without meaning to. Scaling up gradient descent produces systems that work and that nobody understands. It hands you a second existence proof, a working artefact whose inner principles are every bit as opaque as the brain’s. If your goal is a system that performs, the advice is to ignore the brain and scale. If your goal is to understand intelligence, scaling alone simply swaps one mystery for another. And understanding, it turns out, is a third problem altogether, separate from existence and from construction, and far less tractable than either of them.

Twenty watts

One of the resource bounds deserves a moment on its own, because it is the place where the brain still flatly humiliates us. The brain runs on roughly twenty watts, about the draw of a dim light bulb, and on that budget it does everything at once, vision, language, motor control, planning, the entire show, continuously, for decades, while also keeping a body alive. The systems we build to match even a slice of that ability run on megawatts. Training a frontier model pulls the power of a small town for weeks, and even running one to answer a single question burns orders of magnitude more energy per useful thought than the lump of tissue between your ears. The same gap shows up at the opposite end of the scale. A chronic neural implant has to do its sensing and signal processing inside a power budget of only tens to a hundred microwatts, because much more than that cooks the surrounding tissue, and the binding constraint in that work has quietly shifted from whether you can record the signal at all to whether you can record, process, and transmit it within roughly a milliwatt. Part of why the brain wins so lopsidedly is architectural. A conventional computer spends most of its energy shuttling data back and forth between separate memory and processing units, while the brain computes in the same place it stores, with no such commute, and closing that particular gap is exactly what the in-memory and neuromorphic hardware now being built is trying to do.

Set that next to the bitter lesson and the two are in real tension. The lesson says to stop being clever, pay in compute, and let scale win, and on capability it does win. But the brain is standing right there as proof that the same thing can be done on twenty watts, which means the scale-everything strategy, whatever its merits, is enormously far from optimal on the one axis evolution was pushed hardest along. Evolution could not throw a datacenter at the problem; for it, energy was a matter of life and death at every step. So the brain is not only a witness that intelligence is possible, but a witness that it is possible absurdly cheaply, far more cheaply than anything we currently know how to build. That gap is not decoration. It is a standing claim that a route to intelligence many orders of magnitude more efficient than ours exists, because something already took it.

And that loops straight back to priors. A large part of why the brain is so cheap is that it does not learn from scratch on trillions of examples; it arrives pre-loaded by evolution with the structure that lets it learn everything else from almost nothing, and built-in structure is far cheaper than structure brute-forced out of data. The efficiency gap and the data gap are really the same gap wearing two different outfits. The brain knows something we do not, that knowledge is a prior rather than an algorithm, and the prior buys sample efficiency and energy efficiency at the same time. The twenty-watt figure is the proof quietly reminding us, every time we look at it, that we are going about this the expensive way.

We have had a complete brain for forty years

There is a piece of evidence for the third gap that predates the microprocessor and is, if anything, more damning, because it concerns a real nervous system that we have held in full for decades. In the mid-1980s, after years of painstaking work tracing electron-microscope slices by hand, biologists published the complete wiring diagram of a tiny nematode worm: every one of its 302 neurons and roughly every one of the 7,000 connections between them. It was the first complete connectome of any animal, and it remains one of only a handful. People called it the wiring of a mind.

That was forty years ago. We have had the entire wiring diagram of a behaving animal’s nervous system for four decades, and we still cannot predict, from the connectome alone, what the worm will do. We cannot read its behaviour off its wiring. Knowing every neuron and every connection turned out to be necessary and nowhere near sufficient, because the wiring says nothing about the signs and strengths of the connections, the dynamics of each individual cell, or the neuromodulators that wash over the circuit and quietly rewire its function while it runs. The diagram is the map, and the territory is that map plus an enormous amount of state and chemistry the map never shows. A creature with 302 neurons has held out against full understanding for forty years with its connectome sitting in plain view.

This is the cold water for every project built on the belief that the answer is simply more complete data. The dream is seductive and it is everywhere: map the whole brain, simulate every neuron, and understanding will follow. We have already run that experiment, on the worm, at the one scale where it is currently possible, and understanding did not follow. The newer efforts to scale connectomics up, a fruit-fly brain of 140,000 neurons mapped and simulated, a cubic millimetre of mouse cortex, slabs of human tissue, are extraordinary feats and will teach us a great deal. But the worm sits underneath all of them as a warning. Holding the complete witness in full physical detail is not the same as understanding it, and we have four decades of proof.

The third gap: having is not understanding

Everything so far has lived in the space between two problems, existence, which the brain already settled, and construction, which we are still working on. There is a third problem hiding behind both of them, and it is the one that should keep you up at night. Suppose you win. Suppose you get the witness completely in hand, every neuron, every connection, every spike, the full wiring and the full dynamics, perfectly recorded and perfectly simulated. Would you understand it? Would you know how it actually works?

That question has actually been put to the test, and the answer is sobering. Researchers took an object we understand completely, the classic 6502 microprocessor, the chip inside the Atari 2600, the Apple I, and the Commodore 64. It has 3,510 transistors. We have its full connectome, every wire, because people designed it. We can simulate it perfectly, and record the state of every transistor at every instant while it runs three different programs, the games Donkey Kong, Space Invaders, and Pitfall. It is, in other words, the connectomics dream fully realised, with total data, total access, and the ground truth already known. Then the standard toolkit of neuroscience was turned loose on it, the tuning curves, the connectivity analyses, the dimensionality reduction, the lesion studies, to see whether those methods, handed perfect and unlimited data, would recover how the processor actually works.

Blog image

Three problems, not one. The brain settles existence and we are working on construction, but comprehension is separate from both, and the microprocessor shows it can fail even when you have everything: full connectome, perfect data, unlimited recording, and still no understanding.

They did not. The methods produced results that looked meaningful and were nothing of the kind. Lesioning transistors one at a time, the analysis found transistors whose removal broke Donkey Kong but left Space Invaders running, and it is almost irresistible to christen one of those a Donkey Kong transistor, exactly the way we name brain regions after the behaviours their damage disrupts. But it is nonsense. Those transistors are not for Donkey Kong at all; they implement some primitive operation, part of a clock or an adder, that Donkey Kong happens to lean on and Space Invaders happens not to. The analysis picks up a real statistical dependency and draws an entirely wrong picture of the mechanism. The whole apparatus, turned on a system we understand completely, failed to recover the clean modular design that any electrical-engineering student could read straight off the schematic. Unlimited perfect data was not enough for understanding, even for a chip we built on purpose.

Blog image

It is worth sitting with how strange the present moment is, because we have backed into the third gap from the construction side without anyone quite deciding to. When researchers open up a trained network and try to understand it, the obvious move, reading off what each neuron means, fails for the very same reason the microprocessor lesions failed. Individual neurons rarely correspond to single clean concepts. They are polysemantic, firing for unrelated things at once, because the network packs in more features than it has neurons by spreading them across overlapping combinations, a phenomenon now called superposition. To recover anything legible, interpretability researchers have to disentangle those combinations after the fact, and even the recent successes at pulling interpretable features out of a model are painstaking and partial. We built the system, we hold every weight, we can run it on any input as many times as we like, and understanding it remains a research frontier rather than a lookup. That is the microprocessor result reappearing inside the systems we care most about, and it arrived not as a warning we heeded but as a surprise we are still trying to absorb.

That is the third gap, and its symmetry should unsettle anyone who is counting on the connectome to save us. Having all the parameters does not hand you a short, legible explanation. You can possess a system in complete physical detail, whether it is a brain, a microprocessor, or a trained network, and still have no compressed, human-sized account of why it does what it does. And the symmetry runs both ways. A frontier neural network is, more and more, an object we have in full detail, every weight known and every activation open to inspection, that we still cannot fully explain. The entire field of interpretability is an admission that we have built working minds we do not understand, which is exactly the situation we are in with the brain. We have started reproducing the comprehension problem inside our own artefacts before we have solved it in the original.

So what should you actually do with the brain?

Putting it all together, a workable stance emerges, one that takes the existence proof seriously without becoming its prisoner. The brain is a non-constructive witness: it proves the destination exists and says nothing about the route. It was produced by a different optimiser under constraints that are not ours, which makes its implementation the least trustworthy part to copy and its problem statement the most trustworthy. It is best understood not as an algorithm but as a prior, a set of assumptions about a structured world that, by the theorem, you could never have gotten for free. And it is informative exactly to the degree that its solutions are universal, that it converges with independent systems on the same answer.

From that stance a practical rule follows. Take the brain’s problems at the top level, prediction, recognition, reward, control, because those come from the world itself and will carry over. Take the broad architectural principles that show up everywhere, hierarchy, local features, learning from data. Then build your own algorithms and your own implementation in the machine’s native terms, because the brain’s answers at that level are tuned to constraints you do not share. Use the brain as a generator of hypotheses and a source of priors, and as a sanity check for the moments when your system and the brain disagree about something they both have to handle. And let convergence be the filter throughout: where the brain and the machine independently reach the same solution, trust it, and where they diverge, suspect substrate. That is the old skill of telling lift from flapping, in its modern form.

The open questions are the honest part, and none of them are small. We still do not know how much of intelligence is universal across substrates, forced by the problem itself, and how much is specific to the substrate, an artefact of meat or of silicon. We do not know whether there are principles still hidden in the witness that we simply have not learned to abstract yet, some piece of wing-warping we keep mistaking for flapping and throwing away. We do not know whether the brain’s priors can be learned at scale from data, or whether some of them have to be built in the way evolution built them in. And we do not know whether comprehension is even necessary, or whether we will end up building the thing the way evolution did, through an optimisation we cannot read, and simply live alongside a mind we cannot explain.

That last possibility is the one I keep coming back to, because it is the strangest and, increasingly, the most likely. The existence proof guaranteed the destination from the very start. It never promised that we would understand the road, or the thing waiting at the end of it. The likeliest ending is also the most unsettling one: we turn existence into construction the same way evolution did the first time, through an optimisation process whose inner workings we do not grasp, and we come away holding a second working mind that we understand about as well as we understand the first, which is to say hardly at all. That would leave us with two existence proofs and no real comprehension of either. The brain told us the summit was real, and we may well reach it, and stand on top of it, and still not be able to say how we got there or what, exactly, we are standing on.

Sources that shaped this piece: Marr, Vision (1982); McCulloch & Pitts (1943) and Rosenblatt (1958) on the artificial neuron; Hubel & Wiesel (1959, 1962), Fukushima (1980), and LeCun (1989) on the convolutional lineage; Schultz, Dayan & Montague, Science (1997) on dopamine and temporal-difference error; Yamins, DiCarlo et al., PNAS (2014) on networks predicting the ventral stream; Crick (1989) and Lillicrap et al. (2016) on backprop and biological plausibility; Wolpert & Macready (1997) on no free lunch; Olah et al. (2020) on universality and circuits; Sutton (2019) on the bitter lesson; Jonas & Kording, PLOS Computational Biology (2017) on whether a neuroscientist could understand a microprocessor; Friston on the free energy principle and active inference, with the predictive-coding account of cortex, for the brain as a prediction machine that minimises surprise; and Tim de Boer’s Building a Bedroom BCI compilation, for the practitioner’s reality of decoding non-stationary neural signals, online Bayesian calibration and precision weighting, and the power budgets of neural hardware.

Keyboard shortcuts

Navigation

Go to About g then a
Go to Blog g then b
Go to Photography g then p
Go to Work g then w
Scroll to top g then g

Actions

Search /
Keyboard shortcuts Ctrl /

Post Lists

Navigate posts j k
Open selected post Enter

Articles

Jump to nth heading 1 9
Next heading ]
Previous heading [
Previous / Next item
Go back to list Esc
Esc to close