Backpropagation

Plain English. The algorithm every neural network trains with: after the model makes an error, backpropagation works backwards through the network computing how much each weight contributed, and nudges billions of them accordingly. Popularised in 1986; still unreplaced.

Why it moves money. Backpropagation's mechanics quietly dictate the shape of the AI hardware market. Training requires running the network forwards, storing intermediate results, then flowing gradients backwards — which is why training demands far more memory than inference, why GPUs are built around memory bandwidth, and why training and inference are diverging into separate chip markets with separate economics. When analysts talk about the memory wall or HBM scarcity, they are describing constraints backpropagation imposes. It is also a single point of intellectual concentration: the entire industry trains one way, and any commercially viable alternative — several are researched, none deployed at scale — would redraw hardware requirements overnight.

What to watch. Alternatives leaving the lab: forward-only methods and other schemes that promise training without the backward pass's memory bill. Treat any claimed at-scale replacement as extraordinary until a frontier-class model trains with it.

Further reading. Rumelhart, Hinton & Williams, Learning Representations by Back-propagating Errors (Nature, 1986)