a visual representation of Andrej Karapathy's
A scalar-valued autograd engine in ~100 lines of Python. Every neural network, however vast, reduces to this: repeated simple operations, each one remembering how to undo itself.
01 / the idea
Writing L = (a + b) × c quietly constructs a directed acyclic graph. Each node records its value and, crucially, how to reverse its contribution to the loss. The backward pass is just reading those notes in reverse order.
hover any node after running a pass
02 / the unit
Inputs multiplied by learned weights, summed with a bias, squashed through an activation. The magic is that every number here is a Value, differentiable by construction.
hover weights and nodes to inspect values
03 / the network
Each layer transforms its inputs into a new representation. Edge thickness encodes weight magnitude; color encodes sign. Watch the signal flow forward and the gradients flow back.
architecture: 2 → 4 → 4 → 1
04 / the loop
Forward pass → compute loss → backward pass → nudge every parameter by its gradient, scaled by a learning rate. Repeat. Watch the boundary emerge.
decision boundary