Learning Theory · Chapter 3
A definition is only useful if it constrains something. These six statements are what the definition of a lesson says, or seems to say, about learning in general. They are not equally certain, and each is labelled with how sure we are.
PropositionThe law of novelty
An observation the learner already predicts perfectly teaches nothing. More precisely, a lesson can only come from the part of the surprise that is ignorance, D(q ‖ pθ), never from the part that is entropy.
This is the twentieth reading of a picture book, and it is also television static. One is fully predicted; the other is unpredictable but unlearnable. Both lie outside the region where lessons live. The useful observations sit between them: surprising, but surprising in a way the learner can do something about.
TheoremThe law of the ceiling
Over any sequence, lessons sum to at most the learner’s initial ignorance. For a labelled example, what it can teach about its label is at most the label’s entropy, log2 K bits for K equally likely classes.
The first half is the conservation result of chapter 2. The second is the data processing inequality: whatever a model infers about a label from an example, it cannot know more about the label than the label itself contains. (Cover & Thomas)
The ceiling is low. A child told the names of 256 things learns at most 8 bits per naming. Morris and colleagues measured how much a language model can store at all and found about 3.6 bits per parameter. Labels are a narrow pipe; most of what a learner keeps has to come from somewhere else. (Morris et al.)
PropositionThe law of the channel
A learner only receives lessons through what it is asked to predict. If the training signal is the surprise of some part of the data, then only that part enters LF, and the rest of the observation can teach nothing directly, however much it contains.
This is the most consequential of the six, and the reason the rest of the book looks the way it does. A model trained only to predict text, with images as input, learns from an image only through the words that follow it. A caption is a few dozen bits; the picture has thousands of bits of structure. Whatever in the picture the words do not depend on never enters the training signal.
The law has an important qualification, which chapter 1 already showed. Most raw bits are irrelevant. Predicting every pixel would open the channel to a great deal of noise. What should be predicted is a representation of the picture, not the picture itself. Part II follows the first half of this law, and chapter 24 the second.
ConjectureThe law of transfer
The bits of a lesson split into two kinds: those about what stays the same when the observation changes (the name of a cat, the shape of a character) and those about the particular observation (this cat, this font). Only the first kind transfers to new observations. A learner that keeps the second kind in its weights pays for it on the future.
Part of this is a theorem. Xu and Raginsky’s bound in chapter 1 says the generalization gap is controlled by I(W; S), the bits the weights keep about the particular training set. The rest is a conjecture about which bits those are, and the picture book is our strongest evidence: 3.32 bits on pictures seen, 0.03 on pictures not seen. (Xu & Raginsky)
This law also explains why some bits should be remembered after all, just not in the weights. A child should remember that its own cat is called Mimi, but that fact should not change what it thinks cats are. Chapters 22 and 23 give such bits a separate, fast memory.
ObservationThe law of the prior
How many bits an observation can teach depends on the representation the learner already has. A learner with a better prior extracts larger lessons from the same observation, because more of the observation’s surprise is ignorance it can actually reduce.
On Prime, aligning internal features with a pretrained vision model raised the bits recovered on unseen pictures from 0.60 to 0.73 over the same 1,500 steps, across two seeds and 4,000 test images. Children arrive with months of visual experience before their first word; Frank estimates the language they hear is four to five orders of magnitude less than what large models read. The prior is doing much of the work. (REPA) (Frank)
The last law is the one a curriculum depends on, and it comes in two halves that point in opposite directions.
TheoremOrder, first half: an ideal learner does not care
For an ideal Bayesian learner, the total code length of a set of observations is the same in every order, because it equals the negative log of their joint probability, which does not depend on order. The final state is the same too.
ConjectureOrder, second half: a real learner does
A learner that forgets or updates in fixed-size steps, as every trained network does, is order-sensitive. For such learners, interleaving what must be distinguished leaves a more accurate learner than presenting it in blocks, even when blocks are cheaper to read.
With an ideal learner, the three orders cost exactly 49.5 bits each and end in the same place. Turn on forgetting and they come apart. Blocked order now looks cheapest, 30.7 bits, because once a run of cats has started the next cat is no surprise. But it leaves the learner the most wrong about the world: 0.52 bits of remaining ignorance against 0.01 for interleaving.
Cheap to read and good to have learned are different things, and a learner that forgets confuses them. People do too: in Kornell and Bjork’s study, interleaving painters’ styles helped people classify new paintings, though most participants believed blocking had helped more. Curricula in language models are a cautionary tale in the other direction: the BabyLM challenge found that most attempts to order training data by difficulty did not help. Chapter 26 is our attempt to say which order helps, and why. (Kornell & Bjork) (BabyLM)
Chapter 4 follows the transfer law into the picture-book experiment: what exactly the 3.32 bits on seen pictures were made of, and why almost none of them survived on new ones.