The first full day at school

Overnight

Four stages, one after another

8,700 lessons, about eight hours on eight cards. Every spike is a new stage: something new to learn makes it briefly worse.

Morning

Nothing was forgotten

Reading went from zero to 96% in one stage and stayed there through the next. A third of every later lesson was review, and it was enough.

Midday

Checking for cheating

The video score looked too good: 82% at saying what happened. So we checked every test item against the training set. For pictures, voices, sounds and characters the tests were clean: no shared files, and only 6 of 2,048 test photos identical to a training photo, duplicates CIFAR itself contains. The video test was not: every test video used a picture it had already seen, only moving differently. The test was too easy, and a fair one with brand-new pictures is being made.

Afternoon

Opening it up

Pictures are still the weak sense: one new photo in five, out of 256 things. To find out why, we started taking the model apart: the shape of every weight matrix, which loss pulls the gradient where, how features of the same thing cluster layer by layer, compared with what a pretrained eye sees in the same pixels. Its words are well separated: no two of its 724 characters are closer than a cosine of 0.75. The memory gates are open. The gate that reads long-term memory is still shut: no lesson has needed it yet.

Next: richer data (pictures at their real size, varied every time it sees them), a fair video exam, and exams for what it makes.