Day 1 · 2026-10-01
Moving to GPUs, the picture-book mistake, and a borrowed eye.
Morning
A vLLM server was holding all eight cards; we stopped it. The training code learned to run on GPUs: bf16, activation checkpointing (the fast-weight memory needs a copy of its state at every step, and 32 GB filled up), and caching the memory’s index tables, which cut a forward pass from 1.30 to 0.85 seconds. Prime grew to 165 million parameters.
Noon
Report card · 10 things · chance 10%
Perfect on the twenty pictures, and getting worse on new ones the longer it studied. It was not learning what a cat is; it was learning those twenty cats.
Afternoon
Report card · new pictures · 1,000 each
Showing a different picture almost every time doubled the score. Aligning Prime’s inner features with a pretrained vision model, the way a child arrives with months of seeing before its first word, added four points, and the result held across two seeds. A newer, bigger eye (DINOv3) did not help on 32-pixel pictures. A loss from the coding-rate literature did not help either; it went in the negative results.
Evening
A new curriculum: 256 things instead of 10, 32 voices, 34 fonts, real recordings of people saying words, animal sounds. And the theory behind all of it started being written down as a book.