What a small model knows that a large one forgets

When we talk about models, we usually talk about size: more parameters, more data, more of everything. It is worth asking the opposite question — what a smaller system makes easier to see.

The question

A large model is good at being right in ways that are hard to inspect. A small one is wrong more often, but its mistakes tend to be legible — you can usually trace them back to something it didn’t see, or something it saw too much of.

That legibility is a kind of usefulness. It turns the model from an oracle into something closer to a colleague: limited, a little predictable, and easier to argue with.

“A smaller model can’t hide much — which is exactly what makes it useful to think with.”

What scale hides

Scale smooths things over. Edges that were once visible — where knowledge stops, where a pattern was memorised rather than understood — get blended into fluent text. That fluency is valuable, and it is also a little like fog.

None of this argues against large systems. It argues for keeping a small one nearby, the way a writer keeps a pencil next to a word processor.

A small experiment

The simplest version of the idea fits in a few lines: ask both models the same thing and keep only the places where they disagree. The disagreements are where the reading starts.

compare.pyPythondef compare(small, large, prompts):
    """Keep only the disagreements."""
    notes = []
    for p in prompts:
        a, b = small(p), large(p)
        if a != b:
            notes.append((p, a, b))
    return notes

The function is deliberately plain. What matters is the habit it encodes: compare() throws away agreement and keeps the interesting part.

Where this leaves us

Where this leaves us is with a habit rather than a conclusion: look at the small thing first, then the large one, and pay attention to the gap. That is the sort of note this journal is for — unfinished, specific, and written slowly.