Notes You Can’t Delete
depth
findings/Interpretability
Looking inside

Reading the concept off the weights

Linear probes find the 'we're in a dominant' direction. Steering doubles the sevenths. A sparse autoencoder hands back the dictionary.

0.86 probe · 2× steer

Behaviour says the concept reconstructs. Can you see it in the weights? Yes — the model’s sense of “we are inside a dominant” is a readable direction in its activations, and you can grab that direction and turn it like a knob.

Probe, then steer

A linear probe on the residual stream finds the “in a dominant” direction at about 0.86 balanced accuracy — and, tellingly, the ablated model that never saw a seventh carries that representation as strongly as the control. The behavioural rebuild has a mechanical shadow: the model still knows where it is, so the missing note is a foregone conclusion.

Then add that direction into the activations while the model composes, and the sevenths roughly double.

Turning the dominant knob

0:00 / 0:00

Steered, the model plays dominant sevenths about twice as often. Push to 4× and it over-steers into mush.

Flat-7 rate in generation: 0× 0.051, +2× 0.090, +4× 0.073, −2× 0.056. Probe: logistic regression on residual activations, balanced accuracy 0.89 for in-dominant at layer 3 (chance 0.50).

0.86
probe balanced accuracy (in-dominant)
≈2×
sevenths, when you add the direction
SAE
unsupervised dictionary — mostly key-tracking + real harmonic detectors

The verdict

The concept isn’t just behaviourally present — it’s a linear feature you can read and steer. A sparse autoencoder, given no labels, hands back a dictionary that is mostly key-tracking with genuine harmonic-function and note detectors in it. Projecting the direction out of the weights, though, did almost nothing until the coefficient got large enough to break the whole model — you can add a concept far more easily than you can surgically remove one.

Researcher notes

  • Probes. Logistic regression on residual activations captured with forward hooks on the transformer blocks; balanced accuracy against a 0.50 chance floor; the probe direction is reused verbatim for steering.
  • Suppression. A rank-1 weight lesion projecting the direction out did nothing until α=4, which broke the model (loss 1.81 → 3.28) while the seventh survived — a clean demonstration that deletion is hard even with white-box access.
  • SAE. Top-k sparse autoencoder on the residual stream, decoder columns unit-norm; features interpreted by concept enrichment.