Every experiment
Grouped by what each one does to the music: take an idea out, put one in, scramble the structure, look inside the weights, or push the whole thing into metal.
Deletion
The blue note you can't delete
Erase every blue note from the training data. The model still expects it, exactly where a soloist would bend one.
97–98% rebuiltClassicalRemoving the dominant seventh
Edit the seventh out of every dominant chord in the corpus. The model reconstructs it from the harmony around the hole.
~90% rebuiltClassicalHow deep you have to cut
One degree comes back whole. Strip a whole scale's worth at once and reconstruction finally erodes.
down to 71%MetalThe tritone you can't delete
Metal has none of the functional harmony we thought did the rebuilding. Delete the b5 anyway — it comes back at 99%.
99% rebuiltMetalCutting a genre from the family tree
Delete a whole genre. Black metal rebuilds from its thrash siblings; djent's novel sub-bass is the one thing that won't.
shared rebuilds, novel deletesInjection
Teaching classical the blues
The inverse of deletion: edit blue notes in. It works — but only at dose. Quantity beat quality outright.
5× at a 2% doseClassicalAn invented music theory
A made-up rule injected into the corpus. A uniform habit sticks; the same rule made chord-conditional barely learns.
habit yes, rule 3× weakerStructure
Scrambling tonal function
Permute the twelve scale degrees, train, then un-permute the output. Function transplants onto new intervals.
un-scrambles to 0.80JazzWhen lower loss means nothing
Swap each melody note for its chord. The loss falls to a third — and none of it is understanding. It's redundancy.
8× cheaper, 0× smarterLooking inside
Reading the concept off the weights
Linear probes find the 'we're in a dominant' direction. Steering doubles the sevenths. A sparse autoencoder hands back the dictionary.
0.86 probe · 2× steerInterpretabilityThe structure we caught ourselves on
The model lays the twelve keys out in a circle of fifths. Tempting to call that discovery. We ran the control — it's the data.
data, not discoveryInterpretabilityWhat a loss actually measures
Cross-entropy is conditional entropy, in nats. Rank composers by it, and measure how much a concept really adds.
blue note = 90% redundantMetal
A model that only knows metal
22,037 guitar tracks, 163M tokens, a from-scratch model — and an honest look at what its very low loss is really made of.
val 0.23, mostly redundancyMetalForcing a subgenre
Fine-tune the base metal model on tech-death, then black metal. Two twelve-second edits, two measurably different subgenres.
chromatic vs tremoloMetalThe whole band
Guitar, bass, and drums in one model — 316M tokens, arrangements composed together instead of a lone riff.
val 0.18