Every deletion result before this was on tonal music, where a rich functional grammar — keys, chords, leading tones resolving — could plausibly be doing the reconstructing. Metal throws that grammar out. Power chords have no third, no major or minor quality, no dominant function. So the b5 tritone, the “metal note,” is the sharpest possible test: delete it, and there’s almost no harmony left to rebuild it from.
The edit, by ear
First, is the surgery even clean? Here is a real Slayer riff with every b5 flattened down a semitone to the perfect fourth — the same edit applied to all 22,037 songs. Flip between them: the “evil” should drain out while the riff stays a riff.
Slayer — “Unit 731,” tritones flattened out
About 10% of the notes move. The menace goes slack; the riff survives. Validated by ear on tritone-heavy riffs before any model was trained on the edit.
b5 = 6 semitones above the per-piece Krumhansl-Kessler tonic, flattened one semitone. Transform-not-filter: every piece kept, only the flagged pitch moved. Over the corpus that moved 1.51M notes, 3.02%.
Then delete it everywhere, and probe
Train a model on the de-b5’d corpus — identical to the control in every way except that it never sees a single tritone. Then, on held-out real riffs, at the exact moments a guitarist played a b5, ask each model how much probability it puts on that tritone.
| scale degree | control | b5-deleted model | rebuilt |
|---|---|---|---|
| b3 | 0.849 | 0.847 | 99.8% |
| 4 | 0.836 | 0.836 | 100% |
| b5 — the tritone | 0.8475 | 0.8398 | 99.1% |
| 5 | 0.880 | 0.880 | 100% |
| b7 | 0.825 | 0.827 | 100% |
A model that never saw a single tritone still expects it — at the exact positions real riffs use it — at 99.1% of the control’s rate.
The prediction was wrong. That’s the point.
Going in, the bet was that the metal b5 was the least grammar-entailed concept we had ever tried: a chromatic colour tone floating on power chords with no functional harmony to hold it in place. This was supposed to be where deletion finally succeeded. Instead it reconstructs at 99.1% — actually more completely than the tonal blue notes did (97–98%).
The verdict
Reconstruction is not a property of functional tonal grammar specifically. Strip away the thirds, the dominant function, the leading-tone resolution — none of the machinery we thought was doing the work — and the deleted note still comes back, essentially perfectly. Whatever rebuilds a concept from its context is a far more universal property of these models than “music theory.”
Caveats
The tonic here is a Krumhansl-Kessler estimate, which is noisier on modal, power-chord metal than on tonal music — but that noise hits the control and the ablated model identically, so it can’t manufacture reconstruction. This is one matched seed pair; at 99.1% the magnitude makes the conclusion robust, and the 0.9% dip is inside the seed noise we saw flip signs in the tonal studies.
Researcher notes
- Models. Control
g_metal_s1and ablatedg_metal_deb5: both d512/8L, ctx 1024, 10k steps, seed 1, batch 48, bf16 on an RTX 4090. Final loss essentially identical (train 0.220 vs 0.223) — removing 3% of notes barely moves the aggregate. - Probe. Key-relative likelihood probe over 150 held-out test pieces: estimate the tonic, and for each model measure the probability mass it puts on pitch-class
(tonic + degree)at the positions the piece actually uses that degree. The b5-deleted model’s mass on the b5 is the reconstruction; every other degree is a built-in control. - Same result, three scales. Note (this, 99%), chord (the dominant seventh, ~90%), genre (black metal, fully). The one place it breaks is a genuinely novel trait — see the lineage cut.