Switching off redundant parts of a language model cuts how much text it repeats

A preprint on arXiv tests what happens to memorisation when redundant attention heads are switched off in a language model before use. Earlier work suggests these heads act as a store of associations, so more heads may mean more memorised text. Other work shows many heads can be removed with little loss.

In the authors' experiments on GPT-2-XL, a model with 1.5 billion parameters, switching off 729 redundant heads cut the count of memorised sequences by 91.5% on known memorised text. Language-modelling quality on the benchmark improved slightly. No retraining was needed and the training data was left as it was.

Training text that sits inside a model can come back out word for word. That is the route by which personal or copyrighted material leaks. The result was found on one older model, and it reduces recall rather than removing it.

For people who own or license data, the point is that what a model can repeat is a design choice as well as a legal one. Terms that ask how recall is limited are fair questions to put to a builder.