We measure the frequency at which LLMs reproduce training data when not prompted to do so adversarially, and find that it can happen frequently even on accident.

We find that empirical evaluations of heuristic privacy defenses can be highly misleading, and propose a new evaluation protocol that is reliable and efficient.

We show that the strength of a model’s inductive bias determines whether interpolation of noisy data is harmless or harmful.