Definition
Model distillation trains a student model using a teacher’s outputs or behavior to transfer some of the teacher’s capabilities.
Usage and related concepts
It can support compression or capability transfer. Unauthorized output collection and credential theft are different issues; fraudulent API usage alone does not establish distillation. “Distilling experience” is a metaphor, not this training procedure.