The Medium post goes over various flavors of distillation, including response-based distillation, feature-based distillation and relation-based distillation. It also covers two fundamentally different modes of distillation – off-line and online distillation.

OpenAI accuses Chinese AI firm DeepSeek of stealing its content through "knowledge distillation," sparking concerns over security, ethics, and national interests.

David Sacks says OpenAI has evidence that Chinese company DeepSeek used a technique called "distillation" to build a rival model.