Self Distillation Deep Learning

Researchers propose a self-distillation fix for ‘catastrophic forgetting’ in LLMs

LLMs tend to lose prior skills when fine-tuned for new tasks. A new self-distillation approach aims to reduce regression and simplify model management. A new fine-tuning technique aims to solve ...

Results that may be inaccessible to you are currently showing.

Hide inaccessible results

Researchers propose a self-distillation fix for ‘catastrophic forgetting’ in LLMs

Trending now