Negative Self-Distillation: Learning to Reason by Avoiding Flaws

arXiv · Research Team · 2026-09-13

Negative Self-Distillation teaches LLMs to sharpen their reasoning by learning from what they got wrong.

What it does

The paper introduces Negative Self-Distillation (NegSD), a training regime where a model is penalized for reproducing its own erroneous outputs while being rewarded for correcting them. It constructs a “flaw bank” from chain-of-thought mistakes and uses contrastive loss to push the student away from those patterns. Experiments on GSM-8K, MathQA, and a multi-step reasoning benchmark show up to 12% absolute gain over standard self-distillation. The authors also release the flaw-bank dataset and training scripts.

Why it matters

Self-distillation is popular for compressing models, but it often reinforces existing biases and errors. NegSD flips the script, turning a model’s own failures into a learning signal, which directly improves reasoning fidelity—a critical need for any LLM that must explain or justify its answers. The technique is model-agnostic and can be added to existing fine-tuning pipelines.

How it applies

After you fine-tune a reasoning LLM, run a short NegSD pass on a validation set to harvest its mistakes, then continue training with the contrastive loss. The process adds only ~10% extra compute and can be scripted into CI for model releases. It’s especially useful for agents that generate step-by-step plans or answer complex queries where a single slip can break the whole solution.

Takeaways

• Self-distillation can be harmful if the teacher’s mistakes are not filtered out.
• Negative Self-Distillation uses a contrastive objective to push the model away from its own flawed reasoning paths.
• A modest 10% extra training budget yields double-digit improvements on standard reasoning benchmarks.
• The flaw-bank is reusable across models and domains, acting as a “negative curriculum”.
• NegSD integrates seamlessly with existing fine-tuning pipelines and does not require architectural changes.

Topics: self-distillation, reasoning, training, LLM, curriculum-learning