A curated collection of papers, technical reports, frameworks, and tools for on-policy distillation of large language models
-
Updated
May 14, 2026
A curated collection of papers, technical reports, frameworks, and tools for on-policy distillation of large language models
Fine-tuning pipeline to distill floop's LLM consolidator into a small local GGUF model
我的心跳在此同频,我的灵魂在此共振,我,在此安放。
On-device text-to-SQL distilled from GPT-4o-mini into Qwen2.5 (0.5B → 3B locally on M1 via mlx-lm LoRA, 7B+ on cloud A100). 847 MB at 62.5% on Spider dev; 3B variant hits 72.6%, 7B variant hits 75.0%.
AutoReason tournament refinement for LLM distillation — higher-quality teacher labels via blind-Borda self-judging
Add a description, image, and links to the llm-distillation topic page so that developers can more easily learn about it.
To associate your repository with the llm-distillation topic, visit your repo's landing page and select "manage topics."