Compress large language models using knowledge distillation from teacher to student models. Use when deploying smaller models with retained performance, transferring GPT-4 capabilities to open-source models, or reducing inference costs. Covers temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training strategies.
Install
npx skills add davila7/claude-code-templates --skill knowledge-distillationnpx skills add davila7/claude-code-templates --skill knowledge-distillation -a claude-code -gnpx skills add davila7/claude-code-templates --skill knowledge-distillation -a codex -ggemini skills install https://github.com/davila7/claude-code-templates.git --path cli-tool/components/skills/ai-research/emerging-techniques-knowledge-distillationgh skill install davila7/claude-code-templates knowledge-distillationgit clone --depth 1 https://github.com/davila7/claude-code-templates /tmp/davila7__claude-code-templates
mkdir -p ~/.claude/skills ~/.agents/skills
cp -R /tmp/davila7__claude-code-templates/cli-tool/components/skills/ai-research/emerging-techniques-knowledge-distillation ~/.claude/skills/knowledge-distillation # Claude Code
cp -R /tmp/davila7__claude-code-templates/cli-tool/components/skills/ai-research/emerging-techniques-knowledge-distillation ~/.agents/skills/knowledge-distillation # Codex, Cursor, Gemini CLI, Copilot, OpenCode