Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.
Install
npx skills add davila7/claude-code-templates --skill distributed-llm-pretraining-torchtitannpx skills add davila7/claude-code-templates --skill distributed-llm-pretraining-torchtitan -a claude-code -gnpx skills add davila7/claude-code-templates --skill distributed-llm-pretraining-torchtitan -a codex -ggemini skills install https://github.com/davila7/claude-code-templates.git --path cli-tool/components/skills/ai-research/model-architecture-torchtitangh skill install davila7/claude-code-templates distributed-llm-pretraining-torchtitangit clone --depth 1 https://github.com/davila7/claude-code-templates /tmp/davila7__claude-code-templates
mkdir -p ~/.claude/skills ~/.agents/skills
cp -R /tmp/davila7__claude-code-templates/cli-tool/components/skills/ai-research/model-architecture-torchtitan ~/.claude/skills/distributed-llm-pretraining-torchtitan # Claude Code
cp -R /tmp/davila7__claude-code-templates/cli-tool/components/skills/ai-research/model-architecture-torchtitan ~/.agents/skills/distributed-llm-pretraining-torchtitan # Codex, Cursor, Gemini CLI, Copilot, OpenCode