Pytorch Fsdp2

Orchestra Research

Adds PyTorch FSDP2 (fully_shard) to training scripts with correct init, sharding, mixed precision/offload config, and distributed checkpointing. Use when models exceed single-GPU memory or when you need DTensor-based sharding with DeviceMesh.

Install

npx skills add Orchestra-Research/AI-Research-SKILLs --skill pytorch-fsdp2

npx skills add Orchestra-Research/AI-Research-SKILLs --skill pytorch-fsdp2 -a claude-code -g

npx skills add Orchestra-Research/AI-Research-SKILLs --skill pytorch-fsdp2 -a codex -g

gemini skills install https://github.com/Orchestra-Research/AI-Research-SKILLs.git --path 08-distributed-training/pytorch-fsdp2

gh skill install Orchestra-Research/AI-Research-SKILLs pytorch-fsdp2

git clone --depth 1 https://github.com/Orchestra-Research/AI-Research-SKILLs /tmp/Orchestra-Research__AI-Research-SKILLs
mkdir -p ~/.claude/skills ~/.agents/skills
cp -R /tmp/Orchestra-Research__AI-Research-SKILLs/08-distributed-training/pytorch-fsdp2 ~/.claude/skills/pytorch-fsdp2   # Claude Code
cp -R /tmp/Orchestra-Research__AI-Research-SKILLs/08-distributed-training/pytorch-fsdp2 ~/.agents/skills/pytorch-fsdp2   # Codex, Cursor, Gemini CLI, Copilot, OpenCode

Install

Related