Optimize Apache Spark jobs with partitioning, caching, shuffle optimization, and memory tuning. Use when improving Spark performance, debugging slow jobs, or scaling data processing pipelines.
Install
npx skills add wshobson/agents --skill spark-optimizationnpx skills add wshobson/agents --skill spark-optimization -a claude-code -gnpx skills add wshobson/agents --skill spark-optimization -a codex -ggemini skills install https://github.com/wshobson/agents.git --path plugins/data-engineering/skills/spark-optimizationgh skill install wshobson/agents spark-optimizationgit clone --depth 1 https://github.com/wshobson/agents /tmp/wshobson__agents
mkdir -p ~/.claude/skills ~/.agents/skills
cp -R /tmp/wshobson__agents/plugins/data-engineering/skills/spark-optimization ~/.claude/skills/spark-optimization # Claude Code
cp -R /tmp/wshobson__agents/plugins/data-engineering/skills/spark-optimization ~/.agents/skills/spark-optimization # Codex, Cursor, Gemini CLI, Copilot, OpenCode