Runs LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.
安装
npx skills add davila7/claude-code-templates --skill llama-cppnpx skills add davila7/claude-code-templates --skill llama-cpp -a claude-code -gnpx skills add davila7/claude-code-templates --skill llama-cpp -a codex -ggemini skills install https://github.com/davila7/claude-code-templates.git --path cli-tool/components/skills/ai-research/inference-serving-llama-cppgh skill install davila7/claude-code-templates llama-cppgit clone --depth 1 https://github.com/davila7/claude-code-templates /tmp/davila7__claude-code-templates
mkdir -p ~/.claude/skills ~/.agents/skills
cp -R /tmp/davila7__claude-code-templates/cli-tool/components/skills/ai-research/inference-serving-llama-cpp ~/.claude/skills/llama-cpp # Claude Code
cp -R /tmp/davila7__claude-code-templates/cli-tool/components/skills/ai-research/inference-serving-llama-cpp ~/.agents/skills/llama-cpp # Codex, Cursor, Gemini CLI, Copilot, OpenCode