Evaluating Llms Harness

davila7

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

Install

npx skills add davila7/claude-code-templates --skill evaluating-llms-harness

npx skills add davila7/claude-code-templates --skill evaluating-llms-harness -a claude-code -g

npx skills add davila7/claude-code-templates --skill evaluating-llms-harness -a codex -g

gemini skills install https://github.com/davila7/claude-code-templates.git --path cli-tool/components/skills/ai-research/evaluation-lm-evaluation-harness

gh skill install davila7/claude-code-templates evaluating-llms-harness

git clone --depth 1 https://github.com/davila7/claude-code-templates /tmp/davila7__claude-code-templates
mkdir -p ~/.claude/skills ~/.agents/skills
cp -R /tmp/davila7__claude-code-templates/cli-tool/components/skills/ai-research/evaluation-lm-evaluation-harness ~/.claude/skills/evaluating-llms-harness   # Claude Code
cp -R /tmp/davila7__claude-code-templates/cli-tool/components/skills/ai-research/evaluation-lm-evaluation-harness ~/.agents/skills/evaluating-llms-harness   # Codex, Cursor, Gemini CLI, Copilot, OpenCode

Install

Related