Author continuously-running online evaluations in PostHog AI observability, grounded in real failure modes you've identified. Use when the user wants evaluations that automatically score new generations or whole traces going forward — "create an eval to catch X", "continuously check that responses do Y", "turn these failures into evals".
Install
npx skills add PostHog/ai-plugin --skill creating-online-evaluationsnpx skills add PostHog/ai-plugin --skill creating-online-evaluations -a claude-code -gnpx skills add PostHog/ai-plugin --skill creating-online-evaluations -a codex -ggemini skills install https://github.com/PostHog/ai-plugin.git --path skills/creating-online-evaluationsgh skill install PostHog/ai-plugin creating-online-evaluationsgit clone --depth 1 https://github.com/PostHog/ai-plugin /tmp/PostHog__ai-plugin
mkdir -p ~/.claude/skills ~/.agents/skills
cp -R /tmp/PostHog__ai-plugin/skills/creating-online-evaluations ~/.claude/skills/creating-online-evaluations # Claude Code
cp -R /tmp/PostHog__ai-plugin/skills/creating-online-evaluations ~/.agents/skills/creating-online-evaluations # Codex, Cursor, Gemini CLI, Copilot, OpenCode