How to run it

Open Claude Code in your repo and type:

/hyperpower:eval

With nothing after it, the first thing it does is list every prompt file in your repo, numbered and grouped by folder, with each one's baseline state:

plugins/hyperpower/agents/
   11  mapper.md                    no baseline
   16  reviewer.md                  no baseline
   17  skeptic.md                   baseline 4.21 on 2026-10-06
plugins/hyperpower/skills/
   26  eval                         no baseline

Select by number (3), range (1-4), name (skeptic), group (agents), or all.

Pick however you like. 17, 16-17, skeptic, agents, a path, or all to walk every file one at a time. Name it up front to skip the list:

/hyperpower:eval skeptic

If exactly one prompt file is modified in your working tree, it uses that one and says so rather than asking. A selection that matches nothing is an error naming what it could not find, never a quietly smaller run.

What it is actually comparing

The suite does not scan your prompts and decide what to test. You name one file, and it runs that file's old text against its new text.

git show HEAD:plugins/hyperpower/skills/review/SKILL.md > /tmp/base.md
cp plugins/hyperpower/skills/review/SKILL.md /tmp/cand.md

../scripts/run_evals.py run \
  --baseline-cmd  'claude -p --tools '' --strict-mcp-config --no-session-persistence --append-system-prompt "$(cat /tmp/base.md)"' \
  --candidate-cmd 'claude -p --tools '' --strict-mcp-config --no-session-persistence --append-system-prompt "$(cat /tmp/cand.md)"'

Read it as an exam:

Piece	Is
cases.jsonl	the questions. Fifteen, the same for both sides.
--baseline-cmd	the prompt as it was before your edit
--candidate-cmd	the prompt as it is now
rubric.md	the marking scheme
SHIP or HOLD	whether the new text beat the old text

One file per run, the one you changed. A repo with a hundred prompts still runs one comparison, because the question is whether this edit helped, not which of your prompts is best.

Both commands are plain shell. They receive a case on stdin and print a response on stdout, so anything that reads one and writes the other can be scored.

The first time

A clean install has no baseline, so there is nothing to beat. Record one:

/hyperpower:eval --baseline

Until you do, scoring exits 3 and says no baseline recorded. That is never treated as a pass.

Each prompt file keeps its own baseline, under .hyperpower/evals/baselines/. A score for the mapper says nothing about the reviewer, so the two never share a file and are never compared against each other. The baseline.json inside the plugin is a shipped template with every score null. The script owns all of them; never edit one by hand.

Record a baseline after a change has shipped, never to clear a HOLD. Re-baselining a failing run deletes the evidence that it failed. And adding a case invalidates whatever you recorded, because scoring refuses to compare across a changed cases file, so re-record after you add one.

What it costs

Fifteen cases, three trials, two prompts being compared. Ninety model responses, roughly thirty minutes, and real money. It always prints the estimate before spending anything, and the dry run writes nothing at all:

cd plugins/hyperpower/evals
../scripts/run_evals.py run --dry-run --baseline-cmd cat --candidate-cmd cat

Passing cat echoes each prompt back instead of calling a model, which checks the wiring for free.

When to run it

Before any change to a prompt, a skill, or an agent file ships. That is the release gate for the harness itself. Editing code, gates or docs does not need one; editing the words an agent reads does.