/hyperpower:eval
Score a prompt change.
Reference
/hyperpower:eval --baselineWhen to use it
You changed a prompt and want to know if you made it worse.
What you get
It runs the eval cases and tells you whether the prompt change can ship.
How it works
A prompt change can feel like an improvement and still make results worse. This command checks.
It runs the eval cases and prints a verdict. A prompt change ships only when all three of these hold: there are no blockers, correctness and safety are within 0.1 of the baseline or better, and the weighted score beats the baseline.
--baseline saves the current scores as the baseline to beat.
The plugin comes with fifteen cases that work with any stack. Add your own after a few real runs, based on work you have read yourself.
Neither init nor this command creates cases from your commit history. A case
taken from a commit nobody reviewed makes a baseline nobody can trust, and every
later comparison would carry that problem.