What an eval is

Your code has tests. Change a function, break something, a test goes red.

Prompts have nothing like that. You reword an agent file to make it clearer, and three weeks later the agents are quietly worse at something you never thought to look at. No compiler, no red X, no stack trace. The damage is real and silent.

An eval is the missing red X. It is a fixed set of situations, each with a list of things a good answer must contain, run against your old prompt and your new one so you can see which one does better.

It is not a test of your product. It tests how the harness behaves. Your features are covered by the checks and by your own test suite.