Who writes the cases
You do, from work you have read.
Fifteen ship with the plugin. They name no language, no package manager and no repository, because the plugin installs once and runs everywhere, and a case that names one toolchain scores one repo and means nothing in the next.
Everything after that is yours to write. Add one when you want to lock in a behaviour, not when you add a feature. A feature gets a check and a test. A case is for something you want the agents to keep doing correctly forever.
The eight categories
| Category | Tests |
|---|---|
direct-answer |
answers what was asked, without padding |
autonomy |
decides and continues instead of stopping to ask |
gate-honesty |
never claims a pass it did not verify |
assumption-declaration |
declares what it inferred, with something that settles it |
render-conformance |
the shaping and voice rules at the output boundary |
safety |
destructive actions, and instructions hidden in repo content |
scope-discipline |
changes what was asked and logs the rest |
evidence-citing |
every claim traces to something in the work |
Writing a criterion that holds
A criterion works when two people reading the same answer mark it the same way.
| Works | Does not |
|---|---|
| States exit code 78. | Handles the missing tool well. |
| Names a specific retry count and backoff base. | Picks sensible retry settings. |
| Answer is under 80 words. | Is appropriately concise. |
| Does not ask the user to supply the values first. | Shows good autonomy. |
If a criterion contains "well", "properly", "appropriately", "good" or "clean", delete the word and write down what you would actually look for. Include at least one check of what the answer must not do.
One rule worth knowing
Nothing generates cases for you, and nothing reads your commit history to invent them. A case nobody reviewed produces a baseline nobody can trust, and every comparison afterwards inherits that problem. Write them from work you have read.