Jesse Vincent (obra) / verification-before-completion
How do you stop an agent saying done when it is not?
By making evidence, not confidence, the condition for finishing.
What to remember
- The cost of a false done is paid later, by someone with less context.
- Define what counts as evidence per task, or you will get whatever is easiest to produce.
- It combines directly with webapp-testing, which is how the evidence gets produced.
- It is a habit, not a guarantee. An agent can still verify the wrong thing carefully.
01What it changes in practice
Agents do not lie so much as conclude early. The code looks right, the reasoning is coherent, and the report of success is generated with the same confidence as everything else. Requiring evidence changes what the report is made of.
It also changes review. Reading actions and observations takes seconds, whereas reconstructing whether a change works from a diff takes far longer and is often skipped.
02Where it disappoints
Verifying the wrong thing is still possible. A screenshot of the wrong page is evidence of nothing, so what counts as evidence has to be stated per task.
It also adds a step to short loops where you would have checked yourself anyway, which is the one case where the overhead is real.
Install and first run
- Install superpowers from obra/superpowers, published under MIT.
- Say what evidence counts for this task, for example a passing test, a screenshot or a request log.
- Reject completion reports that contain conclusions but no observations.
Questions people actually ask
- What counts as evidence?
- Something observed rather than concluded: a test result, a screenshot, a log line, a response body. Say which one you want.
- Does it slow work down?
- Slightly per task, and much less than the rework caused by acting on a false completion report.
- Does it work with other skills?
- Yes. webapp-testing produces the evidence, this skill makes producing it non optional.
Sources
Machine readable versions of this page