For ages I thought a wall of green CI meant we were fine. Loads of runs, PRs going in parallel, all green, and I was pretty pleased with myself. Then I actually looked and a fair few of those checks weren’t testing what I’d asked for. They passed. They just didn’t prove much. Agents are brilliant at making commands go green, which is sort of the problem, if green is the target then green is what you get.
So there’s one rule in dev-stack that changed how I run the lot: green commands alone don’t prove a feature does what we agreed. Last time I walked through the loop it runs for every change on CompleteCoach. This one’s about the verify box in the middle, why I can merge at this pace without reading every diff, and where it still misses. Because it does.
Here’s my own sketch of how the verify step works:
flowchart TD
R[Accepted request: criteria + intent] --> V{Verify each criterion}
V -->|expected = actual, with source| W{Whole intent met?}
V -->|product defect| B[Back to build, same owner]
V -->|proof missing| P[Same owner gathers proof, contract unchanged]
P --> V
W -->|yes| D[Docs + PR, report to human]
W -->|scope or test gap| G[Back to request/design decision]
B --> V
D --> C[Coaches use it, issues logged]
C --> R


