Measuring verified fixes: precision over volume
The only number we optimize first is how many of Forge's fixes were correct. Here is how we define it, and four quarters of results.
It would be easy to make Forge look productive. Open more pull requests, close more tickets, report a big number. We think the only number that matters is how many of its fixes were correct, so that is the one we measure first.
How we define verified
A fix counts as verified when it passed tests, held through a full canary, and stayed in production for 14 days without being reverted or patched for the same issue. Anything else is counted against us, including fixes a human rejected in review.
| Quarter | Fixes proposed | Approved | Verified after 14 days |
|---|---|---|---|
| Q4 2025 | 612 | 71% | 66% |
| Q1 2026 | 1,044 | 78% | 74% |
| Q2 2026 | 1,530 | 83% | 80% |
| Q3 2026 | 1,912 | 86% | 84% |
Why precision beats volume
Every rejected fix costs a reviewer several minutes and a little trust. When we tried raising the number of proposals by lowering our confidence threshold, approval rates fell faster than the volume grew, and reviewers began skimming. We now hold the threshold steady and only raise it.
The trend in the table comes mostly from context, not from a bigger model. Each quarter Forge knows more about each customer's conventions, ownership and history, and that knowledge is what turns a plausible fix into a correct one.