A coding agent can fix a bug and keep going. How much of that extra work is useful? Musa Cim, Burak Topcu and Mahmut Taylan Kandemir recorded each distinct patch generated during benchmark runs. They tested the intermediate patches to see when each run first met the benchmark and what happened before the agent stopped.
The preprint covers five models, two agent frameworks and three SWE-bench datasets. Among runs that reached a correct patch, the first source edit already passed in 38% to 82% of cases, depending on the setting. In the authors’ main settings, the median share of steps spent after the first correct patch ranged from 35% to 60%. Most of those steps went to checking work and reading the repository. The authors also found that 1% to 8% of runs that reached a correct patch ended without one. Patch timing and regressions
That finding doesn’t give a live agent a clock for knowing when to quit. The researchers could apply the benchmark tests to patches after the fact. On your repository, the first green check might miss an edge case, a compatibility promise or work that belongs in another file.
Try keeping the first diff that passes your existing acceptance checks. Keep the final diff too. At review, compare what the later changes buy: another verified requirement, a clearer fix or just more code to read. If an agent keeps rewriting work a reviewer would already accept, you have a reason to change when it hands the patch over.