An agent-improvement run can spend its budget in two ways: waiting for work to finish and paying to find out which change helped. Two October 27 preprints tackle those costs from different directions.
The GEPA batching study changes the queue. GEPA proposes edits to an agent’s prompts or code and evaluates them. The extension proposes several candidates together and runs their evaluations concurrently. Most batched settings finished in half the elapsed time or less across two tasks. The best reported test scores rose from 68.9% to 72.1% on LiveBench-Math and from 49.0% to 60.0% on HoVer, using different batch configurations.
Each setting received the same task-specific budget of evaluation calls. That makes the result more useful than a comparison that simply buys more trials. But extra parallelism has limits: slow calls and a finite worker pool still determine when a batch finishes. These are benchmark results, rather than a promised speedup for an enterprise workflow.
The EvalAlloc study changes where evaluation effort goes. It fully evaluates a short opening set, then uses shared informative tasks and a fixed scoring reference to guide further search. Afterward, it concentrates additional evaluations on choosing among candidates, reusing observations already collected.
Across four benchmarks, two evolution frameworks and three runs per setting, EvalAlloc reports comparable or higher mean test performance with roughly half as many evaluation cells. A cell is one candidate executed on one task. That count does not translate directly into half the bill or half the elapsed time, because executions can have different costs and durations. The study’s test set remains separate from search and selection.
These methods invite two distinct experiments. If workers sit idle while proposals run in sequence, test batching under a fixed budget. If testing every candidate dominates spending, test selective evaluation while preserving a common comparison basis. Neither paper establishes the gain from combining them.
Before a run starts, reserve the final test set and name the stopping rule. Record elapsed time, evaluation count, spend and the chosen candidate’s test result separately. Otherwise, a faster search can conceal a more expensive decision.