J-Zero: Unified Challenger–Solver–Judge Co-Evolution from Zero Data

We evaluate J-Zero on Qwen3-4B-Base and Qwen3-8B-Base against the base model and two representative zero-data self-play frameworks, R-Zero and G-Zero, across 11 verifiable and 3 unverifiable benchmarks. J-Zero attains the best score on every benchmark group at both scales.

Verifiable domain. J-Zero improves the average performance in verifiable domains by 9.47 and 7.88 points over the corresponding base models (Qwen3-4B-Base and Qwen3-8B-Base, respectively), while outperforming R-Zero by 4.74 and 3.56 points, respectively. Notably, J-Zero surpasses R-Zero even though R-Zero is specifically designed for self-evolution in verifiable domains.

Results across verifiable domains. The Overall score is the mean of the three domain averages. Best results are in bold.

Unverifiable domain. Baselines achieve much smaller gains in the unverifiable domain compared to the verifiable one, and this is where the gap to J-Zero is widest. R-Zero relies on a majority-vote reward that does not extend to open-ended tasks, so its unverifiable average improves by only 3.08 and 2.31 points, roughly half of what it gains on the verifiable side. G-Zero achieves even smaller gains of 1.31 and 2.08 points, which leaves it behind R-Zero and barely above the base model, since G-Zero does not employ the Judge at all. J-Zero improves the average performance in unverifiable domains by 11.23 and 10.18 points, respectively, with the largest gains observed on AlpacaEval 2.0 (6.22 → 28.56 and 12.93 → 33.53), a broad general instruction-following benchmark that covers diverse open-ended tasks while also including tasks in math and factual knowledge.

Results across unverifiable domains. The Overall score is the mean of the three benchmark scores, where the two Arena-Hard subsets are first averaged. Best results are in bold. H.P. and C.W. denote Hard Prompt and Creative Writing, respectively.

Subtask-amplification pairs matters more once the Solver matures. Both components, role-asymmetry and subtask-amplification pairs, contribute to the gains over the fixed-Judge baseline, with subtask amplification providing the larger improvement. Removing subtask-amplification pairs leads to a larger performance drop (\(-1.64\)) than removing role-asymmetry pairs (\(-0.97\)). This is consistent with the design of the two pair types. Role-asymmetry pairs teach the Judge to distinguish responses below the Solver's current capability, whereas subtask-amplification pairs expose it to responses beyond the Solver's one-shot frontier, which is critical for continued improvement. Nevertheless, combining the two sources performs best because they are most reliable at different stages of training (see Section 5.1 of the paper for details).

Ablation results. We disable one component at a time.

Existing LLM self-play methods plateau after only a few iterations: R-Zero and G-Zero peak at iteration 2 and decline thereafter. J-Zero instead improves monotonically through iteration 10 in both domains. The frozen-Judge variant follows J-Zero closely for the first three iterations and then plateaus. The divergence point is informative: it is where the Solver reaches the fixed Judge's own evaluation ceiling, after which the Judge's rewards no longer separate good responses from bad. A co-evolving Judge keeps its evaluation standard above the Solver's current level, so the reward signal stays discriminative as the Solver improves.