Repository navigation
Question about parallel evaluation #435
Description
I have one question. When I run Qwen3-8B with OpenEvolve on the Erdős problem, the performance varies depending on the parallel evaluation configuration. For example, when I set the parallel evaluation to 1, it shows 0.3810, but when I set it to 16, it shows 0.495 (which is worse).
In the AlgoTune task, your team already demonstrated that parallel evaluation performs much better than sequential evaluation. In my case, however, I am trying to understand why the opposite trend appears. Do you have any comments on this?
Activity
codelion commentedon Mar 10, 2026
When
parallel_evaluationsis too high, OpenEvolve can't build on top of successful ideas fast enough. New candidates get generated before recent improvements make it into the database, so exploration stays shallow.The AlgoTune blog showed serial (1) was catastrophically bad vs parallel (16), but that doesn't mean more parallelism is always better. For most examples we run with
parallel_evaluations: 4andnum_islands: 4-5as the recommended default. That balance gives enough throughput without starving the feedback loop.For the Erdős problem with Qwen3-8B, may be the mathematical refinements need that tighter feedback loop. Try
parallel_evaluations: 4as a middle ground between your two data points.passing2961 commentedon Mar 11, 2026
@codelion Thanks for your kind response. By the way, under the
parallel_evaluations: 1setting on the Erdős problem, Qwen3-8B achieves 0.381082 while GPT-5.3-Codex achieves 0.381394. Intuitively, I would have expected GPT-5.3-Codex to perform better. Have you also observed cases where Qwen3-8B performs surprisingly well on the Erdős problem or other tasks?codelion commentedon Mar 19, 2026
On individual tasks there is variance in model responses. May have to test on a number of examples to say if a model is really better than the other.