一句话
代码自我改进的可信证据来自可执行验证、跨代保留、失败候选和迁移检查，而不是单个 benchmark 分数。
Benchmark / Survey Rank 3
代码自我改进，不是一张榜单能说明白。
Code evolution claims are easy to blur. DGM modifies an agent, AlphaEvolve discovers executable algorithms, ADAS searches agent architectures, and GEPA/DSPy optimize prompts or programs. Separate the mutable object before comparing scores.
Object Evaluator Selection Archive Transfer
One sentence
先问“它改了什么”，再问“分数涨了多少”
三句话
DGM/SICA 证明的是 agent 自身代码或工具链能否变好。AlphaEvolve/OpenEvolve/FunSearch 证明的是候选程序能否在强 evaluator 下被发现和保留。ADAS/A-Evolve 证明的是 agent 架构和 workspace 是否能被搜索。
边界
本页只服务读者判断证据强弱：哪些结果可复跑，哪些只有公开分数，哪些仍依赖自评或单一 benchmark。
Benchmark matrix
五类 code-evolution 证据，不要混成一个标签
Evidence boundary
表格中的系统和 benchmark 是该 mode 的代表证据，不表示每个系统都报告了同一组 benchmark、隐藏测试或回归检查。做技术判断时必须回到单项目报告和原论文。
| Mode | 可变对象 | 代表系统 | 强证据 | 主要风险 |
|---|---|---|---|---|
| 自修改 coding agent Self-modifying coding agent | agent code, tool policy, patch loop, review logic | DGM, SICA, A-Evolve | SWE-Bench / SWE-Bench Verified, Polyglot, repository regression tests | benchmark-specific patch, hidden regressions, unsafe self-modification |
| 算法与程序发现 Algorithm and program discovery | candidate programs, kernels, heuristics, optimization code | AlphaEvolve, FunSearch, OpenEvolve, Science-CodeEvolve, OpenTreeSearch | matrix multiplication, Borg scheduling, FlashAttention, attention kernels, circle packing | narrow metric optimization, weak maintainability, incomplete constraints |
| Agent 架构搜索 Agent architecture search | Python-coded agent design, tool topology, control flow | ADAS, Agent Symbolic Learning, A-Evolve | ARC, GFootball, cross-task transfer, cross-model transfer | expensive evaluation, weak transfer, generated-agent safety gaps |
| Prompt / program 优化器 Prompt and program optimizer | prompt, symbolic module, program graph, optimizer state | OPRO, DSPy, GEPA, EvoPrompting | task validation sets, HumanEval-style tests, QA/reasoning benchmarks | prompt overfitting, judge bias, unstable gains |
| 反思与修复回路 Reflection and repair loop | reflection memory, tests, generated critiques, retry strategy | Reflexion, SelfEvolve, ReVeal, EvoMAC | HumanEval, MBPP, LiveCodeBench, generated-unit-test audits | false-positive tests, memory pollution, context growth |
Evidence modes
逐类看：同样是代码进化，证据焦点不同
Mode 1 / Self-modifying coding agent
自修改 coding agent
它是在提高自己的软件工程能力，还是只解了一道题?
- 可变对象：agent code, tool policy, patch loop, review logic
- 代表系统：DGM, SICA, A-Evolve
- 强证据：SWE-Bench / SWE-Bench Verified, Polyglot, repository regression tests
- 主要风险：benchmark-specific patch, hidden regressions, unsafe self-modification
Mode 2 / Algorithm and program discovery
算法与程序发现
是否有可执行 evaluator 拒绝漂亮但错误的程序?
- 可变对象：candidate programs, kernels, heuristics, optimization code
- 代表系统：AlphaEvolve, FunSearch, OpenEvolve, Science-CodeEvolve, OpenTreeSearch
- 强证据：matrix multiplication, Borg scheduling, FlashAttention, attention kernels, circle packing
- 主要风险：narrow metric optimization, weak maintainability, incomplete constraints
Mode 3 / Agent architecture search
Agent 架构搜索
搜索到的是可迁移架构，还是一个 benchmark prompt trick?
- 可变对象：Python-coded agent design, tool topology, control flow
- 代表系统：ADAS, Agent Symbolic Learning, A-Evolve
- 强证据：ARC, GFootball, cross-task transfer, cross-model transfer
- 主要风险：expensive evaluation, weak transfer, generated-agent safety gaps
Mode 4 / Prompt and program optimizer
Prompt / program 优化器
优化产物是否可复用，还是只贴合验证集?
- 可变对象：prompt, symbolic module, program graph, optimizer state
- 代表系统：OPRO, DSPy, GEPA, EvoPrompting
- 强证据：task validation sets, HumanEval-style tests, QA/reasoning benchmarks
- 主要风险：prompt overfitting, judge bias, unstable gains
Mode 5 / Reflection and repair loop
反思与修复回路
失败是否真的转成了下一次更好的尝试?
- 可变对象：reflection memory, tests, generated critiques, retry strategy
- 代表系统：Reflexion, SelfEvolve, ReVeal, EvoMAC
- 强证据：HumanEval, MBPP, LiveCodeBench, generated-unit-test audits
- 主要风险：false-positive tests, memory pollution, context growth
Evidence ladder
从 demo 到强证据的五级台阶
| Level | 证明什么 | 可接受证据 | 弱证据 |
|---|---|---|---|
| 1 | 能生成代码 | HumanEval, MBPP, unit tests | demo-only snippets |
| 2 | 能修复真实软件任务 | SWE-Bench, repository tests | single issue without held-out checks |
| 3 | 能在可执行约束下发现算法 | programmatic evaluator, reproducible scorer | LLM-as-judge only |
| 4 | 能跨代保留改进 | archive, lineage, best-so-far curve | final score only |
| 5 | 能泛化并控制风险 | hidden tests, transfer, cost and regression report | one public benchmark |
How to read
读一个 claim 时按这个顺序拆
先看可变对象
代码补丁、候选程序、agent 架构、prompt、memory 不是同一种自进化。先问系统到底改了什么。
再看评估器
更可靠的信号来自可执行测试、隐藏任务、程序化 scorer 和独立评估。只有 LLM 自评时要降低可信度。
最后看保留机制
严格自进化证据会留下 archive、lineage、best-so-far 曲线、失败候选或可回滚资产。
不要只看分数
分数提升必须同时报告成本、迁移、回归和失败模式，否则容易把 benchmark 热度当成熟度。
Evidence links
证据入口
Repository evidence
DGM paper note
打开仓库中的 processed evidence，继续检查来源、方法和限制。
Repository evidence
ADAS paper note
打开仓库中的 processed evidence，继续检查来源、方法和限制。
Repository evidence
AlphaEvolve paper note
打开仓库中的 processed evidence，继续检查来源、方法和限制。
Repository evidence
Processed matrix
打开仓库中的 processed evidence，继续检查来源、方法和限制。
FAQ
四个常见误读
Is AlphaEvolve the same kind of evidence as DGM?
No. AlphaEvolve is strongest for executable algorithm discovery; DGM is strongest for self-modifying coding agents and archive-based agent improvement.
Why is SWE-Bench not enough by itself?
SWE-Bench is valuable, but self-evolution also needs hidden tests, transfer, cost, regression, rejected candidates, and lineage evidence.
Can prompt optimization count as code evolution?
Only when the optimized artifact changes a reusable program or symbolic module and survives independent validation. Otherwise it is prompt tuning, not code self-improvement.
What should readers distrust first?
Distrust claims where the same model proposes, judges, stores, and celebrates the change without executable checks or external validation.