语言模型可以重写自己的权重
Language Models Can Rewrite Their Own Weights

原始链接: https://arxiv.org/abs/2611.21885

这篇论文介绍了一种“自我重写”机制:语言模型通过推理并编辑自身权重来改进自身,无需梯度或重新训练。一个270亿参数的开放模型接收自身权重、评分后的答案以及运行自身的能力;它诊断失败并修改权重,在未见过的问题上准确率提升6.6个百分点。它能够迭代地测试并提交成功的编辑,准确率在各会话中持续提升。它还能修复自身的一个受损副本,在未见过的图像任务上恢复超过80%的原始准确率,且未使用原始权重。该能力仅在足够大的规模下出现。与测试时扩展不同,增益被存储在权重中;与训练不同,仅需前向传播。作者认为这模糊了训练与推理之间的边界,代表了迈向递归自我改进的一步。

Hacker News 上的一篇帖子链接到一篇 arXiv 论文,标题为《语言模型可以重写自己的权重》。该帖有 22 点和三条评论。评论者讨论论文声称自重写可在不使用更大模型、框架或单独训练与推理的情况下实现递归自我改进。一位评论者批评缺乏对齐讨论,并警告无监督自修改机器人。另一位说该领域回避责任。第三位开玩笑说这是“不是我们想要的 RSI,而是我们应得的 RSI。”
相关文章

原文

View a PDF of the paper titled Language Models Can Rewrite Their Own Weights, by Jiajun Fan and 10 other authors

View PDF HTML (experimental)
Abstract:A language model is trained once and frozen. Making it better takes another training run, or a prompt, a harness or extra computation at inference that is gone once the task ends. We show a third way: to our knowledge for the first time, a language model rewrites its own weights by its own reasoning and becomes better, with no gradients and no training. We call this self-rewriting: the model is its own optimizer. In earlier work, a frontier model edited a much smaller one; here an open 27B model, given its weights, its graded answers and the ability to run itself, diagnoses its own failure and rewrites the weights behind it. It surpasses itself: its own edit makes it 6.6 points more accurate on problems it never saw. It keeps getting better: it tests its edits on practice problems and submits only what works, each edit improving on the last, and over eight consecutive sessions its accuracy only rises. It repairs a damaged copy of itself, like a broken robot that fixes itself without being rebuilt: the copy solves almost nothing, yet with no retraining and without ever reading the original weights, the model brings it back to over 80% of the original accuracy on image tasks it never saw. And the ability emerges with scale: in one model family, the smaller models make no edit or break themselves, and only the largest improves and repairs itself. Unlike test-time scaling, self-rewriting writes its gain into the weights, at no cost per answer; unlike training, it runs only forward passes, as serving does. The wall between training and inference is a limit of our tools, not a property of intelligence. We believe self-rewriting is a necessary step toward recursive self-improvement: a model that, like a human, does not separate training from inference could keep upgrading itself for life, with no larger model, no harness or prompt engineering, and no pre- or post-training.
From: Jiajun Fan [view email]
[v1] Mon, 16 Nov 2026 20:55:08 UTC (6,363 KB)
[v2] Wed, 18 Nov 2026 08:57:59 UTC (6,379 KB)
联系我们 contact @ memedata.com