致达里奥的一封公开信:如果你是认真的,就公开权重。
An open letter to Dario: if you mean it, open the weights

原始链接: https://jacob.gold/posts/open-letter-to-dario-amodei-about-open-weights/

作者向 Anthropic 公司首席执行官达里奥·阿莫代(Dario Amodei)发起挑战,要求他超越近期提出的“嵌入式评估员”方案,转而倡导一项真正能减缓人工智能发展的政策:**强制要求所有公开发布的人工智能模型必须采用开放权重。** 作者认为,当前的监管提案(如算力阈值和行业协调)容易受到“监管俘获”的影响。这些规则通过制造小型竞争对手难以负担的复杂合规负担,有效地保护了成熟企业,最终不仅没能减缓发展,反而巩固了现有巨头的市场垄断地位。 相比之下,要求任何面向公众的模型必须以开放权重的形式发布,将从根本上改变该行业的经济模式。通过削弱这些模型的专有价值,可以减少驱动快速、失控训练运行的大规模资本投入。作者呼吁阿莫代秉持其一贯的原则与使命感,并指出作为一家公益公司,Anthropic 具有独特的地位来引领这一牺牲。通过呼吁开放权重,阿莫代可以证明其将安全置于利润之上的承诺,并迫使整个行业实现无法被操纵或规避的减速。

Hacker News 最近的一篇帖文《致 Dario 的一封公开信:如果你是认真的,就公开模型权重》引发了关于人工智能安全与监管的两极化讨论。作者认为,由于前沿人工智能的发展依赖于巨额资本,而资本又依赖于专有模型,因此如果美国法律强制要求所有公开的人工智能模型必须“开放权重”,这将导致估值缩水、训练资金减少,从而有效地减缓人工智能的进步。 讨论区中的批评者称该提案逻辑不通且不切实际,认为它忽视了全球竞争,因为非美国实验室将继续推进自己的专有模型,同时还能从泄露的权重中获益。另一些人则认为,此举只会将人工智能开发推向“黑暗”或仅限企业的孤岛,进一步集中权力。 许多评论者将该提案视为一种“减速主义”策略或监管俘获的形式。怀疑论者指出,人工智能实验室的动力源于利润和权力,它们不太可能同意这种被批评者称为“经济自杀”的行为。总而言之,这场讨论反映出人们对人工智能公司首席执行官动机的深切怀疑,以及关于透明度与保密性孰优孰劣以实现人工智能安全问题的广泛分歧。
相关文章

原文

Dear Dario,

You published We Must Pace the Frontier today, which committed Anthropic to embedded third-party evaluators and asked governments to require every other frontier company to match. Sam Altman agreed within hours.

Dario Amodei's face rendered out of glowing zeros and ones

I believe you’re being sincere, so I’m asking you to call for a different law, one that would actually slow things down, even though it would require real sacrifice on your part.

Any AI model a company offers to the public has to be released as open weights.

This wouldn’t cover every model you train, just the ones you make available to the public. Internal and research models wouldn’t be affected, and governments could still get access to unreleased models.

But Anthropic would have to publish Fable’s weights the day it was released, OpenAI would have to do the same with Astra, and so would everyone else.

As you know, funding for frontier model development depends on valuations that assume the weights remain proprietary. By changing that assumption, we can reduce the money available for future training runs and slow progress at every lab at once, without a regulator deciding anything.

There’s no slowing down frontier models by regulation that wouldn’t lead to regulatory capture. Embedded evaluators, compute thresholds, industry coordination with antitrust waivers. All of it gets written with the help of the current frontier labs, because nobody else understands the technical details well enough to write it.

Once it’s law, it only gets more complex, because every incident adds a rule and no rule ever gets removed. The big companies can afford the compliance teams and lawyers to keep up, and each new requirement makes it harder for anyone else to catch them. Instead of slowing you down, it protects your position and profits.

You seem like a good and principled person, and you have a record of giving things up for what you believe. You left OpenAI to start a company with safety built in from the beginning, when the safe career move was to stay. You’ve spent years asking for regulation of your own industry and getting called a doomer and a regulatory capturer for it. You pushed for export controls that shrink your own market. Anthropic paid $1.5 billion to settle with authors rather than fight to the end. And today you committed to slowing down before anyone else did.

You also had the foresight to make Anthropic a PBC (Public Benefit Corporation), so you can legally put the mission first:

The responsible development and maintenance of advanced AI for the long-term benefit of humanity.

This is exactly the kind of mission over short-term profit decision the PBC structure exists for. You remain the only person who could propose a law like this and be taken seriously, and the only one likely to actually do it.

If you really want to slow down frontier AI, call for a law that makes every publicly available model open weights. There’s nothing in it to game or dodge, even for you.

联系我们 contact @ memedata.com