Show HN：Parsimony —— 一个衡量 AI 模型新增多少代码的开源基准

Show HN：Parsimony —— 一个衡量 AI 模型新增多少代码的开源基准
Show HN: Parsimony – an open-source benchmark for how much code AI models add

Parsimony 是一个衡量 AI 编程智能体中多余代码的基准。与解决率/成本指标不同，它衡量净编程单元：语句、调用、比较、名称、字面量等语法元素。它减去删除的代码（净 = 新增 − 删除），并避免按行计数，因此堆砌代码不会得到奖励。模型按其在已解决任务上新增的平均净单元排名，从低到高；失败尝试也计入平均值。已解决数据来自 SWE-bench Verified，不重新运行测试。补丁会被应用，变更文件会被解析；仅限 Python，排除测试、文档、生成文件。它使用相同的 mini-SWE-agent，不过智能体会影响输出。更小并不总是更好：单元衡量的是规模，而非可读性/设计，而且写更多代码的模型往往能解决更多任务。用户可以比较模型、查看排行榜，或通过示例 JSON 提交结果。代码/数据：github.com/La5u/Parsimony。

Parsimony 是一个开源基准，用于衡量 AI 模型增加了多少代码。现有基准追踪解决率和 token 成本，但不惩罚多余代码，这可能给长期代码库带来负担。Parsimony 分析来自 DeepSWE、SWE-bench Verified 和 SWE-bench Live 的已发布补丁。它不按代码行数，而是统计编码单元——语句、调用、名称、字面量——并减去删除的代码，以衡量净单元数。由于 GPT-6 和 Claude 5.5 缺少公开补丁，作者通过个人订阅在 3–10 个任务上运行了它们；结果以淡化、未排名的圆圈显示。欢迎反馈和模型贡献，尤其是较新模型的公开补丁导出。GitHub：https://github.com/La5u/Parsimony。与这些基准无隶属关系。

Parsimony: which AI coding agent changes the least code?

Existing benchmarks measure solve rate and cost, but don't punish excess code, which makes a codebase harder to work with in the long term. Parsimony measures how much code AI models add.

It takes the patches that benchmarks already publish and counts coding units: syntax elements like statements, calls, comparisons, names and literals. Lines aren't counted, so cramming code isn't rewarded, and deleted code is subtracted: net units = added − removed.

Compare models

Chart

X axis

How to read this

Models are ranked by mean net units added on the tasks they solved, lowest first. "Net units added (mean)" also includes failed attempts. Solved comes from SWE-bench Verified; tests aren't rerun. Measured is how many solved attempts could be measured.

Leaderboard

Task

Method

Each patch is applied to the task's code, and both versions of every changed file are parsed. Nothing is run.
Tests, docs and generated files are left out. Only Python is measured.
Pass/fail comes from SWE-bench's published results; tests aren't rerun.

Limits

Smaller isn't always better: units measure size, not readability or design.
Models that write more code also tend to solve more tasks (Spearman 0.33–0.60 on the boards with 26+ models).
Every model uses the same agent (mini-SWE-agent), but the agent still affects how much code gets written.

Add your own benchmark

Have code-size numbers from another benchmark? Download the example file, fill in your results and open it to get the same chart and table. The file stays in your browser. Format and how to share it.

Open benchmark JSON

Example: check for a negative number

Both return the same result.

def has_negative(values):
    found = False
    for value in values:
        if value < 0:
            found = True
    return found

def has_negative(values):
    return any(value < 0 for value in values)

Show HN：Parsimony —— 一个衡量 AI 模型新增多少代码的开源基准 Show HN: Parsimony – an open-source benchmark for how much code AI models add

Show HN：Parsimony —— 一个衡量 AI 模型新增多少代码的开源基准
Show HN: Parsimony – an open-source benchmark for how much code AI models add