开源基准 Parsimony 量化 AI 代码冗余：以净语法单元替代行数统计评估模型简洁性

原标题：Show HN: Parsimony – an open-source benchmark for how much code AI models add

68AI 研判分

核心综述

Parsimony 是一个开源基准，旨在解决现有评测忽视代码膨胀的问题。它通过计算“净编码单元”（语句、调用等语法元素）而非行数来衡量 AI 添加的代码量，并扣除删除部分，从而更真实地反映模型生成代码的简洁性与可维护性。该工具基于 SWE-bench Verified 数据，为开发者提供了评估大模型代码效率的新维度。

报道全文原始报道全文

本文目录7 个章节

Existing benchmarks measure solve rate and cost, but don't punish excess code, which makes a codebase harder to work with in the long term. Parsimony measures how much code AI models add.

It takes the patches that benchmarks already publish and counts coding units: syntax elements like statements, calls, comparisons, names and literals. Lines aren't counted, so cramming code isn't rewarded, and deleted code is subtracted: net units = added − removed.

Compare models

Chart Show my runs

X axis Y axis

How to read this

Models are ranked by mean net units added on the tasks they solved, lowest first. "Net units added (mean)" also includes failed attempts. Solved comes from SWE-bench Verified; tests aren't rerun. Measured is how many solved attempts could be measured.

The Artificial Analysis Intelligence Index is an outside score, matched by model and reasoning effort. It doesn't affect the rank.

Faded models were run by me on the first 3–10 DeepSWE Python tasks (OpenAI models with pi, Anthropic models with Claude Code) and graded with each task's own tests. They aren't verified or ranked. GPT-5.6 Sol ran both ways: my run added 0.88× the code of its published run.

Leaderboard

Task

Rank	Model

Data sources

Hashes

Method

Each patch is applied to the task's code, and both versions of every changed file are parsed. Nothing is run.
Tests, docs and generated files are left out. Only Python is measured.
Pass/fail comes from SWE-bench's published results; tests aren't rerun.

Limits

Smaller isn't always better: units measure size, not readability or design.
Models that write more code also tend to solve more tasks (Spearman 0.33–0.60 on the boards with 26+ models).
Every model uses the same agent (mini-SWE-agent), but the agent still affects how much code gets written.

Add your own benchmark

Have code-size numbers from another benchmark? Download the example file, fill in your results and open it to get the same chart and table. The file stays in your browser. Format and how to share it.

Open benchmark JSON

Example: check for a negative number

Both return the same result.

文本

def has_negative(values):
    found = False
    for value in values:
        if value < 0:
            found = True
    return found

文本

def has_negative(values):
    return any(value < 0 for value in values)

作者发帖说明（HN）：

Existing benchmarks measure solve rate and token output/cost, but don't punish excess code, which makes a codebase harder to work with in the long term. I wanted to measure this since it matters most for long-term projects.

Method: it takes the patches that benchmarks already publish (DeepSWE, SWE-bench Verified, SWE-bench Live) and counts "coding units": syntax elements like statements, calls, names and literals. I didn't use lines of code, so cramming code isn't rewarded. Deleted code is subtracted, so it measures net units.

GPT-6 and Claude 5.5 have no public patches yet, so I ran them myself on 3–10 tasks on my own subscriptions (shown as faded circles, not ranked).

Any feedback or model contributions are welcome, especially public patch exports for newer models.

GitHub: https://github.com/La5u/Parsimony

Not affiliated with any of the benchmarks.