AI code generation is overly verbose; need better metrics for maintainability
Problem
Current benchmarks only measure solve rate and cost, ignoring excess code that makes codebases hard to maintain. Commenters repeatedly express a need for metrics that capture code complexity and readability — such as cyclomatic complexity — and prefer abstract‑syntax‑level measurements that remove formatting and comment noise. They seek clearer indicators to assess the long‑term maintainability of AI‑generated code.
Pain points
- Excessive code verbosity makes codebases hard to maintain
- Lack of metrics to measure code complexity and readability (e.g., cyclomatic complexity)
- Current benchmarks do not penalize redundant code, making long‑term maintainability hard to assess
Questions to verify
- Have you encountered AI‑generated code that is overly verbose or complex in your actual projects?
- Would you be willing to use a metric like Parsimony to evaluate code quality?
- What additional metrics do you think are needed to better assess the maintainability of AI‑generated code?
Evidence
fact
Existing benchmarks measure solve rate and token output/cost, but don't punish excess code, which makes a codebase harder to work with in the long term.
fact
it takes the patches that benchmarks already publish (DeepSWE, SWE-bench Verified, SWE-bench Live) and counts "coding units": syntax elements like statements, calls, names and literals. I didn't use lines of code, so cramming code isn't rewarded. Deleted code is subtracted, so it measures net units.
quote · @Anonymous
On the one hand, I absolutely agree that we should try to measure and reduce verbosity that LLMs are slopping into our codebases.
quote · @Anonymous
Yeah, I love this. We can't manage what we don't measure, and adjusting the mesurement to be on the abstract syntax level removes an entire layer of "mess" that comes from comments and formatting.
quote · @Anonymous
Would love to see cyclomatic complexity as a supporting metric for readability. I increasingly find AI-adjacent and AI-written code suffering from this.
Disclaimer: this card is distilled by AI from community discussions as “pain point evidence”. It is a starting point for need analysis and validation, not a “validated market need”. AI has analytical limits; please open the original discussion above to manually verify before key decisions.