I benchmarked Claude on my own Rust commits to pick a daily driver

I built code-bench, a Rust coding benchmark, to pick a Claude model. It takes a real commit from one of my repos, starts the model at the commit before, gives it the bug report the way I’d write it, and grades it with tests it never sees. On four hard Rust tasks, Opus 5.5 at high effort passed 5 of 8 runs and Sonnet 5.5 at high passed none.

How a task works

• The prompt is a symptom report, 40 to 150 words, with no rule or file name in it.
• A task only stays if Sonnet at low effort fails it twice.
• A second agent solves each task blind, to check the tests only ask for what the prompt says.

Four Rust tasks made the cut, with reference changes of 428 to about 3,000 lines.

44 runs on the hard tasks

Each task has 10 checks. The score counts only the 18 that at least one run missed.

• Opus is only worth it at high. 5 of 8 passes, against 2 of 8 at medium and 0 of 8 for Sonnet high.
• Effort buys score for Sonnet and Opus. Sonnet passed 11%, 64% and 83% of the checks at medium, high and xhigh.
• Fable low is the fastest one that passes tasks. 3 of 8 in 8.4 minutes, against 22.5 for Opus high.
• All 32 failed runs said they were done.

What I run now

• Sonnet 5.5 high as my daily driver.
• Opus 5.5 high for the hard pieces, as a sub-agent so I’m not waiting on it.
• Fable 5.1 low when I can’t wait for Opus.

It’s four tasks and one or two runs each, so the smaller gaps are rough. The tasks, tests and every run’s diff are in the repo.