TokenBench runs each tool on the same real coding tasks with the same model and measures the whole bill, scored by hidden tests.
Cost change versus baseline, cheaper first. If the 95% confidence range (see Charts) includes 0%, the difference could be noise.
Success per dollar: the share of runs that pass their hidden tests, divided by cost per task. Higher is better.
I started as a coder on a tight budget, and promises of token saving were attractive. In practice the claims rarely stacked up: each tool published its own results, measured its own way. There are established benchmarks for intelligence, but none for efficiency.
One model, one set of tasks. Your repositories and workflow may behave differently, so validate any tool on your own work.
The short version, if you'd rather not read the tables.