Taskmaxxing > Tokenmaxxing

An AI budget review can start with a familiar spreadsheet. Models down the left, price per million tokens on the right, and a sort button that quietly makes the decision. The number looks rigorous and tells you almost nothing.

Tokens are a means to an end, and the end is a finished task: a merged change, a resolved ticket. Ranking models by token price is like ranking contractors by hourly rate without asking how many hours they take or how often the work comes back wrong.

The sticker lies in both directions

The obvious failure mode is the cheap model that isn’t. A weaker model tends to burn more context getting to the answer and to fail more often. Each failure costs the tokens it consumed plus the human minutes to catch and retry it. A model with half the sticker price that completes a task 60% of the time at double the token volume isn’t cheap. It can come out both more expensive and worse, and I suspect plenty of teams are running exactly that configuration right now without knowing it.

The less obvious failure runs the other way. Frontier models with big reasoning budgets will happily overthink cheap work. Benchmarks that price the task instead of the token now show order-of-magnitude spreads: on Artificial Analysis’s Coding Agent Index, the strongest agents score a handful of points higher while costing 10x to 60x more per task than a near-frontier model that lands just below them for cents. Whether those few points are worth the premium is exactly what your acceptance bar decides. If most of your workload is routine, paying the frontier premium on all of it is tokenmaxxing with extra steps.

Both failures trace back to the same thing: the sticker prices an input, and you buy an outcome. What you want to know is what it costs to get one accepted unit of work out the other side, and no pricing page will tell you that. The per-task benchmarks get closest, but they price the market’s workload. What no index can publish is your number, computed against your task mix and your quality bar.

Tokens are a means to an end. The end is a finished task.

Goodhart’s law, token edition

Goodhart’s law says that when a measure becomes a target, it stops being a good measure. Token pricing is the law running in production, on both sides of the market at once.

The vendor side writes itself: buyers sort by the sticker, so the sticker is what gets optimized. A vendor can make tokens cheaper by making each token worth less, by quantizing harder or shipping a model that needs three attempts where one used to suffice. The price per million falls, the price per outcome rises, and the spreadsheet says you won. Do not expect an email about the devaluation. You notice it in your own numbers, when the cost per accepted task drifts upward across a version bump.

On the buyer side, the incentive points the same way. Put a token-spend dashboard in front of a team and the dashboard becomes the target. Prompts get shrunk below the context the task needs. Everything routes to the cheapest model. The bill falls and everyone celebrates the savings, right up until someone counts what shipped.

What to measure instead

The alternative is cost per accepted task, where accepted means the work cleared your quality bar. The numerator is everything it took to get there: every token across every retry (input, cached context, reasoning, output), the tool calls, the wall-clock time a person spent blocked, the review minutes, and the escalations where a human had to take the task over. A model that produces plausible-but-wrong output is exporting its costs to your senior people, and that export belongs in the number.

Define accepted. This is the hard part, because most teams I’ve worked with have never defined “done” at that level of precision. The definition might end up worth more than the measurement itself.

Count the all-in cost. The tokens are the easy half. The time terms need a dollar conversion, and the honest one is your loaded rate for an engineer’s blocked hour. Without it you cannot compare a slow-but-cheap model against a fast-but-priced one, which is the comparison routing depends on.

Route by task class. The cheap model probably wins on easy work, and the expensive one earns its rate on the hard tail. Public benchmarks show where the line falls for their task mix; your own traces are how you find it for yours. The probe that settles most routing arguments: pull the last twenty tasks from your traces and price out what each one actually cost, review minutes included.

Re-measure on a cadence. Keep your tooling model-agnostic, because the answer changes and a measurement you can’t act on doesn’t help you. Model families turn over fast enough that a routing decision made in March is folklore by June.

The obvious objection is that this regime is its own tokenmaxxing project, a dashboard built to optimize a dashboard. Fair. So keep it cheap: start with the one task class that dominates your spend and sample its traces rather than instrumenting everything. The expensive part is defining accepted, and you had to do that anyway.

The general form

This problem is older than AI. Whenever the unit you’re billed in differs from the unit you value, someone will optimize the billing unit and call it efficiency. We’ve done it before with keystrokes, billable hours, and story points.

Taskmaxxing is the boring discipline of making the number you watch the same as the number you want. It has no leaderboard and it doesn’t demo well. The benchmarks can tell you what the market pays per task. Your number lives in your own traces, and nowhere else.