TokenMaxing

The cheapest model call is the one you never needed. The most valuable model call may be the eighth one, if the first seven discovered why the task was hard.

That tension is TokenMaxing.

TokenMaxing does not mean setting the output limit to its maximum or asking every model to “think longer.” It means allocating inference work across context, attempts, branches, critiques, tools, and verifiers so the next unit of spend has the highest chance of changing the verified outcome.

The governing question is:

If we spend one more call, branch, or thousand tokens here, what new evidence or candidate can it produce, and is that worth more than its full cost?

Test-time compute is a second scaling axis

Training compute shapes model weights. Test-time or inference-time compute is spent after a task arrives. It can fund:

• longer internal or visible reasoning;
• multiple independent candidates;
• sequential revision after feedback;
• tree or beam search;
• retrieval and tools;
• critique, debate, or ranking;
• deterministic and learned verification.

Self-Consistency samples multiple reasoning paths and aggregates answers, reporting improvements across evaluated arithmetic and commonsense benchmarks. Tree of Thoughts explicitly searches over intermediate “thought” units with evaluation and backtracking on three tasks.

These papers establish mechanisms and scoped results. They do not say every task improves monotonically with tokens. A factual question with one current database answer may need a tool call, not ten sampled explanations.

Coverage is not selection

Generate 100 code patches. If one passes all required tests and policy checks, a deterministic verifier can find it. Generate 100 strategic plans without a reliable outcome signal, and the system may simply have 100 persuasive guesses.

Large Language Monkeys, an author preprint, studies repeated sampling over large budgets. It distinguishes coverage, whether any sample is correct, from the ability to select that sample. The paper reports strong scaling in verifiable domains and limits for common selection methods in less verifiable settings.

This produces the TokenMaxing hierarchy:

candidate diversity
  -> correct candidate coverage
  -> verifier discrimination
  -> selected candidate correctness
  -> executed outcome correctness

More generation only moves the first two. Weak verification can erase the gain.

The independent-sample curve is an upper-bound intuition

Assume each attempt independently succeeds with probability p. Probability of at least one success in N attempts is:

P(any success) = 1 - (1-p)^N

At p=.25:

Attempts	Any-success probability
1	25.00%
2	43.75%
4	68.36%
8	89.99%

Real attempts are correlated. Same model, prompt, retrieved evidence, and tool errors produce shared failure. Temperature can diversify wording without diversifying the decision that matters. The formula is an optimistic intuition unless correlation is measured.

Estimate empirical pass@N by task slice and sampling policy. Report confidence intervals and candidate similarity. Use distinct models or prompts only when they produce useful error diversity, not cosmetic variety.

More success probability can still worsen unit economics

Continue the toy example. Each candidate costs $0.010 to generate and $0.002 to verify. Assume the verifier recognizes success perfectly and every candidate runs fully.

N	Task cost	P(any success)	Cost per expected successful task
1	$0.012	.2500	$0.0480
2	$0.024	.4375	$0.0549
4	$0.048	.6836	$0.0702
8	$0.096	.8999	$0.1067

The eight-candidate policy delivers more successes but pays more per expected success. It may still be correct when failure consequence or latency value justifies the difference.

Use value, not accuracy alone:

expected net value(N)
= P(success by N) * value_of_success
  - inference_and_tool_cost(N)
  - latency_value_loss(N)
  - verifier_and_review_cost(N)
  - expected_failure_and_side_effect_loss(N)

The best N changes with success value. A high-value reversible migration deserves more search than a low-value label suggestion.

Parallel attempts and sequential retries are different bets

Parallel attempts buy diversity and latency

Several candidates start from the same initial state. The first verified winner can stop losers. This helps when candidate generation dominates latency and branches are isolated.

Costs:

• all branches may consume tokens before cancellation;
• shared context and model create correlated errors;
• tool fan-out hits rate and capacity limits;
• multiple writes can duplicate effects unless sandboxed;
• verifier becomes a queue.

Sequential retries buy feedback

Attempt two sees attempt one's failure evidence. It can change plan, retrieve new information, or target a failing test. Sequential work has a longer critical path but can reduce correlation.

Costs:

• full context may be replayed;
• vague feedback can cause cosmetic revision;
• repeated state becomes a loop;
• tool effects may be ambiguous after timeout.

Revision buys local correction

Keep the candidate and ask for targeted changes under a verifier signal. This is efficient when the solution is close and failures are localized.

Search buys structural alternatives

Explore different intermediate states and prune with an evaluator. Tree of Thoughts is one research example. Search is valuable when early choices control later feasibility and backtracking is meaningful.

Do not call every additional token “reasoning.” Name what the spend is buying.

Retry math exposes the hidden distribution

Suppose each loop attempt succeeds with probability .55, only for a toy independent model, and maximum attempts are four.

P(success by 4) = 1 - .45^4 = .95899

Expected attempts actually used with early stop are:

E[attempts] = 1 + .45 + .45^2 + .45^3
            = 1.743625

At $0.018 generation/tool cost plus $0.004 verification each:

expected attempt cost = 1.743625 * .022 = $0.03836

Now add context replay, queueing, and terminal failure loss. An average call price hides the long tail of tasks that use all four attempts.

Plot outcome cost distribution:

Attempts used	Probability in toy model	Cumulative success state
1	.55	success
2	.2475	success
3	.1114	success
4 success	.0501	success
4 fail	.0410	terminal failure

The p95 task can cost and wait much more than the median even with a cap.

Context replay turns loops into token multipliers

An agent has 12,000 tokens of stable policy, history, and evidence. Each of four attempts adds 2,000 new tokens.

Naive full replay input is roughly:

attempt 1: 12,000
attempt 2: 14,000
attempt 3: 16,000
attempt 4: 18,000
total:     60,000 input tokens

The task did not “have an 18k prompt.” It transmitted 60k input tokens across the loop. Prefix caching may reduce compute or provider billing under a specific contract, but the runtime still needs memory and state discipline.

Use a stable cached prefix, typed state reducer, retrieved failure artifacts, and minimal delta. Track unique versus replayed tokens by task.

The verifier sets the ceiling

A verifier can be:

• compiler, parser, type checker, or test suite;
• database constraint or transaction result;
• exact answer or formal proof checker;
• evidence entailment check;
• learned reward or process model;
• human review.

Deterministic verifiers are strong within their scope. Passing unit tests does not prove untested behavior. A learned verifier can be exploited or confidently wrong. Majority voting assumes errors are sufficiently independent and the modal answer corresponds to truth.

Measure the verifier confusion matrix against audited outcomes:

true accept, false accept, true reject, false reject

False accepts waste the entire TokenMaxing budget by selecting a bad candidate. False rejects make good candidates look like missing coverage and trigger needless sampling.

When no reliable verifier exists, more samples can increase review burden faster than accuracy. Spend on better evidence or a narrower action contract first.

Difficulty-aware allocation beats one global budget

Scaling LLM Test-Time Compute Optimally studies allocating compute using search or revision as a function of problem difficulty in evaluated reasoning settings. The broad lesson is conditional allocation: the best strategy depends on task and model.

Build an online budget policy:

base budget from consequence and value
adjust for predicted difficulty and verifier strength
reserve for final verification and recovery
spend one increment
observe marginal evidence
continue only if feasible expected value remains positive

Signals include:

• deterministic verifier failure class;
• uncertainty or disagreement, calibrated carefully;
• retrieval coverage;
• novelty relative to prior attempts;
• remaining deadline;
• budget already spent;
• side-effect reversibility;
• model and tool health.

Easy tasks should stop after one verified attempt. Hard but solvable tasks receive targeted revisions or branches. Out-of-capability tasks should escalate rather than burn the maximum.

100 billion tokens a month is a factory, not a typing habit

Consider the provocative forecast: “By 2027 a top developer will spend 100 billion tokens per month.” No primary dataset currently establishes that forecast, so do not print it as fact. Reverse-engineering it is still useful.

Across 22 working days:

100,000,000,000 / 22
= 4.545 billion tokens per working day

Across an eight-hour workday:

4.545B / 8
= 568.2 million tokens per hour
= 9.47 million tokens per minute
= 157,828 tokens per second

One person is not reading, judging, or steering 157,828 new tokens every second. If this scale becomes real, “developer token use” actually means an autonomous fleet executing many tasks, branches, verifiers, simulations, and background checks on that person's behalf.

That changes the denominator. The useful dashboard is not:

tokens consumed by developer

It is:

verified outcomes per owner
automation spend per verified outcome
human minutes per exception
unreviewed mutation exposure
tokens cancelled after a winner

Build an illustrative task mix that reaches the headline:

Work class	Tasks/month	Tokens/task	Monthly tokens
bounded maintenance	20,000	250,000	5.0B
repository changes	12,000	2,000,000	24.0B
parallel evaluation	5,000	8,000,000	40.0B
migration simulations	1,000	20,000,000	20.0B
monitoring and repair	11,000	1,000,000	11.0B
total	49,000	mixed	100.0B

These are invented scenario inputs. They describe roughly 2,227 automated tasks per working day. The person is no longer manually coding each task. They own policies, queues, budgets, validation, escalation, and change approval for a software factory.

The scenario can be economically bad. If parallel evaluation burns 40B tokens but its verifier cannot distinguish good candidates, that spend creates heat and review burden. If 15 percent of branch tokens arrive after a winner or cancellation, 15B tokens are scheduling waste. If mutations outpace validation, throughput increases risk rather than value.

Human attention becomes the scarce resource. A developer can review a bounded number of exceptions and architectural decisions. Scaling tokens without reducing exception rate moves work into a growing human queue.

Let:

49,000 tasks/month
* 8% human escalation
* 18 minutes/escalation
= 70,560 human minutes
= 1,176 human hours/month

One owner cannot supply that. At a 1 percent escalation rate, the same scenario needs 147 human hours per month, close to a full working month before meetings or proactive design.

The hard inference is this: billion-token development is viable only when verification is machine-speed, mutations are bounded and reversible, exception rates fall below human capacity, and outcome value grows faster than inference spend. Otherwise TokenMaxing creates a factory whose loading dock is blocked by approvals.

This scenario also explains why the “1 percent coding, 15 percent steering, 84 percent validation” style of claim must be handled carefully. It may be a useful target operating model, but without observed time studies it is not a measured workforce fact. A team should instrument active coding, policy design, review, exception handling, and verification before publishing its own ratio.

Marginal stopping is the core control

Let Delta P_k be the estimated increase in verified success probability from the next spend increment, V the value of success, and Delta C_k its full cost.

Continue when:

Delta P_k * V > Delta C_k + added failure risk

Stop when:

• the outcome is verified;
• new attempts reproduce the same failure or candidate;
• verifier cannot discriminate candidates;
• remaining paths cannot meet the deadline;
• mutation or cost budget is exhausted;
• authority or evidence is missing;
• marginal expected value is non-positive.

This decision will be estimated imperfectly. Conservative bounds are appropriate when failure loss is asymmetric.

A builder playbook

1. Define the outcome and verifier

Do not scale inference around a preference score when the product needs executable correctness. Measure verifier false accepts and rejects.

2. Instrument the task distribution

Record candidates, similarity, calls, input/output tokens, tools, verifier cost, retries, wall time, queueing, cancellation waste, and terminal outcome.

3. Establish budget curves

Evaluate success, severe failure, latency, and cost at 1, 2, 4, 8, and relevant larger budgets. Slice by difficulty and task type.

4. Compare spend mechanisms

At equal cost, compare larger model, longer reasoning, parallel samples, feedback revision, search, retrieval, and stronger verification. Use the same outcome set.

5. Add adaptive stopping

Stop on verified success, repeated state, low novelty, infeasible deadline, and marginal-value threshold. Reserve enough budget for final verification.

6. Protect the serving system

Apply per-task and per-tenant concurrency, token, tool, and mutation limits. Cancel losers promptly and propagate backpressure.

7. Price verified outcomes

Include the full distribution, not average tokens. Compare cost per verified outcome and value-weighted failure loss across policies.

The decision

TokenMax when the task has multiple plausible solution paths, a verifier can recognize progress, and success value justifies additional inference. Spend on the mechanism that produces new information: diversity, feedback, search, retrieval, or verification.

Do not TokenMax tasks with weak verification, irreversible side effects, shared failure evidence, or no feasible path inside the deadline. Escalate or narrow them.

The goal is not to spend more tokens. It is to make the last useful token obvious and the next useless one impossible to justify.

References

• Self-Consistency Improves Chain of Thought Reasoning in Language Models, Wang et al., ICLR 2023. Samples diverse reasoning paths and aggregates answers.
• Tree of Thoughts: Deliberate Problem Solving with Large Language Models, Yao et al., NeurIPS 2023. Explores evaluated search, lookahead, and backtracking over intermediate states.
• Large Language Monkeys: Scaling Inference Compute with Repeated Sampling, Brown et al., 2024 preprint. Separates sample coverage from selection in verifiable and less-verifiable domains.
• Scaling LLM Test-Time Compute Optimally Can Be More Effective Than Scaling Model Parameters, Snell et al., ICLR 2025. Studies difficulty-dependent allocation across search and revision.

All costs, probabilities, and token traces in the worked examples are illustrative. Their exact calculations and assumptions are retained in the claim ledger.