[aitianshi]

Closes #14

What

nullius standard run --max-usd USD stops a run at a spending cap. Every relay of the job (the units' own, in this process or in their containers, and the benchmark's) books each call in one ledger before sending it (app/nullius/standard/ledger.py): the call's characters over three as input tokens, and as output the limit the call names, else the model's own from its price row, else 4,096 tokens, at the user's prices. A call is sent only while what was spent, what the calls in flight have booked and its own booking stay within the cap. The first that does not fit is refused by the relay itself (HTTP 402, a body and a header that name the relay, recorded refused), and so is every call after it, so a run stops at one point. A reply turns the booking into what the call cost; a provider's error gives it back; a reply with no token count, or cut off, keeps the whole booking as unsettled.

A unit the cap cut off ends as SpendingCapError, an API error outside the agent's rate, never a miss. Units not begun are not run. result.json carries budget: max_usd, spent_usd, unsettled_usd when there is some, stopped, cut_off and not_run. A capped run is refused before it starts when a model it calls has no price.

Not in this change, recorded in #45: the cap in the builder and on a host, shards sharing one cap, a cap per unit, and the job page showing the budget.

Stacked on #39 (the user's price list). This branch also carries main as of 01dc1cf, merged in because that commit changes the relay container's start, where the ledger is mounted: until #39's branch has main too, the diff here shows those three commits beside the cap. The cap itself is the commit eddfe3c and the merge's resolution in container.py, relay.py and their tests.

Touches

backend · docs and site · the standard

Verified

• New app/tests/test_standard_ledger.py (10 tests): a call is sent while spent, in flight and its booking fit; the first that does not fit closes the ledger for every later one; an unknown cost keeps its booking and an error gives it back; a ledger that cannot be read lets no call out; 60 calls at once never book past the cap; four processes booking in one ledger lose nothing; what a booking is (the named limit, the model's own, the default, cache-write input, embeddings, an encoded file not counted).
• test_standard_relay.py: a call that fits is sent and settled at its cost; one that does not is answered 402 by the relay, marked, recorded refused and never reaches the provider, and neither does a smaller one after it; a provider's 429 gives the booking back; a reply without tokens keeps it; a model without a price is not sent; a relay without a cap books nothing; the relay as a process of its own books in the ledger it is given.
• test_standard_run.py: eight units under a cap the fourth call does not fit: three run, one is cut off (SpendingCapError, classified an API error), four are not run, result.json says so and no call went out past the cap; an ample cap changes nothing but the result; no cap, no ledger; a cap without a price for each model, or with a bad amount, is refused before anything is begun.
• test_classify.py: the cut-off unit is infrastructure for the classifier and the live board. test_standard_container.py: the relay's container is given the ledger and its folder alone.
• Backend suite python3 app/tests/parallel.py (1657 tests) and route suite (356 tests), OK, after main was merged in. docs-drift script: 0 findings.
• By hand, in a temporary profile against a stand-in provider with fixed usage (GSM8K, gpt-4o-mini at list price, 0.00024 USD a call, booked 0.0099):
  • ten units one at a time under --max-usd 0.011: five ran, the sixth was cut off, four were not run, 0.0012 spent, as computed by hand;
  • twelve units four at a time with slow replies under 0.03: three calls in flight filled the cap, the fourth was refused, eight units were not run;
  • --backend docker, on the merged code: the containers' relays share the ledger; three ran, one was cut off, two were not run;
  • an ample cap, and the benchmark's own script (Original) with --price: the run ended and the result carries what was spent;
  • a cap of 0, and a model with no price: refused before anything ran.
• Not verified: a real provider (no key was used), a run on a host (the cap is not sent there yet, A spending cap can only be set on the command line: not in the builder, on a host, per unit, or shown on the job page #45), and Docker Desktop, where the lock is shared between this machine's process and a VM's containers.

Risk

• The relay now gives one answer of its own. It does so only for a run given --max-usd; without it no ledger exists and no call is booked.
• A booking is an estimate at list prices, not a bill. A call that costs more than was booked takes the spend past the cap by the difference; the next call is then refused.
• The calls in flight take room: with several units at a time and a cap close to one call's booking, the run stops early with little spent. The refusal says what was spent and what was booked.
• A cut-off unit reads as an API error with SpendingCapError beside it, not as an outcome of its own: a new outcome would reach the palette, the judge's vocabulary and the field guide.
• The lock is flock on a file beside the ledger, as an install is locked. On Docker Desktop a benchmark's own relay (an option's embeddings) runs on the machine while the units' relays run in the VM; whether the lock holds between the two was not tested.
• A capped run on OpenRouter needs prices under the names it sends (--price or an own price): the public list names models as their providers do.

Undone by a revert; a job written with budget in its result stays readable.

[github-actions]

🔴 PR judge · 4 of 8 yes · #46 ↔ #14

Merge risk	🔴 MAJOR	the judge's reading of the diff; app/nullius/execution.py is in the path of every run
Priority	🔴 EXTREME	the deepest path is app/nullius/execution.py

Question	Verdict
Changes do what the issue asks · 2 of 2 criteria	✅ yes
Issue and pull request describe the same change	❔ undecided
Description matches the diff, no more, no less	⚠️ partly
Diff stays in scope	❌ no
Tests cover the change, or Verified says how	✅ yes
Docs follow the change	✅ yes
Risk named in the description	✅ yes
Before and after shown · 0 images · the diff touches what is visible	❌ no

Acceptance criteria, 2 of 2 met

#	Criterion	Verdict
1	A run can be given a spending cap, and no call goes out once the spend plus the calls in flight would pass it.	✅ met (cut)
2	A capped run's result says the cap stopped it, what was spent, and which units did not run.	✅ met (cut)

Checks the script runs itself, 4 of 4 pass

No machine attribution trailer in the commits or the description	✓
New code under app/, docs/, .github/ or .claude/	✓
No Flask import under the backend package	✓
No key-shaped string in the diff	✓

_{some of the diff was cut before the judge read it}