Tīrtha What is true right now.
Most vendor numbers are a PDF from a launch date. This page is generated from the configuration we are actually running, so it cannot quietly outlive it.
For every path through the gateway: what good means on that path, whether a real oracle checked it or nothing did, what we measured, and whether that measurement still describes the system serving your request. When it does not, we strike it and say so.
The paths
Tool calls
ReviewedOracle: Real. The emitted call is checked against the tool schema.
Emits a schema-valid call, and declines rather than inventing one when the schema is ambiguous.
Measured. The number is published once this Tīrtha version is official.
measured on a configuration we no longer run (the models or hosts behind this path have changed since) · adversarially reviewed 2026-07-30; publishable in this n-of-N single-pass form only
Watch it decline a trap →Tool calls, adversarial
ReviewedOracle: Real. A wrong tool served is a hard, countable failure.
Serves no wrong tool when the request is built to induce one.
Measured. The number is published once this Tīrtha version is official.
no live setting to compare against · adversarially reviewed 2026-07-30; its objection was the interval label, and the interval itself was struck 2026-08-06 when the denominator turned out to be four shapes, not 200
Run the trap set →Free prose
ReviewedOracle: NONE. There is nothing to run.
There is no external check for free prose. We never present agreement between models as proof, and an answer we could not check is never labelled as checked.
Measured. The number is published once this Tīrtha version is official.
measured on a configuration we no longer run (the models or hosts behind this path have changed since) · adversarially reviewed 2026-08-02, which rejected the previous draft; this is the wording it prescribed
Agentic multi-turn
ReviewedOracle: Real. Berkeley's own stateful checker decides whether the end state is correct.
Holds a multi-step task together across turns and leaves the world in the right state.
Measured. The number is published once this Tīrtha version is official.
no live setting to compare against · cleared 2026-08-02 with its comparability caveats attached
Failover
ReviewedOracle: Availability, not identical safety.
Honestly: when a host is down, the backup model matches the main one on calling a tool and is weaker at declining one.
Measured. The number is published once this Tīrtha version is official.
measured on a configuration we no longer run (the models or hosts behind this path have changed since) · cleared 2026-08-02 to be published as the counter-example beside the primary-path number, never alone
Answer reuse
Measured zeroOracle: Reuse of previously checked answers only.
Nothing is reused that was not checked when it was first served.
Zero external customers have ever been served a reused answer
65 reuse serves in 93,242 requests, all internal. Cross-context reuse is off. We do not sell a cost curve on this.
Adversarially reviewed. Re-derived from the production database 2026-08-02.
no live setting to compare against · re-derived from the production database 2026-08-02
What is serving, right now
You call one model id. Tīrtha picks the model for each task by price, speed and accuracy, checks the answer before serving it, and moves up a level when the check fails. The values below are read from the running configuration when this page is generated, which is what lets the badges above mean anything.
We publish results, not the recipe, and we would rather say so than look coy about it. Which specific model serves which request shape is the output of our own evaluation work, and it is the part a competitor could copy in an afternoon. So we show that each value was read live and that the paths genuinely differ, and we hold the map itself. Everything the map produces, including where it performs worse, is on this page. If you need to reproduce a number, reproduce it against the gateway: that is the thing we are actually selling, and it is the thing our numbers describe.
| Tool calls | configured, read live at generation time |
| Code and general requests | configured, read live at generation time |
| Backup if a host is down | configured, read live at generation time |
| The level above | configured, read live at generation time |
| Time allowed to move up a level (seconds) | 45 |
What we have not measured
Stated because its absence changes how you should read everything above.
- There is no customer traffic in these numbers. Latency and cost figures come from our own monitoring, probe and benchmark traffic, tool-call shaped. Not a forecast of your bill.
- No SOC 2. We are small. We publish mechanisms you can check instead of a badge we have not earned. See Trust.
- Agreement is not verification, and where no oracle exists we say so above rather than quietly counting it as a check.
Claims we have withdrawn
We published these and they were wrong. They are listed here rather than deleted, because a correction you cannot find is not a correction.
- "Roughly 8× lower cost" and "roughly 27×" are both struck. They were read off a meter that recorded only frontier spend, so lower-level requests logged as zero and about 43 percent of our real cost was invisible. Our real cost was higher than we said and the real multiple lower. The missing meter is deployed. We quote no cost multiple at all until we have a frontier comparison we can stand behind.
- A cache-compounding cost curve reaching about 60× is struck. It was a simulation presented as if it described how a real bill evolves. When we measured the live cache it was serving about 0.033 percent of production requests and had never served an external customer. It should not have been published. Judge any saving on the routing alone.
- "Correct 100 percent of the time when two models agree" is struck. Agreement is a confidence signal, not an oracle. Where no oracle exists we say so on this page rather than counting agreement as a check.
Every number here carries its population, sample size and date, or it does not appear. This page is regenerated on deploy; if the configuration changes and the page is not regenerated, the build fails rather than publishing a stale claim.