Cheaper multi-turn chats and more places to fail over
In a multi-turn chat, most of every prompt is the same text the model saw one turn earlier. Upstream providers cache that prefix and charge far less to reread it, but only when the request actually reaches the cache. v1.2.0 routes and accounts for that, and widens the set of providers behind the catalogue.
Prompt caching
Requests to Claude tiers now mark the tools, system prompt, and conversation so far as cacheable, so a follow-up turn rereads the earlier ones at about a tenth of the input price. For providers whose caches live on individual replicas, every turn of a conversation now carries the same opaque routing key, derived from its opening messages. The turns land where the prefix is already cached, and separate conversations still spread across replicas.
Cached tokens in usage
Responses now report the OpenAI-standard prompt_tokens_details.cached_tokens, whichever provider served the request. prompt_tokens still counts the whole prompt, cached or not, and the operator spend view breaks out cached reads and cache writes per bucket.
New tiers and more failover
Two tiers join the catalogue: onde-flare and onde-zenith. onde-flux and onde-orbit gain a second provider, so they no longer go down with a single upstream. Two more upstream providers now sit at the end of several tiers as last-resort candidates. As before, public model ids and responses never name the provider behind them.