Spend limits, and apps you can manage
v1.4.0 adds spend limits and app management. A request from an app with no limit set behaves as it did in v1.3.1.
A monthly limit per organization
An organization can set a monthly hard limit in micro-USD. Once its spend for the calendar month (UTC) reaches the limit, /v1/chat/completions answers 429 with the type insufficient_quota and the code spend_limit_reached, before any model is called. There is no Retry-After: waiting a few seconds will not help. The limit lifts at the start of the next month, or within about half a minute of being raised.
A daily limit per app
An app can also have its own hard limit per UTC day. This is for a product that gives each of its own users an app, so that one user cannot spend the organization's month. An app at its daily limit gets a 429 with the type insufficient_quota and the code daily_limit_reached until 00:00 UTC.
A daily limit is strict. If Onde Cloud cannot confirm today's spend for an app that has one, it answers 503 rather than let the request through. Apps without a daily limit, and the monthly limit, are not affected by this.
Both limits are counted within seconds
Spend counts toward a limit within a request on the server that handled it, and within about half a minute on the others. A request already in flight when a limit is reached still completes, so spend can pass a limit by that much.
The console API
Members of an organization can read its spend, usage, rates and invoices under /console/v1/orgs/{org}/billing, signed in with their own Onde account. Owners and admins can set its monthly limit there too.
Owners and admins can now also create an app with POST /console/v1/orgs/{org}/apps, limited to the tiers they list, read it with its key, suspend or reactivate it, and set its daily limit at /console/v1/orgs/{org}/apps/{app}/budget. A suspended app's key stops working within five minutes. A request for an organization you are not a member of, or for an app in another organization, answers 404.