Models
Updated
- Request
- Response
- Notes
- Model-specific operating rules
- Available and paused
- Retrieve one model
- Authentication and rate limits
- Related
| Connection | Value |
|---|---|
| API base | https://api.runinfra.ai/v1 |
| Workspace key | rp_... |
| Process variable | RUNINFRA_GATEWAY_KEY |
| Billing | The existing funding rules of your open-model account. |
Replace MODEL_ID with an available id from Open models; read current capabilities, limits and prices from GET /v1/models.
This is how an application discovers, at boot, which model ids its key can call and what binds them. Never hard-code a list you guessed.
Request
Response
Read the values from your own response; they move with capacity. The fields below describe the discovery contract.
| Field | Type | Value |
|---|---|---|
object | string | list at the top level, model for each item. |
data[].id | string | The id to send in the model field of an inference request. |
data[].owned_by | string | runinfra. |
data[].created | integer | Unix seconds, the creation time of the underlying record. |
data[].display_name | string | The model's display name. |
data[].supported_endpoints | string[] | Endpoints that serve this id: chat_completions, responses and messages for a chat model. |
data[].modality | string | llm for a chat model. |
data[].tool_calling | boolean | true when the model accepts tools. |
data[].json_mode | boolean | true when the model accepts a JSON response_format. |
data[].input_modalities | string[] | ["text"], or ["text", "image"] when the model accepts images. |
data[].reasoning_efforts | string[] | reasoning_effort values measured to behave distinctly on this model. Empty when none are declared. |
data[]. | string or null | The measured default when you omit reasoning_effort. null means none is declared, not that reasoning is off. |
data[]. | integer or null | For hosted chat models with a declared context, this equals max_output_tokens. It is null otherwise. |
data[].availability | string | available or paused. |
data[].max_request_bytes | integer | Largest request body the gateway accepts, 32MB. Bodies over 4MB need a Content-Length header, which every SDK sends. Enforced at the gateway, so it is the same for every model. |
data[].context_window | integer | Tokens the model can hold in one request. Present only where a served window has been reported. |
data[].context_length | integer | The same served window under the name chat-compatible catalog readers commonly use. |
data[].max_output_tokens | integer | Generated tokens allowed in one request. Oversized output budgets are clamped to the model's remaining context and reported with X-RunInfra-Output-Clamped: true and X-. |
data[]. | integer | Wall-clock seconds one response may run. See Output limits. |
data[].pricing | object | Hosted input and output prices in USD per one million tokens. |
data[]. | number | Hosted cached-input price in USD per one million tokens, when the effective cached tier is active. |
data[].cached_input_price | number | The same cached-input price in USD per one million tokens under the short discovery key. |
data[].cache_isolation | string | isolated when the active request path separates cache entries by tenant, otherwise shared. |
data[].cache_tiers | string[] | Proven storage tiers, such as gpu or host_ram. |
data[].cache_retention | string | best_effort when cached prefixes are kept as long as capacity allows and may be evicted under memory pressure, or tiered_host_memory when the proven cache also spans host memory. Both are best effort and no TTL is implied. The human-readable statement is on the model's page. |
data[]. | integer | Requests one key may have in flight against this model once the model is busy. A workspace resolves to the same number. While the model has capacity to spare, requests beyond it are admitted; see rate limits. |
data[]. | integer | Present only while the workspace's coding plan serves a model it covers. Requests the plan funds, on the plan or on credits after its limits, may run this many at once across the workspace once the model is busy. While the model has capacity to spare the gateway may admit more, so this understates and never overstates the allowance. |
data[]. | integer | Present beside the field above on a chat model. Standby requests the workspace may run at once, one allowance shared across every model it calls on Standby. 0 means Standby is closed on this model. |
data[]. | integer | Tokens per minute one workspace may spend on this model, over a rolling 60 second window. |
data[]. | integer | Idle seconds before a session's replica placement is forgotten. See Session affinity. |
data[].paused_until | string | ISO 8601 time availability is next rechecked, not a promised return. Present only when availability is paused. |
Notes
- The limit fields exist so you never learn a limit from a
429. They are read from the code that enforces them, so a published limit and an applied limit cannot drift apart. - These fields are additive to and disjoint from the fields the chat API Model object declares, so a client typed against the chat API SDK keeps parsing this response and ignores what it does not recognize.
- Absent means undeclared, never zero and never unlimited. A field appears only where the limit it names is actually enforced for that model; a model that has not reported a served window carries no
context_windowrather than borrowing another model's number. max_request_bytesis a byte ceiling, not a token ceiling, and the two bind independently. For a very long prompt the byte ceiling is often the one that binds first, whatever the context window says.- Hosted model prices are returned here for discovery and repeated on each model's page in the Model Library, with the human-readable cache eviction and retention statement, served precision when verified, capabilities, and the recommended output budget.
- A workspace key lists every public hosted model, available or paused.
Model-specific operating rules
The model-specific source reference describes the two served configurations below and identifies the models they cover. These requirements belong to those configurations, not to every model in the live list. Read each selected model's live context_length, reasoning_efforts, default_reasoning_effort, tool_calling, json_mode and max_output_tokens for the fields discovery actually carries.
Text-only served configuration
For the text-only configuration in that reference, use its primary id from the live list when your key can reach it. It accepts text input only. Check the current served window in context_length. Start with max_tokens: 16384 where the live output limit permits it, and count reasoning tokens within that budget. Check its current reasoning_efforts and default_reasoning_effort before changing the setting. The tool and structured-output rules explain the strict-tool setting required for forced calls.
For this configuration, the open-models service's GPU prefix caching is tenant-isolated and best effort. Cached prefixes may be evicted, and no retention time is promised. Send full conversation history on each turn because durable session storage and failover have not been qualified. The served artifact is covered by both the serving-vendor open model agreement and the model license named in the source reference.
Text-and-image served configuration
For the text-and-image configuration in that reference, the primary id and the model-hub alias both resolve when your key can reach it. It is served with BF16 weights and a BF16 KV cache, both unquantized. It accepts text and up to 8 images per request; video is refused. Read the current served window from context_length.
This configuration reasons by default, and its thinking arrives on reasoning, never on content. Start with max_tokens: 16384 where the live output limit permits it, and count reasoning tokens within that budget. No reasoning effort levels are published for it. Tool calling, json_object and json_schema work.
Keep tools in every request whose history carries tool calls, and prefer tool_choice: "auto" over "none". Nothing refuses a request that leaves them out, but on some models the answer can then come back empty, with choices[].runinfra.output_status.code set to no_answer_content.
A non-empty logit_bias is refused on this configuration with 400 hosted_parameter_not_supported and param: "logit_bias". Remove the field or send an empty object; see Chat completions for the related sampling restriction.
For this configuration, the open-models service's GPU prefix caching is tenant-isolated and best effort. Cached prefixes may be evicted, and no retention time is promised. Send full conversation history on each turn. The served model is covered by the model license, version 1.0, named in the source reference.
Available and paused
The two availability states a buyer sees: available serves 200 responses, paused answers 503 hosted_model_paused with its next availability check, and the model stays listed in both.
- Model availability
- GET /v1/models
- availability: available
- requests are served and billed
- Listed with its id, prices on its page.
- availability: paused
- Nothing is charged. Honor retry headers.
- return
- A paused model is not a missing model: the id stays valid and stays listed, with paused_until
- carrying the next availability check. Poll and retry; never drop the id from your configuration.
A paused model keeps its place in the list. It is a real configured model that is temporarily not serving, so poll this endpoint or retry the call itself to detect it coming back. The id does not change across a pause. While paused, the row omits pricing, the cached-input price fields, max_ and max_.
Retrieve one model
Returns a single object with the same fields and no list wrapper. An id your key cannot reach returns 404 model_not_found. URL-encode any id containing a slash.
Authentication and rate limits
The endpoint needs a workspace API key, sent as Authorization: Bearer or x-api-key. It uses that key's requests-per-minute limit and returns X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset and X-RateLimit-Tier. Listing models reserves no inference credit, but it is still an authenticated request against your per-minute budget, so do not call it in a hot loop.