# Models

[Open models](https://www.rightnowai.co/docs/open-models) / [Models](https://www.rightnowai.co/docs/open-models/models)

Updated 2026-11-11

**Connection**

| Connection | Value |
| --- | --- |
| API base | `https://api.runinfra.ai/v1` |
| Workspace key | `rp_...` |
| Process variable | `RUNINFRA_GATEWAY_KEY` |
| Billing | The existing funding rules of your open-model account. |

Replace `MODEL_ID` with an available id from [Open models](https://runinfra.ai/inference-api?utm_source=rightnowai.co&utm_medium=referral&utm_content=docs-open-models); read current capabilities, limits and prices from `GET /v1/models`.

**Text**

```text
GET https://api.runinfra.ai/v1/models
```

This is how an application discovers, at boot, which model ids its key can call and what binds them. Never hard-code a list you guessed.

## Request

**cURL**

```shell
curl https://api.runinfra.ai/v1/models \
  -H "Authorization: Bearer $RUNINFRA_GATEWAY_KEY"
```

**Python**

```python
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.runinfra.ai/v1",
    api_key=os.environ["RUNINFRA_GATEWAY_KEY"],
)
for model in client.models.list().data:
    print(model.id)
```

**TypeScript**

```typescript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.runinfra.ai/v1",
  apiKey: process.env.RUNINFRA_GATEWAY_KEY,
});
const models = await client.models.list();
console.log(models.data.map((model) => model.id));
```

## Response

Read the values from your own response; they move with capacity. The fields below describe the discovery contract.

**Reference 82**

| Field | Type | Value |
| --- | --- | --- |
| `object` | string | `list` at the top level, `model` for each item. |
| `data[].id` | string | The id to send in the `model` field of an inference request. |
| `data[].owned_by` | string | `runinfra`. |
| `data[].created` | integer | Unix seconds, the creation time of the underlying record. |
| `data[].display_name` | string | The model's display name. |
| `data[].supported_endpoints` | string\[\] | Endpoints that serve this id: `chat_completions`, `responses` and `messages` for a chat model. |
| `data[].modality` | string | `llm` for a chat model. |
| `data[].tool_calling` | boolean | `true` when the model accepts `tools`. |
| `data[].json_mode` | boolean | `true` when the model accepts a JSON `response_format`. |
| `data[].input_modalities` | string\[\] | `["text"]`, or `["text", "image"]` when the model accepts images. |
| `data[].reasoning_efforts` | string\[\] | `reasoning_effort` values measured to behave distinctly on this model. Empty when none are declared. |
| `data[].default_reasoning_effort` | string or null | The measured default when you omit `reasoning_effort`. `null` means none is declared, not that reasoning is off. |
| `data[].max_completion_tokens` | integer or null | For hosted chat models with a declared context, this equals `max_output_tokens`. It is `null` otherwise. |
| `data[].availability` | string | `available` or `paused`. |
| `data[].max_request_bytes` | integer | Largest request body the gateway accepts, 32MB. Bodies over 4MB need a `Content-Length` header, which every SDK sends. Enforced at the gateway, so it is the same for every model. |
| `data[].context_window` | integer | Tokens the model can hold in one request. Present only where a served window has been reported. |
| `data[].context_length` | integer | The same served window under the name chat-compatible catalog readers commonly use. |
| `data[].max_output_tokens` | integer | Generated tokens allowed in one request. Oversized output budgets are clamped to the model's remaining context and reported with `X-RunInfra-Output-Clamped: true` and `X-RunInfra-Output-Token-Ceiling`. |
| `data[].response_time_ceiling_seconds` | integer | Wall-clock seconds one response may run. See [Output limits](https://www.rightnowai.co/docs/open-models/chat-completions#output-limits). |
| `data[].pricing` | object | Hosted input and output prices in USD per one million tokens. |
| `data[].cached_input_price_usd_per_mtok` | number | Hosted cached-input price in USD per one million tokens, when the effective cached tier is active. |
| `data[].cached_input_price` | number | The same cached-input price in USD per one million tokens under the short discovery key. |
| `data[].cache_isolation` | string | `isolated` when the active request path separates cache entries by tenant, otherwise `shared`. |
| `data[].cache_tiers` | string\[\] | Proven storage tiers, such as `gpu` or `host_ram`. |
| `data[].cache_retention` | string | `best_effort` when cached prefixes are kept as long as capacity allows and may be evicted under memory pressure, or `tiered_host_memory` when the proven cache also spans host memory. Both are best effort and no TTL is implied. The human-readable statement is on the model's page. |
| `data[].max_concurrent_requests_per_api_key` | integer | Requests one key may have in flight against this model once the model is busy. A workspace resolves to the same number. While the model has capacity to spare, requests beyond it are admitted; see [rate limits](https://www.rightnowai.co/docs/open-models/rate-limits). |
| `data[].max_concurrent_plan_requests_per_workspace` | integer | Present only while the workspace's coding plan serves a model it covers. Requests the plan funds, on the plan or on credits after its limits, may run this many at once across the workspace once the model is busy. While the model has capacity to spare the gateway may admit more, so this understates and never overstates the allowance. |
| `data[].max_concurrent_standby_requests_per_workspace` | integer | Present beside the field above on a chat model. Standby requests the workspace may run at once, one allowance shared across every model it calls on Standby. `0` means Standby is closed on this model. |
| `data[].max_tokens_per_minute_per_workspace` | integer | Tokens per minute one workspace may spend on this model, over a rolling 60 second window. |
| `data[].session_affinity_idle_ttl_seconds` | integer | Idle seconds before a session's replica placement is forgotten. See [Session affinity](https://www.rightnowai.co/docs/open-models/session-affinity). |
| `data[].paused_until` | string | ISO 8601 time availability is next rechecked, not a promised return. Present only when `availability` is `paused`. |

## Notes

- The limit fields exist so you never learn a limit from a `429`. They are read from the code that enforces them, so a published limit and an applied limit cannot drift apart.
- These fields are additive to and disjoint from the fields the chat API Model object declares, so a client typed against the chat API SDK keeps parsing this response and ignores what it does not recognize.
- Absent means undeclared, never zero and never unlimited. A field appears only where the limit it names is actually enforced for that model; a model that has not reported a served window carries no `context_window` rather than borrowing another model's number.
- `max_request_bytes` is a byte ceiling, not a token ceiling, and the two bind independently. For a very long prompt the byte ceiling is often the one that binds first, whatever the context window says.
- Hosted model prices are returned here for discovery and repeated on each model's page in the [Model Library](https://runinfra.ai/inference-api?utm_source=rightnowai.co&utm_medium=referral&utm_content=docs-open-models), with the human-readable cache eviction and retention statement, served precision when verified, capabilities, and the recommended output budget.
- A workspace key lists every public hosted model, available or paused.

## Model-specific operating rules

The [model-specific source reference](https://runinfra.ai/docs/api-reference/models?utm_source=rightnowai.co&utm_medium=referral&utm_content=docs-open-models) describes the two served configurations below and identifies the models they cover. These requirements belong to those configurations, not to every model in the live list. Read each selected model's live `context_length`, `reasoning_efforts`, `default_reasoning_effort`, `tool_calling`, `json_mode` and `max_output_tokens` for the fields discovery actually carries.

### Text-only served configuration

For the text-only configuration in that reference, use its primary id from the live list when your key can reach it. It accepts text input only. Check the current served window in `context_length`. Start with `max_tokens: 16384` where the live output limit permits it, and count reasoning tokens within that budget. Check its current `reasoning_efforts` and `default_reasoning_effort` before changing the setting. The [tool and structured-output rules](https://www.rightnowai.co/docs/open-models/chat-completions#model-specific-tool-and-json-rules) explain the strict-tool setting required for forced calls.

For this configuration, the open-models service's GPU prefix caching is tenant-isolated and best effort. Cached prefixes may be evicted, and no retention time is promised. Send full conversation history on each turn because durable session storage and failover have not been qualified. The served artifact is covered by both the serving-vendor open model agreement and the model license named in the source reference.

### Text-and-image served configuration

For the text-and-image configuration in that reference, the primary id and the model-hub alias both resolve when your key can reach it. It is served with BF16 weights and a BF16 KV cache, both unquantized. It accepts text and up to 8 images per request; video is refused. Read the current served window from `context_length`.

This configuration reasons by default, and its thinking arrives on `reasoning`, never on `content`. Start with `max_tokens: 16384` where the live output limit permits it, and count reasoning tokens within that budget. No reasoning effort levels are published for it. Tool calling, `json_object` and `json_schema` work.

Keep `tools` in every request whose history carries tool calls, and prefer `tool_choice: "auto"` over `"none"`. Nothing refuses a request that leaves them out, but on some models the answer can then come back empty, with `choices[].runinfra.output_status.code` set to `no_answer_content`.

A non-empty `logit_bias` is refused on this configuration with `400 hosted_parameter_not_supported` and `param: "logit_bias"`. Remove the field or send an empty object; see [Chat completions](https://www.rightnowai.co/docs/open-models/chat-completions#sent-untouched-the-forwarding-allowlist) for the related sampling restriction.

For this configuration, the open-models service's GPU prefix caching is tenant-isolated and best effort. Cached prefixes may be evicted, and no retention time is promised. Send full conversation history on each turn. The served model is covered by the model license, version 1.0, named in the source reference.

## Available and paused

> The two availability states a buyer sees: available serves 200 responses, paused answers 503 hosted\_model\_paused with its next availability check, and the model stays listed in both.
>
> - Model availability
> - GET /v1/models
> - availability: available
> - requests are served and billed
> - Listed with its id, prices on its page.
> - availability: paused
> - Nothing is charged. Honor retry headers.
> - return
> - A paused model is not a missing model: the id stays valid and stays listed, with paused\_until
> - carrying the next availability check. Poll and retry; never drop the id from your configuration.

A paused model keeps its place in the list. It is a real configured model that is temporarily not serving, so poll this endpoint or retry the call itself to detect it coming back. The id does not change across a pause. While paused, the row omits `pricing`, the cached-input price fields, `max_concurrent_requests_per_api_key` and `max_tokens_per_minute_per_workspace`.

## Retrieve one model

**Shell**

```shell
curl https://api.runinfra.ai/v1/models/MODEL_ID \
  -H "Authorization: Bearer $RUNINFRA_GATEWAY_KEY"
```

Returns a single object with the same fields and no `list` wrapper. An id your key cannot reach returns `404` `model_not_found`. URL-encode any id containing a slash.

## Authentication and rate limits

The endpoint needs a workspace API key, sent as `Authorization: Bearer` or `x-api-key`. It uses that key's requests-per-minute limit and returns `X-RateLimit-Limit`, `X-RateLimit-Remaining`, `X-RateLimit-Reset` and `X-RateLimit-Tier`. Listing models reserves no inference credit, but it is still an authenticated request against your per-minute budget, so do not call it in a hot loop.

## Related

[### Chat completions Send one of these ids as the `model` field.](https://www.rightnowai.co/docs/open-models/chat-completions) [### Authentication The key that decides which ids you see.](https://www.rightnowai.co/docs/open-models/authentication) [### Rate limits The four layers that can refuse you.](https://www.rightnowai.co/docs/open-models/rate-limits)

[Source reference](https://runinfra.ai/docs/api-reference/models?utm_source=rightnowai.co&utm_medium=referral&utm_content=docs-open-models)

## Next

[Chat completions and Responses](https://www.rightnowai.co/docs/open-models/chat-completions)