Call an open model

Get a workspace API key

Open the Model Library, pick a model that is not marked Paused, and select Get API key. If you already have a key you can copy, the page highlights that key's copy button instead of making another one.

Put the key in your environment

Keep the key in your server's secret store and inject it into the calling process as RUNINFRA_GATEWAY_KEY. The example reads that variable explicitly. Do not put the key in browser code or a prompt.

List the live models

Run GET /v1/models for the current list; paused models are reported there and on their model pages. Choose an available ID with chat support. See Models for the discovery fields.

Make the call

Replace MODEL_ID below with that ID. This example sends one generation request and can spend open-model credit.

import json
import os
from urllib.request import Request, urlopen

base = "https://api.runinfra.ai/v1"
model_id = "MODEL_ID"
if model_id == "MODEL_ID":
    raise SystemExit("Replace MODEL_ID with an ID from /v1/models.")

headers = {
    "Authorization": "Bearer " + os.environ["RUNINFRA_GATEWAY_KEY"],
    "Content-Type": "application/json",
}

with urlopen(Request(base + "/models", headers=headers)) as response:
    models = json.load(response)
available = {row["id"]: row for row in models["data"]
             if row.get("availability") == "available"
             and "chat_completions" in row.get("supported_endpoints", [])}
if model_id not in available:
    raise SystemExit("Choose an available ID from the list.")

limit = available[model_id].get("max_output_tokens")
if type(limit) is not int or limit <= 0:
    raise SystemExit("The model list does not declare a usable output limit.")
output_budget = min(16_384, limit)

body = {
    "model": model_id,
    "messages": [{"role": "user", "content": "Say hello in one sentence."}],
    "max_tokens": output_budget,
}
request = Request(base + "/chat/completions", headers=headers,
                  data=json.dumps(body).encode("utf-8"), method="POST")
with urlopen(request) as response:
    request_id = response.headers.get("x-request-id")
    reply = json.load(response)
print(reply["choices"][0]["message"]["content"])

The example uses a generous output budget of 16,384 tokens where the selected model's published limit permits it; reasoning shares that budget. It takes the minimum of 16,384 and the live max_output_tokens. That sample budget is not a service maximum. If discovery omits the limit, the sample stops instead of guessing. The gateway may clamp further to the remaining context; see Output limits.

What comes back

In a normal completion, reasoning carries the model's thinking and content carries the answer. Token counts and ids in response examples are illustrative. Check finish_reason before using the answer: stop means the model finished on its own, and length means the output budget ended the reply.

Keep the request ID when reporting a problem. Use the open-model errors and retry guides before adding automatic retries.

How usage is counted

The usage counts in the response are the billed counts; a local tokenizer can count differently. Cached input is part of prompt_tokens, shows in usage.prompt_tokens_details.cached_tokens on models with a cached-input price, and bills at that rate.

For a model that reports reasoning support, reasoning tokens are billed output that count toward max_tokens. You pay for what is generated, not for the ceiling, so a generous ceiling costs nothing extra and a small one can leave you without an answer.

A reasoning model can exhaust a 2,048-token budget before an answer; a 16,384-token budget leaves more room where the live model limit permits it. In measurements on 2026-08-14, a 2,048-token budget produced an empty or truncated answer. Under the current usage contract, a response with no answer settles at zero; older measurements that billed output are historical.

Read the live model's capabilities before requesting reasoning, tools, structured output or images.

Next steps

Source reference

Chat completions and Responses, Streaming, compatible clients, or use the RightNow model.