Additional inference clients

Python

pip install "huggingface_hub>=1.0"
export RUNINFRA_GATEWAY_KEY="rp_your_key_here"

import os
from huggingface_hub import InferenceClient

client = InferenceClient(
    base_url="https://api.runinfra.ai/v1",
    api_key=os.environ["RUNINFRA_GATEWAY_KEY"],
)

completion = client.chat.completions.create(
    model="MODEL_ID",
    messages=[{"role": "user", "content": "Write a TypeScript retry helper."}],
    max_tokens=16384,
)

print(completion.choices[0].message.content)

Streaming is the same call with stream=True, reading chunk.choices[0].delta.content and skipping the usage frame, which has no choices.

JavaScript

Pass endpointUrl in the client options and the open-models service key as the token. The client appends /v1/chat/completions for you, so give it the host with or without /v1.

npm install @huggingface/inference
export RUNINFRA_GATEWAY_KEY="rp_your_key_here"

import { InferenceClient } from "@huggingface/inference";

const client = new InferenceClient(process.env.RUNINFRA_GATEWAY_KEY, {
  endpointUrl: "https://api.runinfra.ai",
});

const completion = await client.chatCompletion({
  model: "MODEL_ID",
  messages: [{ role: "user", content: "Write a TypeScript retry helper." }],
  max_tokens: 16384,
});

console.log(completion.choices[0].message.content);

chatCompletionStream takes the same arguments and yields deltas.

Repository mappings

These clients normally take a model-hub repository id. In the base-URL forms above you pass the open-models service model id, because you are talking to the open-models service directly.

For the mappings verified in the dated source table, the gateway also accepts the repository id in the model field. For another model, use its id from the live model list unless alias support is explicitly documented. The source mappings were available on September 21, 2026; verify the current mapping before using it.

Model capabilities

Working JSON snippets are in Structured output. When a model needs a specific reasoning_effort for JSON output, the gateway applies it if you omit the field. When the gateway chooses an effort that the request did not name, its response header X-RunInfra-Reasoning-Effort-Applied carries the applied value. A temporarily paused model is reported by the model list and on its Model Library page.

Where the model's thinking arrives

For the reasoning models described on their own pages, the model thinks before it answers. That text streams on delta.reasoning and appears on message.reasoning, never on content.

Which client should you use

If you already live inside the model hub ecosystem, the clients on this page work and nothing here changes later. Otherwise use the chat API client, which is the shortest path and what the rest of these docs assume. See Compatible clients.

Either way, these calls are billed like any other open-models service call. See Pricing and credits and Rate limits.

Related