# Additional inference clients

[Open models](https://www.rightnowai.co/docs/open-models) / [Additional inference clients](https://www.rightnowai.co/docs/sdks/open-models/inference-clients)

Updated 2026-11-11

Both the model hub inference clients, `huggingface_hub` for Python and `@huggingface/inference` for JavaScript, accept a chat-compatible base URL, so both call the open-models service today.

**Connection**

| Connection | Value |
| --- | --- |
| API base | `https://api.runinfra.ai/v1` |
| Workspace key | `rp_...` |
| Process variable | `RUNINFRA_GATEWAY_KEY` |
| Billing | The existing funding rules of your open-model account. |

Replace `MODEL_ID` with an available id from [Open models](https://runinfra.ai/inference-api?utm_source=rightnowai.co&utm_medium=referral&utm_content=docs-open-models); read current capabilities, limits and prices from `GET /v1/models`.

The open-models service is not a registered inference provider in the model hub, so `provider="runinfra"` does not resolve. Use the base-URL forms below.

## Python

**Shell**

```shell
pip install "huggingface_hub>=1.0"
export RUNINFRA_GATEWAY_KEY="rp_your_key_here"
```

**Python**

```python
import os
from huggingface_hub import InferenceClient

client = InferenceClient(
    base_url="https://api.runinfra.ai/v1",
    api_key=os.environ["RUNINFRA_GATEWAY_KEY"],
)

completion = client.chat.completions.create(
    model="MODEL_ID",
    messages=[{"role": "user", "content": "Write a TypeScript retry helper."}],
    max_tokens=16384,
)

print(completion.choices[0].message.content)
```

Streaming is the same call with `stream=True`, reading `chunk.choices[0].delta.content` and skipping the usage frame, which has no choices.

## JavaScript

Pass `endpointUrl` in the client options and the open-models service key as the token. The client appends `/v1/chat/completions` for you, so give it the host with or without `/v1`.

**Shell**

```shell
npm install @huggingface/inference
export RUNINFRA_GATEWAY_KEY="rp_your_key_here"
```

**JavaScript**

```javascript
import { InferenceClient } from "@huggingface/inference";

const client = new InferenceClient(process.env.RUNINFRA_GATEWAY_KEY, {
  endpointUrl: "https://api.runinfra.ai",
});

const completion = await client.chatCompletion({
  model: "MODEL_ID",
  messages: [{ role: "user", content: "Write a TypeScript retry helper." }],
  max_tokens: 16384,
});

console.log(completion.choices[0].message.content);
```

`chatCompletionStream` takes the same arguments and yields deltas.

> **Warning**
>
> Do not combine `endpointUrl` with an explicit third-party `provider`. The client rejects that combination with `Cannot use endpointUrl with a third-party provider`. Pass `endpointUrl` alone.

## Repository mappings

These clients normally take a model-hub repository id. In the base-URL forms above you pass the open-models service model id, because you are talking to the open-models service directly.

For the mappings verified in the dated source table, the gateway also accepts the repository id in the `model` field. For another model, use its id from the [live model list](https://runinfra.ai/inference-api?utm_source=rightnowai.co&utm_medium=referral&utm_content=docs-open-models) unless alias support is explicitly documented. The source mappings were available on September 21, 2026; verify the current mapping before using it.

## Model capabilities

Working JSON snippets are in [Structured output](https://www.rightnowai.co/docs/open-models/guides/structured-output). When a model needs a specific `reasoning_effort` for JSON output, the gateway applies it if you omit the field. When the gateway chooses an effort that the request did not name, its response header `X-RunInfra-Reasoning-Effort-Applied` carries the applied value. A temporarily paused model is reported by the model list and on its Model Library page.

## Where the model's thinking arrives

For the reasoning models described on their own pages, the model thinks before it answers. That text streams on `delta.reasoning` and appears on `message.reasoning`, never on `content`.

## Which client should you use

If you already live inside the model hub ecosystem, the clients on this page work and nothing here changes later. Otherwise use the chat API client, which is the shortest path and what the rest of these docs assume. See [Compatible clients](https://www.rightnowai.co/docs/sdks/open-models).

Either way, these calls are billed like any other open-models service call. See [Pricing and credits](https://www.rightnowai.co/docs/open-models/pricing) and [Rate limits](https://www.rightnowai.co/docs/open-models/rate-limits).

## Related

[### Compatible clients The shortest path, and what the rest of these docs assume.](https://www.rightnowai.co/docs/sdks/open-models) [### Chat completions The full request contract.](https://www.rightnowai.co/docs/open-models/chat-completions) [### Pricing and credits How these calls are billed.](https://www.rightnowai.co/docs/open-models/pricing)

## Next

[CLI reference](https://www.rightnowai.co/docs/open-models/cli)