# Native Python SDK

[Open models](https://www.rightnowai.co/docs/open-models) / [Native Python SDK](https://www.rightnowai.co/docs/sdks/open-models/native-python)

Updated 2026-11-11

Use the native Python SDK with Python 3.9 or newer. `RunInfra` is intentionally sync-only in v0.2.0 and uses Python's standard library HTTP stack. Run blocking calls in a worker thread or queue from an asynchronous application; there is no public async client.

**Connection**

| Connection | Value |
| --- | --- |
| API base | `https://api.runinfra.ai/v1` |
| Workspace key | `rp_...` |
| Process variable | `RUNINFRA_API_KEY` |
| Billing | The existing funding rules of your open-model account. |

Replace `MODEL_ID` with an available id from [Open models](https://runinfra.ai/inference-api?utm_source=rightnowai.co&utm_medium=referral&utm_content=docs-open-models); read current capabilities, limits and prices from `GET /v1/models`.

**Shell**

```shell
pip install runinfra
```

**Python**

```python
import os
from runinfra import RunInfra

api_key = os.environ.get("RUNINFRA_API_KEY")
if not api_key:
    raise RuntimeError("Set RUNINFRA_API_KEY before running this snippet.")

client = RunInfra(api_key=api_key)

response = client.chat.completions.create(
    model="MODEL_ID",
    messages=[{"role": "user", "content": "Hello!"}],
)

print(response["choices"][0]["message"]["content"])
```

These packages are beta, version 0.2.0. A helper does not establish that a model or endpoint is available. Read `/v1/models` and the selected model's `supported_endpoints` before sending a request.

Browser apps should call your server route or backend proxy first, then your server calls the open-models service with the workspace or pipeline-scoped key. Keep the API key on your server. The example reads `RUNINFRA_API_KEY` and passes it explicitly as `api_key`. The SDK does not load the variable automatically. Compatible-client examples use `RUNINFRA_GATEWAY_KEY`. These are distinct process variable names and are not automatic aliases.

## Compatibility limits

The Responses route is a chat-completions adapter, not a complete stateful Responses service. The native typed helpers do not claim GA support for tool calls, structured JSON schema outputs, logprobs, seeds, service tiers, parallel tool calls, Responses state/include/reasoning controls, embedding base64 output, image streaming or partial images, audio streaming, audio translations, or direct browser API-key use. Gateway support and native SDK coverage are separate contracts. Use the [endpoint reference](https://www.rightnowai.co/docs/open-models/chat-completions#responses) and [compatible clients](https://www.rightnowai.co/docs/sdks/open-models) for their supported behavior.

Embedding helpers are beta. Images Preview. Available when the deployment exposes image generation. Audio (TTS) Preview. Available when the deployment exposes speech generation. Audio (ASR) Preview. Available when the deployment exposes transcription. Voice pipeline Preview. Pipeline-scoped helper for co-located audio-to-response deployments. A published SDK method does not promise a hosted model for any of them. Unsupported operations return a typed error instead of falling back to a different operation.

## Webhook verification

Webhook helpers verify signatures locally against the exact raw body before JSON parsing. They do not create, deliver, list or manage webhook subscriptions.

The `RunInfra-Signature` timestamp must be a non-negative integer Unix second.

`construct_webhook_event` verifies the signature, checks timestamp tolerance, and parses JSON.

Use `verify_webhook_signature` when your framework parses JSON separately and you only need to validate the raw delivery.

Invalid signatures, stale timestamps, and invalid webhook JSON raise `WebhookVerificationError`.

## Retries and request ids

Safe GET requests retry transient transport failures and 408, 409, 429, 500, 502, 503 and 504 responses. A charge-bearing POST retries automatically only with an idempotency key on a replay-safe, non-streaming JSON Responses or chat request. Streaming, embeddings, images, binary audio and multipart uploads are sent once. Automatic retries honor reasonable Retry-After values up to 60 seconds when the header is a plain integer second value or HTTP-date, then fall back to bounded exponential backoff.

Authentication, insufficient-credit and unsupported-operation errors are not retried.

Use a fresh idempotency key for each logical operation. If a completed response is too large to replay, the same key returns `idempotency_replay_unavailable` without another generation or charge. Keep automatic retries off for work whose replay contract is not documented.

Log the request id with errors and support reports. Successful JSON object responses include `_request_id` when the gateway returns `x-request-id`. Streaming exposes a language-specific request id property. Malformed stream frames raise `RunInfraStreamParseError` with that request id. `RunInfraTimeoutError` also covers stalled streaming reads and default non-streaming body reads after headers arrive, and includes `request_id` when the response was traced. `RunInfraConnectionError` also covers streaming body transport failures and default non-streaming body transport failures after headers arrive, and includes `request_id` when the response was traced. When the gateway returns `x-request-id`, binary audio responses expose it as `audio.request_id`.

## Streaming

**Python**

```python
stream = client.responses.create(
    model="MODEL_ID",
    input="Hello",
    max_output_tokens=512,
    stream=True,
)

for event in stream:
    if event.get("type") == "response.output_text.delta":
        print(event.get("delta", ""), end="")

print(stream.request_id)
```

Close the active iterator when you stop consuming a stream early so local response resources are released:

**Python**

```python
iterator = iter(stream)
try:
    first = next(iterator)
finally:
    iterator.close()
```

Streaming transport-level backend cancellation is best effort, and streaming calls are not automatically retried. The Python type surface overloads `stream=True` calls for `client.chat.completions.create()` and `client.responses.create()` to return `RunInfraStream`; non-stream calls keep their typed response envelopes.

The open-models service route `/v1/responses` is a chat-completions compatibility adapter. The gateway converts supported `input` and `instructions` values into chat messages, forwards the supported request through the chat-completions serving path, and rewraps the result into a Responses-style envelope. It does not claim full chat API Responses state, include, reasoning, tool, conversation-item, or background-job semantics.

## Client options

Use `base_url`, `timeout_seconds`, `max_retries` and `retry_base_seconds`. Unknown request options and mixed snake\_case/camelCase aliases are rejected.

`timeout_seconds` must be positive, `max_retries` must be a non-negative integer, and `retry_base_seconds` must be non-negative.

Custom base URLs must use `http` or `https`. Other schemes and malformed URLs are rejected before a bearer API key can be sent. Remote custom base URLs must use `https`. Plain `http` is accepted only for local development hosts: `localhost`, `127.0.0.1`, `0.0.0.0`, and `[::1]`. Custom base URLs must not include usernames or passwords. Custom base URLs must not include query strings or fragments.

Invalid values raise `RunInfraError` with `type == "invalid_request_options"` before any network request is sent.

## Verified deployments and keys

Access optimized deployments through the open-models service's verified public gateway. LLM helpers are beta, with typed helpers for verified LLM and vision-language deployments. The surfaces have different verification levels; webhook verification runs locally.

Use a workspace-scoped key to reach verified active deployments through the `model` field.

In the open-models service dashboard, open Settings, API Keys, Create key, and keep Scope set to Workspace.

The Deploy tab can create a pipeline-scoped key for one optimized pipeline.

The one-time secret is shown once after creation.

Store it as `RUNINFRA_API_KEY` for app snippets before leaving the page.

After an optimization run finishes, open the Deploy view from the dashboard.

Deploy only shows SDK operations that the verified endpoint supports, so copy the native or chat-compatible snippet from there instead of guessing a route.

Use `pipeline_id` when the key or integration should be locked to one optimized pipeline. The default base URL is `https://api.runinfra.ai/v1`. `pipeline_id` is stripped and URL-encoded before it is added to the base URL.

Use either `pipeline_id` with the default base URL, or a `base_url` already scoped to the pipeline. If both point to the same pipeline, the SDK keeps the URL scoped once. The open-models service's generated native SDK snippets prefer `pipeline_id` with the root base URL. Chat-compatible snippets use the pipeline-scoped base URL because the chat SDK has no native pipeline option. Chat-compatible clients can use the same verified base URL.

## Async Python runtimes

For one-off calls inside an asyncio handler, move the blocking SDK call to the default worker thread pool:

**Python**

```python
import asyncio

result = await asyncio.to_thread(
    client.responses.create,
    model="MODEL_ID",
    input="Summarize this incident.",
)
```

For request paths that should return immediately, hand work to your framework's background execution path or an external queue:

## Typed public request fields

The native SDK validates the minimum request fields locally, then forwards chat API-style JSON or multipart fields that preserve the typed response shape.

Chat Completions: `model`, `messages`, `stream`, `temperature`, `top_p`, `max_tokens`, `stop`, `presence_penalty`, `frequency_penalty`, `user`, and `metadata`; streaming usage chunks are covered separately with `stream_options.include_usage`.

Responses: `model`, `input`, `stream`, `instructions`, `temperature`, `top_p`, `tools`, `tool_choice`, `response_format`, and `max_output_tokens`.

Embeddings: `model`, `input`, `encoding_format="float"`, and `dimensions` when the deployed embedding backend advertises dimension control.

Images: `model`, `prompt`, `n`, plus optional `size` and `response_format` when the deployed image backend advertises them.

Image `quality`, `style`, and `user` are typed pass-through chat API-style options when the deployed image backend supports them.

Audio speech: `model`, `input`, `voice` or `ref_audio` plus `ref_text`, and optional `task_type` and `response_format`.

Audio transcriptions: `model`, `file`, `filename`, optional `language`, optional `prompt`, and JSON response formats only.

Embedding `encoding_format` values other than `"float"` and transcription `response_format` values other than `"json"` or `"verbose_json"` are rejected locally because they would not match the typed native SDK response objects.

Unsupported chat API-style body parameters must fail with a clear traced 4xx gateway error.

LLM pass-through options are typed for parity with the TypeScript SDK and chat API-style request shapes, but actual support depends on the deployed backend.

Embedding `user`, TTS `speed`, and ASR `temperature` are typed pass-through options for SDK parity, but actual support depends on the deployed backend.

## Request validation

Required request fields are validated before any network request is sent.

The model must be a non-blank string, chat messages must be a non-empty array, each chat message must be an object with a non-empty role, Responses input must be a non-empty string or array, Responses input array items must be objects, JSON request bodies must be serializable and contain only finite numbers, embedding input must be a non-empty string or array of non-empty strings, TTS input and image prompts must be non-empty strings, and ASR file must be non-empty bytes or bytearray.

ASR multipart filenames and content types are validated before the multipart body is built.

Invalid request values raise `RunInfraError` with `type == "invalid_request_options"` and do not reach the gateway or billing path.

Use per-request options when a call needs a shorter timeout, a trace ID, or a retry-safe idempotency key.

Python request helpers expose explicit chat API-style keyword parameters instead of arbitrary `**kwargs`, so unknown direct request fields fail before any network request is sent.

For deliberate gateway compatibility probes or newly rolled out gateway fields, pass an `extra_body` mapping on JSON body helpers.

`extra_body` is only accepted on JSON body helpers.

`extra_body` cannot override typed request fields such as `model`, `input`, or `messages`.

Custom headers are for app metadata only.

They cannot override SDK-controlled headers such as `Authorization`, `Content-Type`, `X-Client-Request-Id`, `Idempotency-Key`, `X-RunInfra-SDK`, or `X-RunInfra-SDK-Version`, and they cannot set transport or credential headers such as `Host`, `Cookie`, `Content-Length`, `Transfer-Encoding`, `Connection`, `Proxy-Authorization`, `Api-Key`, `X-API-Key`, `X-Auth-Token`, or `X-Access-Token`.

## Typed errors and tracing

The SDK exposes `AuthenticationError`, `PermissionDeniedError`, `RateLimitError`, `InsufficientCreditsError`, `DeploymentError`, `ModelNotFoundError`, `RunInfraTimeoutError`, `RunInfraConnectionError`, and `RunInfraStreamParseError`.

`UnsupportedOperationError` remains exported for compatibility with older v0.1.x code, but current public helpers do not raise it.

`RateLimitError` includes `retry_after_seconds` when the gateway returns `Retry-After`.

`PermissionDeniedError.type` preserves a specific gateway discriminator on `403` responses when one is present (for example `byoc_plan_required` when a workspace below the deploy tier calls a BYOC-deployed endpoint); it falls back to `permission_denied`.

Branch on `err.type` instead of matching the message string.

`InsufficientCreditsError` includes `current_balance_cents`, `required_cents`, and `topup_url` when the gateway returns them on a `402` response, so you can render an exact top-up prompt without parsing the message.

Every request includes `X-RunInfra-SDK: python`, `X-RunInfra-SDK-Version`, and `X-Client-Request-Id`.

These headers help support trace requests without changing billing or routing.

When `idempotency_key` is provided, the SDK sends it as `Idempotency-Key`.

Idempotency keys must be non-blank, ASCII, 255 characters or less, and must not contain secrets or personal data.

Gateway errors expose `request_id`, `type`, and, when returned by the API, chat API-style `code` and `param` metadata such as `unsupported_parameter` and `dimensions`.

The wheel ships `py.typed` so type checkers can inspect the package.

Fixed-shape helpers expose `TypedDict` response contracts: `ModelListResponse`, `ModelObject`, `ResponsesCreateResponse`, `ChatCompletionResponse`, `EmbeddingResponse`, `TranscriptionResponse`, and `ImageGenerationResponse`.

Stream-capable helpers are typed as either the JSON response contract or `RunInfraStream` when `stream=True`.

## Speech and voice pipelines

TTS deployments can expose named voices or Base/reference-audio voice cloning.

Use `RUNINFRA_TTS_VOICE` when the deployment lists a voice or speaker.

Use `RUNINFRA_TTS_REF_AUDIO` and `RUNINFRA_TTS_REF_TEXT` when the deployment expects reference-audio input.

Co-located voice pipelines are available through the native `client.voice.pipeline.create()` helper on pipeline-scoped keys.

The helper posts binary audio to the pipeline-scoped `/pipeline` route and returns the JSON transcript / response envelope.

## Packages and support

See each package's README and changelog for surface-level docs, language-specific examples and exact helper names. The open-models service SDK is proprietary and source-available; see `LICENSE`. For commercial licensing inquiries about this SDK, contact `licensing@runinfra.ai`.

The open-models service publishes SDK packages through trusted publishing where the registry supports it. npm releases include provenance metadata, and PyPI releases use the configured trusted publisher chain.

Inspect npm provenance with `npm view @runinfra/sdk@latest dist.attestations`. For PyPI, open the release page for `runinfra` and inspect the publishing details for the selected version.

Open an issue or pull request against the SDK repository. See `CONTRIBUTING.md` for the workflow and style rules. For security issues, do not open a GitHub issue; see `SECURITY.md` for the disclosure process. For open-models service issues involving deployments, billing or accounts, email `support@runinfra.ai`.

## Next

[Additional inference clients](https://www.rightnowai.co/docs/sdks/open-models/inference-clients)