# Open-model retries

[Open models](https://www.rightnowai.co/docs/open-models) / [Open-model retries](https://www.rightnowai.co/docs/open-models/idempotent-retries)

Updated 2026-11-11

Send an `Idempotency-Key` header when a lost response or a network failure might make you retry. It is optional, it works on streaming and non-streaming requests alike, and it guarantees one generation and one charge. The same header and replay rules apply to `/v1/messages`.

**Connection**

| Connection | Value |
| --- | --- |
| API base | `https://api.runinfra.ai/v1` |
| Workspace key | `rp_...` |
| Process variable | `RUNINFRA_GATEWAY_KEY` |
| Billing | The existing funding rules of your open-model account. |

Replace `MODEL_ID` with an available id from [Open models](https://runinfra.ai/inference-api?utm_source=rightnowai.co&utm_medium=referral&utm_content=docs-open-models); read current capabilities, limits and prices from `GET /v1/models`.

Use a non-blank printable ASCII value of 255 characters or fewer after trimming. One fresh key per logical request, reused only when you are retrying that same request. A UUID generated at the call site is a convenient source.

**cURL**

```shell
curl https://api.runinfra.ai/v1/chat/completions \
  -H "Authorization: Bearer $RUNINFRA_GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: YOUR_STABLE_IDEMPOTENCY_KEY" \
  -d '{"model":"MODEL_ID","messages":[{"role":"user","content":"Write one sentence."}],"max_tokens":64}'
```

**Python**

```python
response = client.chat.completions.create(
    model="MODEL_ID",
    messages=[{"role": "user", "content": "Write one sentence."}],
    max_tokens=64,
    extra_headers={"Idempotency-Key": "YOUR_STABLE_IDEMPOTENCY_KEY"},
)
```

**TypeScript**

```typescript
const response = await client.chat.completions.create(
  {
    model: "MODEL_ID",
    messages: [{ role: "user", content: "Write one sentence." }],
    max_tokens: 64,
  },
  { headers: { "Idempotency-Key": "YOUR_STABLE_IDEMPOTENCY_KEY" } },
);
```

## What a reused key does

> Idempotent retry outcomes replay a stored response with a replay header, return 409 while the first request is still running, return 400 for different request parameters, and return 422 when the stored response is too large to replay; a streamed request with the same key replays its terminal receipt instead of generating again.
>
> - Reuse rules
> - Returns the stored response
> - `X-RunInfra-Idempotent-Replay`: true
> - 409 idempotency\_conflict
> - The first request is still running
> - Retry-After: 1
> - Retry with the same key.
> - 400 idempotency\_mismatch
> - The key was used with different request parameters
> - Use a different key only for a new logical request.
> - 422 idempotency\_replay\_unavailable
> - The stored response is too large to replay
> - Do not retry with a new key because that can start new inference work.
> - A streamed request replays its terminal receipt, never the delivered tokens.
> - Idempotency-Key
> - guarantees one generation and one charge.

Reuse the same key only with the same HTTP method, route, and request body. A key sent with a different body is a different logical request, and it is refused with `400` `idempotency_mismatch` rather than answered with the wrong reply.

A non-streamed response up to 6 MiB is held for 24 hours; a larger one answers `422 idempotency_replay_unavailable`.

## Retrying a stream

A stream is not replayed byte for byte. We do not store the tokens we deliver, so there is nothing to send again. What the key protects on a streaming request is the part that costs money.

**Reference 59**

| Situation | What you get |
| --- | --- |
| The original stream is still open | `409 idempotency_conflict` with `Retry-After: 1`. No second generation starts. |
| The original stream finished and settled | `200` with `X-RunInfra-Idempotent-Replay: true` and a non-streaming JSON receipt, or a short event stream on a streaming `/v1/messages` retry. Nothing is regenerated and nothing is billed again. |
| The original delivered nothing and was billed nothing | The key is released, so a retry runs normally, exactly as a request without a key would. |

The receipt is an ordinary chat completion envelope, so a chat-compatible client can parse it. `content` is `null` because the delivered text was never stored. `finish_reason` is `length` or `content_filter` when the original reply was cut short, and `null` otherwise. The replay facts ride in `idempotent_replay`.

**JSON**

```json
{
  "id": "chatcmpl-8f0a1c2e-...",
  "object": "chat.completion",
  "created": 1786800000,
  "model": "MODEL_ID",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": null },
      "finish_reason": null
    }
  ],
  "usage": {
    "prompt_tokens": 1000,
    "completion_tokens": 500,
    "total_tokens": 1500,
    "prompt_tokens_details": { "cached_tokens": 200 },
    "cost": 0.000282,
    "runinfra": { "cost_microcents": 28200, "cached_input_tokens": 200 }
  },
  "idempotent_replay": {
    "replayed": true,
    "streamed": true,
    "content_replayed": false,
    "regenerated": false,
    "billed_again": false,
    "original_status": 200,
    "stream_outcome": "completed",
    "usage_basis": "provider_reported",
    "cost_microcents": 28200,
    "detail": "The original request with this Idempotency-Key streamed to completion and settled once. Streamed tokens are not stored, so this reply carries the terminal usage and cost instead of the generated text. Nothing was regenerated and nothing was billed a second time."
  }
}
```

`X-RunInfra-Idempotent-Replay` is the signal on every route: read it before handing a response to a streaming parser. `/v1/chat/completions` and `/v1/messages` return the `idempotent_replay` object; on `/v1/responses` the reply is converted to a Responses object and only the header survives. A reply the original cut short keeps `status: "incomplete"` and its `incomplete_details.reason` there, as the original stream did.

If you need the generated text after a dropped stream, send a new request with a new key. That is new paid inference. Reusing the old key returns the receipt, not the text.

## Only one charge per key

A retried request is charged once, streamed or not: a duplicate settlement for the same key is refused rather than billed twice. The key holds for the whole time a request can run, so a retry sent while a slow non-streaming request is still generating gets `409 idempotency_conflict` and never starts a second generation.

When a key cannot be checked, a keyed request is refused rather than run without protection. Every endpoint, chat completions, Responses and messages included, returns `503 idempotency_unavailable` with `Retry-After` before any inference starts or any charge is made; retry with the same key. On `/v1/messages` the same refusal arrives as Messages protocol's `overloaded_error`, which the Messages protocol SDKs retry. A request without an `Idempotency-Key` is never affected.

## Client request ids

`X-Client-Request-Id` is an optional correlation header. The gateway echoes it back on the response. It is not a substitute for `Idempotency-Key`. Use a non-blank printable ASCII value of 512 characters or fewer. Any other value is refused with `400` `invalid_trace_header`.

## Related

[### Streaming The frame shapes a retried stream is not replayed as.](https://www.rightnowai.co/docs/open-models/streaming) [### Data retention The 24 hour replay window is the one place a response body is stored.](https://www.rightnowai.co/docs/open-models/data-retention)

[Source reference](https://runinfra.ai/docs/api-reference/idempotent-retries?utm_source=rightnowai.co&utm_medium=referral&utm_content=docs-open-models)

## Next

[Open-model troubleshooting](https://www.rightnowai.co/docs/open-models/troubleshooting)