Sessions

The identity headers

The gateway chooses the first valid identity in this order:

The gateway validates each candidate separately. An invalid header is skipped and never masks a valid header lower in the order. It does not cause an error response. After these identities comes the body field prompt_cache_key. An identity sharpens grouping from the workspace to the session.

Claude Code, Grok Build, and Codex identities use separate namespaces so they cannot collide with your own x-session-id. Raw session ids never leave the gateway: it hashes each identity with the workspace id before storing routing state or writing logs. Session headers are not forwarded to the model. Recorded agent usage contains header names, counts, and keyed fingerprints, never raw ids, raw header values, or prompt text.

Fan-out: workers follow the parent

An agent that fans out sends the orchestrator's turns on its own session, and each worker names that session as its parent. The workers then read the same warm prefix the orchestrator built, the shared system prompt and plan, with no coordination between them.

# Orchestrator turn, on its own session
curl https://api.runinfra.ai/v1/chat/completions \
  -H "Authorization: Bearer $RUNINFRA_GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -H "x-session-id: run-4127" \
  -d '{"model":"MODEL_ID","messages":[{"role":"user","content":"Plan the review, then dispatch workers."}],"max_tokens":16384}'

# Worker request, co-located with the parent
curl https://api.runinfra.ai/v1/chat/completions \
  -H "Authorization: Bearer $RUNINFRA_GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -H "x-parent-session-id: run-4127" \
  -d '{"model":"MODEL_ID","messages":[{"role":"user","content":"Review module A against the shared plan."}],"max_tokens":16384}'

A request's own x-session-id or x-session-affinity wins over x-parent-session-id, so a worker that should follow its parent sends x-parent-session-id only. Give a worker its own x-session-id once it carries its own multi-turn context worth keeping warm.

Coding agents

• Claude Code sends x-claude-code-session-id on every request. The main conversation is one session, and each subagent is a session of its own, told apart by the x-claude-code-agent-id it adds. A resumed conversation keeps its session.
• Codex CLI sends its own conversation thread as thread-id. A subagent's session-id names the root conversation, so the gateway prefers thread-id to keep each subagent's session separate. Both headers require a nonempty x-codex-window-id.
• Grok Build sends x-grok-session-id on every request. When Grok Build has no session, it sends an empty value, which the gateway skips.
• Mistral Vibe sends x-affinity with its session id on completion and streaming requests.
• OpenCode sends its own session as x-session-id and x-session-affinity, and a subagent adds x-parent-session-id. Each subagent keeps its own session.

An x-session-id you add yourself wins over an agent's own headers.

prompt_cache_key, user, and the opening messages

Clients that already send the standard prompt_cache_key request field get the same routing with no new headers: when no session identity is present, prompt_cache_key is read as the identity. The field is consumed at the gateway. It shapes routing and is removed before the request reaches the model. prompt_cache_options and prompt_cache_retention are rejected with 400 hosted_parameter_not_supported.

user is not a routing identity. It remains a standard request field and reaches the model unchanged. Without a session identity or prompt_cache_key, the gateway may place the request by its opening messages instead: the user value together with the leading system or developer text and the first user message. Two requests that share a user value but open with different messages may be placed apart. A request with no message text keeps workspace-level placement. Prefer prompt_cache_key, or a session header, in new code.

A session header keeps an established session's requests close to its warm cache even as load changes, until the session has been idle for 15 minutes or its warm cache becomes unavailable. A body hint such as prompt_cache_key groups requests by value but does not hold an established session in place. Neither controls cache retention.

How placement behaves

• A new session's placement considers current load. An established session keeps its placement while it stays active; load changes alone never move it.
• A session's placement is remembered for 15 minutes of idle time, the window GET /v1/models publishes as session_affinity_idle_ttl_seconds. After a longer gap, the next request is placed afresh, with load considered again. This is a routing window, not a cache retention promise; what the cache kept is covered below.
• Affinity is a preference, not a delivery constraint. If the session's warm cache is unavailable, the request is still served and the session settles where it was served, so its cache rebuilds from that request onward.
• Identity values are scoped to your workspace, and the raw value is never stored. The same literal value sent by another workspace shares nothing with yours.

Affinity and cache retention

Affinity keeps a session's requests close to the cache that already holds its prefix. Retention is best effort on every model, with no keep window. GET /v1/models states each model's cache_isolation, cache_tiers and cache_retention.

Related