Streaming

Connect

Streaming is server to server only. Use Node 22 or later or Python 3. Browsers cannot set the Authorization header on the WebSocket upgrade. Keep the key in the server process.

Use the API origin with WSS and /v1/system-zero/stream. Authenticate on the upgrade with Authorization: Bearer <key>. Request only system-zero.stream.v1 in Sec-WebSocket-Protocol.

Use text JSON messages. Send no browser Origin, URL credential, cookie authentication, compression, binary data, query or trailing slash.

Use an SDK

Install an SDK, then read the key from the server environment. Both languages export SystemZeroStreamClient.

TypeScript requires upgrade headers. Supply a server constructor through WebSocket if native support is missing.

import { SystemZeroStreamClient } from '@rightnowai/system-zero';

const stream = new SystemZeroStreamClient();
stream.on('decision', event => console.log(event.reply, event.request_id));
stream.on('error', event => console.error(event.envelope));
try {
  await stream.connect();
  await stream.start({
    model: 'rightnow-decision', state: 'Order received.',
    questions: { ready: { type: 'boolean', instructions: 'Is the order ready?' } },
  }, { decisionMode: 'on_request' });
  const ack = await stream.appendText(' Payment received.');
  await stream.decide();
  const measured = await stream.waitForDecision(ack);
  console.log(measured.decision.reply);
} finally {
  await stream.close();
}

Await each mutation for its acknowledgement. Read decision for the reply and request ID, and error for its envelope. The clients answer heartbeats and pace changes using the advertised limits.

Reconnect restores accepted state with its idempotency keys. It does not resume a durable stream. Read possibly_decided when a lost connection leaves an unknown outcome. Reopening does not prove the work was uncounted.

Message flow

Your server       RightNow
|                        |
|<======== hello ========|
|======== start ========>|
|<========= ack =========|
|===== append_text =====>|
|<========= ack =========|
|======= decide ========>|
|<====== scheduled ======|
|<======== ping =========|
|======== pong =========>|
|<====== decision =======|
|<======= goodbye =======|

start  within 10 s of hello
pong  within 10 s of ping
ping  every 30 s
idle  closes after 120 s
life  60 minutes

Read hello and its limits, then send start before its deadline. Questions, routing, scheduling and the cache key stay fixed for the connection. Read whole decisions in a stream with hello, ack, decision, error frames. The stream does not send individual tokens.

Use append_text for string state, append_items for whole array elements, or replace for a complete new state. Objects are replaced whole; field-level patches are not supported. append_attachments changes only the inline attachment list. Each accepted mutation receives ack and advances state_version.

With on_change, accepted updates may be combined into a decision. With on_request, send decide(state_version) for the current accepted version and read scheduled. A decision embeds the ordinary HTTP reply. Each decision reports its snapshot's state_version and covered_event_ids. Check state_version before using the answer because newer state may already have arrived. Use on_change (the default) when you want an answer for every state change: the decision starts the moment the event is accepted. on_request starts a decision only when you ask for one, after the acknowledgement of your event, so each decision adds one network round trip from your client. measured 2026-11-10 02:42Z, three sessions from a US west-coast client: 80-150 ms from event accepted to decision started in on_request mode; the server's own step in between took 2-5 ms.

Answer each JSON ping with pong and the same nonce within 10000 milliseconds of the completed ping write. Native WebSocket controls are separate. A missed pong closes with 1013 and reason slow_reader.

Idle = 120 seconds with no client application message and no decide in flight; the clock restarts when a decide settles; heartbeats do not count. The client ends normally with WebSocket close code 1000. There is no client goodbye or client ping JSON message.

A server close can send goodbye first. Durable stream resume is unsupported. Reconnect and replay the full desired state with the same per-state idempotency key and byte-identical request. Successful keys have a 24-hour answer window and retain an identity tombstone without request or answer content for 30 days after that window; after answer expiry, the same body returns 409 idempotency_replay_unavailable and a changed body returns 400 idempotency_key_reuse.

On a new connection the body is the same only if start carried your own prompt_cache_key. Without one the server uses the new stream_id, and the key returns 400 idempotency_key_reuse. If a decision did not arrive before the connection closed, the same key and body can return the completed answer. Replay availability is not guaranteed. Read Idempotency. Without a key, a decision lost in transit cannot be retrieved. Send the state again; that starts new work.

[ Socket closed ]
  |
  |-- 1002 1003 1007 1008 1009
  |   never reconnect
  |-- 1001
  |   reconnect at once
  |-- 1012
  |   wait 500 ms
  |-- 1013
  |   wait the hint or back off
  |-- 1011
  |   back off, then reconnect
  '-- 1000
      reconnect when needed

[ Replay full state, same keys ]
[ Read possibly_decided ]

Update an image

Save prepared PNG files as shelf-before.png and shelf-after.png. This Python example prints four client messages. Send them individually over your authenticated server connection after hello.

import base64
import json
from pathlib import Path

messages = [
    {
        "type": "start",
        "event_id": "evt_1",
        "decision_mode": "on_request",
        "request": {
            "model": "rightnow-decision",
            "state": "Check the current shelf image.",
            "attachments": [
                {
                    "name": "shelf",
                    "type": "image",
                    "data": "shelf-before.png"
                }
            ],
            "questions": {
                "stock": {
                    "type": "choice",
                    "instructions": "Are items visible on shelf?",
                    "criteria": {
                        "present": "Items are visible.",
                        "empty": "The shelf is empty.",
                        "unreadable": "The shelf cannot be assessed."
                    }
                }
            }
        }
    },
    {
        "type": "decide",
        "state_version": 1
    },
    {
        "type": "replace",
        "event_id": "evt_2",
        "state": "Check the current shelf image.",
        "attachments": [
            {
                "name": "shelf",
                "type": "image",
                "data": "shelf-after.png"
            }
        ]
    },
    {
        "type": "decide",
        "state_version": 2
    }
]
for message in messages:
    request = message.get("request", message)
    for attachment in request.get("attachments", []):
        path = Path(attachment["data"])
        encoded = base64.b64encode(path.read_bytes()).decode("ascii")
        attachment["data"] = f"data:image/png;base64,{encoded}"
    print(json.dumps(message))

Send start, wait for ack at version 1, then send decide. Read that decision before sending replace. Wait for its ack at version 2, then request the second decision. Keep answering heartbeats and honoring the advertised limits.

Replacement supplies the complete image list. Omit it to retain the previous list; use append_attachments only when you want to keep existing images and add more. All retained files must still fit the ordinary count and shared byte limits.

Each new answered snapshot charges every image it contains, including retained images: USD 0.003 per image, USD 0.0001 per answered question and USD 0.00001 per answered call, with residual text charged separately. Each of these two snapshots includes one image fee if answered.

An acknowledgement is not a billable answer. Media-token cache hits do not remove file fees. There is no durable media cache. This example requests separate snapshots; automatic decisions may combine updates.

Client messages

Server messages

Shared fields

State is a JSON object, array or string. Nested numbers, booleans and null remain JSON. Use compact keys and send large integers or exact decimals as numeric strings when precision matters. Supply your own units, coordinate frames and freshness data.

Each decision follows the ordinary decide API's validation, availability, billing and limits. Keep choice options and score levels within the model list's limits. A larger schema bound is not a serving promise. An ack accepts state into connection memory; it grants no input availability and confirms no decision, charge or durable save.

Question names use at most 64 ASCII letters, digits and underscores, starting with a letter or underscore. Attachment names use at most 64 ASCII letters, digits, underscores or hyphens, starting with a letter. Attachment names must be unique.

If start omits prompt_cache_key, the server uses stream_id. For byte-identical replay across connections, send an explicit stable prompt_cache_key.

A request policy goes in start.request.

Omitting replace.attachments retains the current list. An empty list removes every attachment. Text and array appends keep attachments unchanged. Inline data has no separate MIME field. Document files must be inline PDF data. PDF and video (preview) use the same limits as HTTP. Read Attachments. The stream does not take upload references.

Ordering and sequence

Each accepted mutation receives ack and increments state_version once.

IDs are precisely evt_1, evt_2 and so on; a rejected update leaves the next ID available for a corrected retry.

Text appends concatenate exactly, arrays append complete values, and replace changes the complete string, array or object state.

Use replace to change an object state; attachments can change independently.

An ack means accepted state in gateway memory, not a decision, bill or durable save.

Every accepted mutation, including an attachment append or replacement, creates a new version with its own supplied key; an unchanged version retains its key.

Rejected mutations leave the previous key intact; an unkeyed mutation clears it.

The gateway maintains one decide in flight and one pending intent.

Dense changes coalesce into the latest state.

In on_request mode, decide must name the current accepted version; its snapshot is chosen when dispatch becomes eligible.

state_version is its cumulative coverage watermark; covered_event_ids names only the mutation batch newly frozen into that dispatch.

An ack precedes any decision that covers it.

Only a successful decision, never an ack or error, resolves an event-to-decision observation.

During shutdown, valid mutations are applied and acknowledged with scheduled: false; they never start another decide on this connection.

Server seq starts at 1 and increases for every server message.

Idempotency keys contain 1 to 255 printable ASCII characters after trimming surrounding ASCII spaces. An unkeyed mutation clears the previous key.

Reconnect creates a new stream with new event numbering.

Replay the full desired state in start, then append only mutations absent from that replay.

Use full-jitter reconnect backoff, honoring any longer server retry delay.

Heartbeats neither bill inference nor extend idle/lifetime.

Limits

Read the limits advertised in hello. Stream limits do not enlarge ordinary HTTP limits. Idle time excludes heartbeats and time spent waiting for a decision. Account stream capacity is one stream socket across machines, for free, credit-only and paid accounts.

receipt_timeout_ms is a legacy wire hint, not a complete receipt deadline. Continue checking peer liveness and receipt inactivity. Both clients use a receipt deadline of 3818000 ms from command send; confirmed liveness can refresh inactivity timers but cannot extend it, and JSON-only EventTarget transports use a 3848000 ms heartbeat fallback without extending the receipt deadline.

max_duration_ms

3600000 milliseconds

idle_timeout_ms

120000 milliseconds

start_timeout_ms

10000 milliseconds

heartbeat_interval_ms

30000 milliseconds

native_heartbeat_interval_ms

10000 milliseconds

receipt_timeout_ms

240000 milliseconds

max_control_queue_frames

2 frames

pong_timeout_ms

10000 milliseconds

max_message_bytes

4194304 UTF-8 bytes

max_decide_body_bytes

3665920 UTF-8 bytes

max_pending_events

4096 events

max_events

100000 events

max_events_per_second

5 events per second

max_send_queue_bytes

4194304 bytes

max_send_queue_frames

64 frames

min_interval_ms

0 milliseconds

event_burst

10 events

max_messages_per_second

40 messages per second

message_burst

80 messages

max_streams_per_process

16 streams

max_native_control_frames_per_second

8 frames per second

native_control_frame_burst

16 frames

max_native_control_bytes_per_second

512 bytes per second

native_control_byte_burst

1024 bytes

maximum_decision_interval

60000 milliseconds

mutation_burst

10 events

application_message_rate

40 messages per second

application_message_burst

80 messages

max_pending_probes

2 probes

write_progress_timeout

5000 milliseconds

decision_settlement_timeout

5000 milliseconds

close_handshake_timeout

5000 milliseconds

consecutive_retryable_failures

3 failures per state version

protocol_refusal_limit

3 refusals per window

protocol_refusal_window

60000 milliseconds

account_decision_burst

5 starts

account_decision_ceiling

60 starts per minute

Errors and close codes

Read error.envelope and error.retry. Decision failures keep the ordinary HTTP status and error envelope. A rejected mutation leaves state, version and its next event ID unchanged. A fatal error requires a following close.

In on_change mode, a 429 or 503 other than maintenance, or HTTP 409 idempotency_conflict, with retry permitted by the HTTP response schedules the latest state again after its Retry-After delay.

The gateway emits an error frame for each failure and stops automatic retries at three consecutive retryable failures for the same version.

The third error has fatal: true and retains the HTTP retry fields.

It is followed by goodbye with reason overload, the Retry-After delay in resume.retry_after_ms, and close 1013 so the client reconnects and replays.

The same fatal outcome applies immediately when the next automatic retry would reach or exceed the stream's remaining lifetime.

On_request never retries itself.

Retaining the key does not guarantee recovery of an original reply after a dropped connection.

Retry-After-Ms takes precedence over Retry-After.

Honor goodbye.resume.retry_after_ms and any longer server retry delay. Use reconnect backoff. A goodbye is best effort; a transport close can arrive without it.

The server never sends 1006; clients can observe it after abrupt loss and reconnect with backoff.

Manual WebSocket

Illustrative JSON messages from a server runtime. Replace MODEL_ID and SERVED_BY with values from authenticated discovery. Timing, counts and probabilities illustrate the reply shape. In this exchange the server restarts after the decision, sends goodbye, then closes with code 1012. Wait at least 500 milliseconds before reconnecting.

{
  "type": "hello",
  "stream_id": "szws_0123456789abcdef0123456789abcdef",
  "seq": 1,
  "protocol": "system-zero.stream.v1",
  "request_id": "example-opening",
  "limits": {
    "max_duration_ms": 3600000,
    "idle_timeout_ms": 120000,
    "start_timeout_ms": 10000,
    "heartbeat_interval_ms": 30000,
    "native_heartbeat_interval_ms": 10000,
    "receipt_timeout_ms": 240000,
    "max_control_queue_frames": 2,
    "pong_timeout_ms": 10000,
    "max_message_bytes": 4194304,
    "max_decide_body_bytes": 3665920,
    "max_pending_events": 4096,
    "max_events": 100000,
    "max_events_per_second": 5,
    "max_send_queue_bytes": 4194304,
    "max_send_queue_frames": 64,
    "min_interval_ms": 0,
    "event_burst": 10,
    "max_messages_per_second": 40,
    "message_burst": 80,
    "max_streams_per_process": 16,
    "max_native_control_frames_per_second": 8,
    "native_control_frame_burst": 16,
    "max_native_control_bytes_per_second": 512,
    "native_control_byte_burst": 1024
  },
  "rate_limit": null
}

{
  "type": "start",
  "event_id": "evt_1",
  "request": {
    "model": "MODEL_ID",
    "state": "Initial log\n",
    "questions": {
      "ready": {
        "type": "boolean",
        "instructions": "Is the job ready?"
      }
    },
    "prompt_cache_key": "example-affinity"
  },
  "decision_mode": "on_request",
  "min_interval_ms": 0,
  "idempotency_key": "example-state-1"
}

{
  "type": "ack",
  "stream_id": "szws_0123456789abcdef0123456789abcdef",
  "seq": 2,
  "event_id": "evt_1",
  "state_version": 1,
  "state_bytes": 15,
  "synthesized_body_bytes": 158,
  "scheduled": false
}

{
  "type": "append_text",
  "event_id": "evt_2",
  "text": "Job is ready.\n",
  "idempotency_key": "example-state-2"
}

{
  "type": "ack",
  "stream_id": "szws_0123456789abcdef0123456789abcdef",
  "seq": 3,
  "event_id": "evt_2",
  "state_version": 2,
  "state_bytes": 30,
  "synthesized_body_bytes": 173,
  "scheduled": false
}

{
  "type": "decide",
  "state_version": 2
}

{
  "type": "scheduled",
  "stream_id": "szws_0123456789abcdef0123456789abcdef",
  "seq": 4,
  "state_version": 2,
  "in_flight": false
}

{
  "type": "ping",
  "stream_id": "szws_0123456789abcdef0123456789abcdef",
  "seq": 5,
  "nonce": 1
}

{
  "type": "pong",
  "nonce": 1
}

{
  "type": "decision",
  "stream_id": "szws_0123456789abcdef0123456789abcdef",
  "seq": 6,
  "state_version": 2,
  "covered_event_ids": [
    "evt_1",
    "evt_2"
  ],
  "reply": {
    "id": "szd_0123456789abcdef0123456789abcdef",
    "model": "MODEL_ID",
    "served_by": "SERVED_BY",
    "created": 1790000000,
    "context": {
      "route": "direct",
      "tokens": 32
    },
    "answers": {
      "ready": {
        "type": "boolean",
        "probability": 0.9,
        "confidence": 0.9,
        "calibration_version": "unpublished",
        "decision": true,
        "alternatives": [
          {
            "decision": false,
            "confidence": 0.1
          }
        ],
        "calibrated": false
      }
    },
    "usage": {
      "input_tokens": 32,
      "cached_input_tokens": 0,
      "media_tokens": 0,
      "long_input_tokens": 0,
      "media_items": 0,
      "thought_tokens": 0
    }
  },
  "request_id": "example-decision",
  "http_status": 200,
  "server_timing": null,
  "latency_ms": null,
  "rate_limit": null,
  "queue_ms": 0,
  "decide_ms": 40000
}

{
  "type": "goodbye",
  "stream_id": "szws_0123456789abcdef0123456789abcdef",
  "seq": 7,
  "reason": "shutdown",
  "state_version": 2,
  "last_decision_state_version": 2,
  "resume": {
    "supported": false,
    "action": "reconnect_and_replay",
    "retry_after_ms": 500
  }
}