Inputs

Text input

Send text in state as a string, a JSON object or a JSON array. Standard text decisions support boolean, choice and score questions.

JSON state can describe robot state, for example {"battery_percent":80,"moving":false}. Send code as text in state.

To ask about a document's text, put that text in state without an attachment.

Attachment names

Each attachment name must match ^[A-Za-z][A-Za-z0-9_-]{0,63}$. A dotted name returns 400 invalid_request; use invoice_pdf instead of invoice.pdf.

Upload references are not available to outside callers today and return 400 media_unsupported at attachments[i].upload. Send files inline.

Image input

Send files inline. Only the listed formats are available. Request body limits still apply.

Formats

image/png, image/jpeg

Profile

Static PNG, including interlaced PNG, and JPEG.

Source

data

File rate

USD 0.003 per image

Minimum per file

USD 0.003

Maximum measure

16,777,216 source pixels

Prepare static PNG or JPEG files. Export one still from an animated file. Rotate the pixels upright before export and inspect the saved file; do not rely on orientation metadata.

Images per request

4

Source pixels per image

16,777,216

Maximum aspect ratio

50:1

Use positive width and height. Multiply them to check source pixels; divide the longer side by the shorter to check ratio. These are source acceptance limits.

All files share 2,621,440 decoded bytes. Check their combined size after export. If needed, resize or compress them yourself, then inspect again. Your inline input is not silently resized to fit these limits.

Send canonical base64 in an inline data URL. Base64 uses four characters per three file bytes, rounded up. Leave room for state and questions within the 3,665,920-byte JSON request limit.

To compare two images, save your prepared files as reference.png and observed.png. Run this Python example to print the request JSON, then send it to /v1/system-zero/decide with your server API key.

import base64
import json
from pathlib import Path

request = {
    "model": "rightnow-decision",
    "state": (
        "Compare reference and observed. Use the attachment names to identify e"
        "ach image."
    ),
    "attachments": [
        {
            "name": "reference",
            "type": "image",
            "data": "reference.png"
        },
        {
            "name": "observed",
            "type": "image",
            "data": "observed.png"
        }
    ],
    "questions": {
        "match": {
            "type": "choice",
            "instructions": "Does observed show the same item as reference?",
            "criteria": {
                "same": "The same item is visible in both images.",
                "different": "The images show different items.",
                "unreadable": "You cannot determine whether the items match."
            }
        }
    }
}
for attachment in request["attachments"]:
    path = Path(attachment["data"])
    is_jpeg = path.suffix.lower() in (".jpg", ".jpeg")
    mime = "image/jpeg" if is_jpeg else "image/png"
    encoded = base64.b64encode(path.read_bytes()).decode("ascii")
    attachment["data"] = f"data:{mime};base64,{encoded}"
print(json.dumps(request))

For OCR value verification, save your receipt as receipt.png. Ask whether its payment status reads PAID and include an unreadable option. This returns a choice, not a transcription.

import base64
import json
from pathlib import Path

request = {
    "model": "rightnow-decision",
    "state": (
        "Check the printed payment status on receipt against the expected value"
        " PAID."
    ),
    "attachments": [
        {
            "name": "receipt",
            "type": "image",
            "data": "receipt.png"
        }
    ],
    "questions": {
        "status": {
            "type": "choice",
            "instructions": (
                "Does the printed payment status on receipt read PAID?"
            ),
            "criteria": {
                "matches": "The status reads PAID.",
                "differs": "The status is readable and differs from PAID.",
                "unreadable": "The status is missing or cannot be read."
            }
        }
    }
}
for attachment in request["attachments"]:
    path = Path(attachment["data"])
    is_jpeg = path.suffix.lower() in (".jpg", ".jpeg")
    mime = "image/jpeg" if is_jpeg else "image/png"
    encoded = base64.b64encode(path.read_bytes()).decode("ascii")
    attachment["data"] = f"data:{mime};base64,{encoded}"
print(json.dumps(request))

Each new answered snapshot charges USD 0.003 per image: the comparison includes 2 image fees and the receipt includes one. Add USD 0.0001 per answered question and USD 0.00001 per answered call, with residual text charged separately.

Audio input

Audio: one WAV file with PCM or float samples at exactly 16,000 Hz per request. The WAV file is the only attachment and must fit the 2.5 MiB decoded limit. Channels and sample encoding affect how much audio fits. Audio is limited to 60 seconds per request.

Send files inline. Only the listed formats are available. Request body limits still apply.

Formats

audio/wav

Profile

Send inline RIFF WAV with integer PCM or IEEE float samples at exactly 16,000 Hz. Mono and multichannel files are accepted; pricing counts sample frames, not channels. Other audio formats, other sample rates, unknown audio MIME types and audio upload references are unavailable. Refused audio is not processed or charged and creates no billing receipt.

Source

data

File rate

USD 0.0002 per started second (about USD 0.012 per minute)

Minimum per file

30 seconds

Maximum measure

60 seconds, 1 pass

The byte and duration limits both apply. At 16,000 Hz, use the lower limit.

These estimates divide the byte budget by sample rate, bytes per sample and channels. They also apply the duration bound. WAV headers use part of that budget. Check the final file size. Send the WAV file without other attachments.

Convert your recording explicitly to 16,000 Hz, mono, 16-bit PCM WAV:

ffmpeg -i recording.wav -ar 16000 -ac 1 -c:a pcm_s16le clip.wav

Listen to clip.wav and check its bytes before encoding it as data:audio/wav;base64,.... Split a longer recording into files that each fit a separate request.

Each answered audio file has a 30-second billing minimum of USD 0.006, even for a shorter clip. Longer clips are billed per started second. Text, answered questions and the answered call fee are separate.

Keep each recording within the published duration and byte limits. Audio over the decoded byte budget answers 413 media_too_large.

Document input

Send files inline. Only the listed formats are available. Request body limits still apply.

Formats

application/pdf

Profile

Unencrypted PDF only.

Source

data

File rate

USD 0.002 per page

Minimum per file

USD 0.001

Maximum measure

500 PDF pages

Send PDF files inline. Answers use readable text; pictures may be omitted. For a scan with no readable text, send extracted text in state. A scan whose text can't be read may not be answered reliably. Other native document formats are unavailable; send their extracted text in state.

Text, office and archive documents return 415 document_unsupported, including code sent as a text document and ZIP archives. Send PDF inline or put extracted text in state.

Send PDF files inline. Document upload references are not available to outside callers today (400 media_unsupported at attachments[i].upload).

Up to 500 pages per PDF; 2,621,440 decoded bytes per request.

Encrypted PDFs are not supported. Export an unencrypted PDF and send it again.

Answers use the PDF text and include a note when pictures were left out.

PDF is supported. Export this document to PDF and send the PDF file.

PDF file fees count physical pages. Text tokens and answered questions are charged separately.

Video input (preview)

Send one silent H.264 MP4 video inline.

Format

H.264 MP4

Duration

Up to 10 seconds

Dimensions

Up to 1920 x 1080

File size

Up to 2.5 MiB

Attachments

One video only

Send video without other files.

Soundtracks and rotated videos are not supported.

The complete JSON request must fit 3,665,920 bytes, including base64 and questions.

Video costs USD 0.0008 per started second, with a USD 0.008 minimum per clip; clips up to 10 seconds. Each answered question costs USD 0.0001.

How refusals work

Send files inline.

[ Prepare allowed files ]
            |
            v
[ Check format and count ]
            |
            v
[ Check decoded file bytes ]
            |
            v
[ Encode inline data ]
            |
            v
[ Check complete JSON size ]
            |
            v
[ Send decide request ]

An unavailable input type is refused in full. These availability refusals use error.type: not_found_error, error.param: attachments[i] for the zero-based attachment index i and error.retryable: false, with no Retry-After header. For size and format refusals, see Errors.

The RightNow decision model checks attachment class availability before validating attachment contents. Authentication, request body limits and JSON parsing still run first.