Text caching

RightNow AI Decision model Text caching Updated 2026-11-12

On this page

How reuse works
Cached and uncached prices
Read the text cache hit rate
Usage fields and coverage

A cached text input token is a text token whose earlier processing the RightNow decision model reuses.

How reuse works

Text caching is automatic. An optional prompt_cache_key improves reuse within a workspace. Cache lifetime, minimum size and eviction are not published.

Keep shared leading text identical and reuse a stable prompt_cache_key for related work. The hint does not guarantee a cache hit.

Text caching is isolated by workspace. API keys in the same workspace share a cache partition.

Replaying a stored answer is different from reusing text processing. A stored replay adds no new input or cache usage to the totals.

Keep the leading text identical
API keys in the same workspace share a cache.
Send the same prompt_cache_key
Add prompt_cache_key beside model in your request body. See API reference, Request fields.
Optional, length 1 to 256. It improves reuse and does not guarantee a hit.
Read usage.cached_input_tokens
The reused part of the prompt, including any cached media positions.

Uncached: USD 0.19 per million tokens
Cached: USD 0.019 per million tokens
38,000 tokens, uncached: USD 0.00722
38,000 tokens, cached: USD 0.000722

How to get cached text input, and what it costs.

Cached and uncached prices

Input	Price per million input tokens
Uncached text input	USD 0.19
Cached text input	USD 0.019

Each million cached text tokens saves USD 0.171 at these prices. This is a saving on text input, not a discount on the whole bill.

See Billing and credits for complete charges.

Read the text cache hit rate

Open Usage on your workspace's Overview, select a period and key, and read Cache hit rate. Requests by key shows each key's text cache hit rate for the selected period.

Add the counts before dividing. Text input is total input_tokens minus media_tokens and cached_media_tokens. Cached text is total cached_input_tokens minus cached_media_tokens.

Express cached text divided by text input as a percentage. A selection can include media when every row has all required measured counts.

Combine the token counts before calculating the percentage. Do not average the percentages of separate calls.

Date selections include today and use UTC bounds. Small positive cache hit rates are shown as less than the display threshold.

No data means a missing, invalid or inconsistent count, or no text input. Zero percent means measured text input with no cached text.

Savings use the selected period's cached text tokens at current prices. They do not replace the billed total.

Usage fields and coverage

The console reads input_tokens, cached_input_tokens, media_tokens and cached_media_tokens from its usage data.

cache reading

Text

[ Send complete request ]
           |
           v
[ Read returned usage ]
           |
           v
[ Separate cached text ]
[ from fresh text      ]
           |
           v
[ Apply each text rate ]

Illustration of text caching.

Read usage to measure reuse. A routing hint cannot guarantee cached input.

Field	Meaning
`input_tokens`	Total counted input, including reused input and media positions.
`cached_input_tokens`	The reused part of the prompt, including any cached media positions.
`media_tokens`	Uncached media positions, not files, seconds or pages.
`cached_media_tokens`	Media positions served from cache.

Older usage can return null for cached_media_tokens when any included hour was recorded before that count was measured. Missing or invalid counts make the rate unavailable for the whole selection.

Recent completed work can take time to appear. Failed requests add no tokens to these totals; successful requests with refused questions can still report measured tokens.

If the read fails, use Try again. A failed refresh keeps the last available data only for the same selection.