Providers

Docs RightNow Agent Providers Updated 2026-10-17

Release status

RightNow Agent, the terminal coding agent, is paused. Its guides stay online as reference.

With an existing installation, inspect the configured model and endpoint before choosing a provider.

Select a model

Text

rightnow doctor

Expected: the report includes any configured model and endpoint facts.

The model listing shows your credential source, default model and available model entry IDs.

The -m option selects a model entry ID.

Inside [model.<id>], model_provider selects a provider ID. model names the upstream model.

Shipped RunInfra entry

TOML

[models]
default = "runinfra"

[model_providers.runinfra]
base_url = "https://api.runinfra.ai/v1"
env_key = "RIGHTNOW_API_KEY"
supports_tool_choice_required = false
context_window = 1048576

[model.runinfra]
model = "qwen3-8-flash-next"
model_provider = "runinfra"
name = "RightNow (RunInfra)"
context_window = 1048576
auto_compact_threshold_percent = 85

Set RIGHTNOW_API_KEY. The runinfra preset also accepts RUNINFRA_GATEWAY_KEY as a fallback.

The template stores only the variable name, never the key. Check the RunInfra key resolution when credentials overlap.

Featured providers

Provider	Provider ID	Key variable	Endpoint
RunInfra	ID: runinfra	Key: RIGHTNOW_API_KEY, then RUNINFRA_GATEWAY_KEY	Endpoint: https://api.runinfra.ai/v1
Cerebras	ID: cerebras	Key: CEREBRAS_API_KEY	Endpoint: https://api.cerebras.ai/v1
Groq	ID: groq	Key: GROQ_API_KEY	Endpoint: https://api.groq.com/openai/v1
OpenRouter	ID: openrouter	Key: OPENROUTER_API_KEY	Endpoint: https://openrouter.ai/api/v1
Vercel AI Gateway	ID: vercel-ai-gateway	Key: AI_GATEWAY_API_KEY	Endpoint: https://ai-gateway.vercel.sh/v1

Point a model entry at the provider ID; the preset supplies the endpoint and key variable, so no base_url is needed:

TOML templateReplace each <model-id>

[model.cerebras-model]
model = "<model-id>"
model_provider = "cerebras"

[model.groq-model]
model = "<model-id>"
model_provider = "groq"

[model.openrouter-model]
model = "<model-id>"
model_provider = "openrouter"

[model.vercel-gateway-model]
model = "<model-id>"
model_provider = "vercel-ai-gateway"

These four presets set no context window. Set context_window in the model entry, or it resolves at 200000 tokens, which decides when compaction fires.

Startup skill listings keep a separate 6,000-character cap; some discovery announcements use a context-derived budget.

Other providers

Direct Anthropic models use ANTHROPIC_API_KEY with api_backend = "messages". Direct OpenAI models use OPENAI_API_KEY with api_backend = "responses".

The anthropic and openai presets set the endpoint, key variable, and backend for you.

For any other OpenAI-compatible endpoint, define a provider with its base URL, key variable, context window, and capability settings, then point model entries at it:

TOML templateReplace each <model-id>

[model_providers.my-provider]
base_url = "https://api.example.com/v1"
env_key = "PROVIDER_API_KEY"
context_window = 128000
supports_tool_choice_required = false

[model.first]
model = "<model-id>"
model_provider = "my-provider"

Local inference is not a product capability: the 1.0.4 release removed local runtime paths, local weights, GPU inference, and local embeddings. The keyless ollama, lmstudio, and llamacpp preset IDs still parse but are not supported.

API backends

Value	Protocol
chat_completions	OpenAI Chat Completions; default
responses	OpenAI Responses
messages	Anthropic Messages
gemini	Google Gemini generateContent

Provider and model fields

[model_providers.<id>]: Provides shared defaults to referenced model entries.
base_url
api_base_url
env_key
api_key
api_backend
auth_scheme
keyless
extra_headers
query_params
env_http_headers
auth_provider
auth
context_window
supports_tool_choice_required
supports_tools_with_response_format
[model.<id>]: Model scalars override provider defaults.
Nonempty header maps replace provider maps.
Blank query values are filtered before a nonempty model query map replaces the provider map.
When no model API key is set, environment key lists merge with model names first.
model
model_provider
base_url
api_base_url
name
description
env_key
api_key
auth_provider
api_backend
auth_scheme
keyless
temperature
top_p
max_completion_tokens
max_retries
inference_idle_timeout_secs
context_window
auto_compact_threshold_percent
extra_headers
query_params
env_http_headers
reasoning_effort
reasoning_efforts
supports_reasoning_effort
supports_backend_search
supports_tool_choice_required
supports_tools_with_response_format
use_concise
agent_type
hidden
supported_in_api
compactions_remaining
compaction_at_tokens
show_model_fingerprint
stream_tool_calls
system_prompt_label

Keep secrets out of query_params. Although api_key is accepted, it stores the secret in config.toml; prefer env_key or the credential store. Use env_http_headers to map a request header to an environment variable without writing its value to disk.

Custom model catalog

RIGHTNOW_MODELS_BASE_URL selects an OpenAI-compatible base for inference and model discovery; RIGHTNOW_MODELS_LIST_URL selects a different discovery URL. The config-file equivalents are [endpoints] models_base_url and models_list_url.

With a custom catalog endpoint, model resolution uses configured model entries and models returned by that endpoint; it does not append the built-in catalog.
Custom catalog discovery first tries the static key reader, which checks RIGHTNOW_API_KEY, RUNINFRA_GATEWAY_KEY, then the legacy aliases. It falls back to an available API-key credential supplied to the fetch.
A fresh cached catalog can supply models at startup. A failed refresh leaves existing models unchanged. With a custom endpoint and no fetched or cached catalog, only configured model entries remain.

If selection fails, run rightnow doctor and use the configuration guide to check the active file.

Next step

Use your selected model for a first task in the Quickstart.