Providers

Select a model

rightnow models

It prints the credential in use, the default model, and the model-entry IDs your configuration and credentials make available.

The -m option selects a model-entry ID. Inside [model.<id>], model_provider selects a provider ID, and model names the upstream model.

Shipped RunInfra entry

[models]
default = "runinfra"

[model_providers.runinfra]
base_url = "https://api.runinfra.ai/v1"
env_key = "RIGHTNOW_API_KEY"
supports_tool_choice_required = false
context_window = 1048576

[model.runinfra]
model = "qwen3-8-flash-next"
model_provider = "runinfra"
name = "RightNow (RunInfra)"
context_window = 1048576
auto_compact_threshold_percent = 85

Set RIGHTNOW_API_KEY; the runinfra preset also accepts RUNINFRA_GATEWAY_KEY as a fallback. The template stores only the variable name, never the key.

Featured providers

Point a model entry at the provider ID; the preset supplies the endpoint and key variable, so no base_url is needed:

[model.cerebras-model]
model = "<model-id>"
model_provider = "cerebras"

[model.groq-model]
model = "<model-id>"
model_provider = "groq"

[model.openrouter-model]
model = "<model-id>"
model_provider = "openrouter"

[model.vercel-gateway-model]
model = "<model-id>"
model_provider = "vercel-ai-gateway"

These four presets set no context window. Set context_window in the model entry, or it resolves at 200000 tokens, which decides when compaction fires. Startup skill listings keep a separate 6,000-character cap; some discovery announcements use a context-derived budget.

Other providers

Direct Anthropic models use ANTHROPIC_API_KEY with api_backend = "messages". Direct OpenAI models use OPENAI_API_KEY with api_backend = "responses". The anthropic and openai presets set the endpoint, key variable, and backend for you.

For any other OpenAI-compatible endpoint, define a provider with its base URL, key variable, context window, and capability settings, then point model entries at it:

[model_providers.my-provider]
base_url = "https://api.example.com/v1"
env_key = "PROVIDER_API_KEY"
context_window = 128000
supports_tool_choice_required = false

[model.first]
model = "<model-id>"
model_provider = "my-provider"

API backends

Provider and model fields

[model_providers.<id>]

Provides shared defaults to referenced model entries.

• base_url
• api_base_url
• env_key
• api_key
• api_backend
• auth_scheme
• keyless
• extra_headers
• query_params
• env_http_headers
• auth_provider
• auth
• context_window
• supports_tool_choice_required
• supports_tools_with_response_format

[model.<id>]

Model scalars override provider defaults; non-empty header maps replace provider maps; blank query values are filtered before a non-empty model query map replaces the provider map; when no model API key is set, environment-key lists merge with model names first.

• model
• model_provider
• base_url
• api_base_url
• name
• description
• env_key
• api_key
• auth_provider
• api_backend
• auth_scheme
• keyless
• temperature
• top_p
• max_completion_tokens
• max_retries
• inference_idle_timeout_secs
• context_window
• auto_compact_threshold_percent
• extra_headers
• query_params
• env_http_headers
• reasoning_effort
• reasoning_efforts
• supports_reasoning_effort
• supports_backend_search
• supports_tool_choice_required
• supports_tools_with_response_format
• use_concise
• agent_type
• hidden
• supported_in_api
• compactions_remaining
• compaction_at_tokens
• show_model_fingerprint
• stream_tool_calls
• system_prompt_label

Keep secrets out of query_params. Although api_key is accepted, it stores the secret in config.toml; prefer env_key or the credential store. Use env_http_headers to map a request header to an environment variable without writing its value to disk.

Custom model catalog

RIGHTNOW_MODELS_BASE_URL selects an OpenAI-compatible base for inference and model discovery; RIGHTNOW_MODELS_LIST_URL selects a different discovery URL. The config-file equivalents are [endpoints] models_base_url and models_list_url.

• With a custom catalog endpoint, model resolution uses configured model entries and models returned by that endpoint; it does not append the built-in catalog.
• Discovery does not read RIGHTNOW_API_KEY. If discovery fails, only your configured model entries are available, so give each one its own env_key or model_provider.

If selection fails, run rightnow doctor and use the configuration guide to check the active file.