# Providers

[Docs](https://www.rightnowai.co/docs) / [RightNow Agent](https://www.rightnowai.co/docs/agent) / [Providers](https://www.rightnowai.co/docs/providers)

Updated 2026-10-17

> **Release status**
>
> RightNow Agent, the terminal coding agent, is paused. Its guides stay online as reference.

With an existing installation, inspect the configured model and endpoint before choosing a provider.

## Select a model

**Text**

```text
rightnow doctor
```

Expected: the report includes any configured model and endpoint facts.

The model listing shows your credential source, default model and available model entry IDs.

The `-m` option selects a model entry ID.

Inside `[model.<id>]`, `model_provider` selects a provider ID. `model` names the upstream model.

## Shipped RunInfra entry

**TOML**

```toml
[models]
default = "runinfra"

[model_providers.runinfra]
base_url = "https://api.runinfra.ai/v1"
env_key = "RIGHTNOW_API_KEY"
supports_tool_choice_required = false
context_window = 1048576

[model.runinfra]
model = "qwen3-8-flash-next"
model_provider = "runinfra"
name = "RightNow (RunInfra)"
context_window = 1048576
auto_compact_threshold_percent = 85
```

Set `RIGHTNOW_API_KEY`. The `runinfra` preset also accepts `RUNINFRA_GATEWAY_KEY` as a fallback.

The template stores only the variable name, never the key. Check the [RunInfra key resolution](https://www.rightnowai.co/docs/authentication#runinfra-key-resolution-caption) when credentials overlap.

## Featured providers

**Featured providers**

| Provider | Provider ID | Key variable | Endpoint |
| --- | --- | --- | --- |
| RunInfra | runinfra | RIGHTNOW\_API\_KEY, then RUNINFRA\_GATEWAY\_KEY | https://api.runinfra.ai/v1 |
| Cerebras | cerebras | CEREBRAS\_API\_KEY | https://api.cerebras.ai/v1 |
| Groq | groq | GROQ\_API\_KEY | https://api.groq.com/openai/v1 |
| OpenRouter | openrouter | OPENROUTER\_API\_KEY | https://openrouter.ai/api/v1 |
| Vercel AI Gateway | vercel-ai-gateway | AI\_GATEWAY\_API\_KEY | https://ai-gateway.vercel.sh/v1 |

Point a model entry at the provider ID; the preset supplies the endpoint and key variable, so no `base_url` is needed:

TOML template Replace each &lt;model-id&gt;

**TOML template for featured providers**

```text
[model.cerebras-model]
model = "<model-id>"
model_provider = "cerebras"

[model.groq-model]
model = "<model-id>"
model_provider = "groq"

[model.openrouter-model]
model = "<model-id>"
model_provider = "openrouter"

[model.vercel-gateway-model]
model = "<model-id>"
model_provider = "vercel-ai-gateway"
```

These four presets set no context window. Set `context_window` in the model entry, or it resolves at 200000 tokens, which decides when compaction fires.

Startup skill listings keep a separate 6,000-character cap; some discovery announcements use a context-derived budget.

## Other providers

Direct Anthropic models use `ANTHROPIC_API_KEY` with `api_backend = "messages"`. Direct OpenAI models use `OPENAI_API_KEY` with `api_backend = "responses"`.

The `anthropic` and `openai` presets set the endpoint, key variable, and backend for you.

For any other OpenAI-compatible endpoint, define a provider with its base URL, key variable, context window, and capability settings, then point model entries at it:

TOML template Replace each &lt;model-id&gt;

**TOML template for a custom provider**

```text
[model_providers.my-provider]
base_url = "https://api.example.com/v1"
env_key = "PROVIDER_API_KEY"
context_window = 128000
supports_tool_choice_required = false

[model.first]
model = "<model-id>"
model_provider = "my-provider"
```

Local inference is not a product capability: the 1.0.4 release removed local runtime paths, local weights, GPU inference, and local embeddings. The keyless `ollama`, `lmstudio`, and `llamacpp` preset IDs still parse but are not supported.

### API backends

**API backends**

| Value | Protocol |
| --- | --- |
| chat\_completions | OpenAI Chat Completions; default |
| responses | OpenAI Responses |
| messages | Anthropic Messages |
| gemini | Google Gemini generateContent |

### Provider and model fields

**\[model\_providers.&lt;id&gt;\]**

Provides shared defaults to referenced model entries.

- `base_url`
- `api_base_url`
- `env_key`
- `api_key`
- `api_backend`
- `auth_scheme`
- `keyless`
- `extra_headers`
- `query_params`
- `env_http_headers`
- `auth_provider`
- `auth`
- `context_window`
- `supports_tool_choice_required`
- `supports_tools_with_response_format`

**\[model.&lt;id&gt;\]**

- Model scalars override provider defaults.
- Nonempty header maps replace provider maps.
- Blank query values are filtered before a nonempty model query map replaces the provider map.
- When no model API key is set, environment key lists merge with model names first.

- `model`
- `model_provider`
- `base_url`
- `api_base_url`
- `name`
- `description`
- `env_key`
- `api_key`
- `auth_provider`
- `api_backend`
- `auth_scheme`
- `keyless`
- `temperature`
- `top_p`
- `max_completion_tokens`
- `max_retries`
- `inference_idle_timeout_secs`
- `context_window`
- `auto_compact_threshold_percent`
- `extra_headers`
- `query_params`
- `env_http_headers`
- `reasoning_effort`
- `reasoning_efforts`
- `supports_reasoning_effort`
- `supports_backend_search`
- `supports_tool_choice_required`
- `supports_tools_with_response_format`
- `use_concise`
- `agent_type`
- `hidden`
- `supported_in_api`
- `compactions_remaining`
- `compaction_at_tokens`
- `show_model_fingerprint`
- `stream_tool_calls`
- `system_prompt_label`

Keep secrets out of `query_params`. Although `api_key` is accepted, it stores the secret in `config.toml`; prefer `env_key` or the credential store. Use `env_http_headers` to map a request header to an environment variable without writing its value to disk.

## Custom model catalog

`RIGHTNOW_MODELS_BASE_URL` selects an OpenAI-compatible base for inference and model discovery; `RIGHTNOW_MODELS_LIST_URL` selects a different discovery URL. The config-file equivalents are `[endpoints] models_base_url` and `models_list_url`.

- With a custom catalog endpoint, model resolution uses configured model entries and models returned by that endpoint; it does not append the built-in catalog.
- Custom catalog discovery first tries the static key reader, which checks `RIGHTNOW_API_KEY`, `RUNINFRA_GATEWAY_KEY`, then the legacy aliases. It falls back to an available API-key credential supplied to the fetch.
- A fresh cached catalog can supply models at startup. A failed refresh leaves existing models unchanged. With a custom endpoint and no fetched or cached catalog, only configured model entries remain.

If selection fails, run `rightnow doctor` and use the [configuration guide](https://www.rightnowai.co/docs/configuration) to check the active file.

## Next step

Use your selected model for a first task in the [Quickstart](https://www.rightnowai.co/docs/quickstart).

## Previous

[Configuration](https://www.rightnowai.co/docs/configuration)

## Next

[Permissions and Safety](https://www.rightnowai.co/docs/permissions)