# Tool calling

[Open models](https://www.rightnowai.co/docs/open-models) / [Tool calling](https://www.rightnowai.co/docs/open-models/guides/tool-calling)

Updated 2026-11-11

Use tool calling whenever the model needs to act on the world: call an API, query a database, run a calculation, fetch a document.

**Connection**

| Connection | Value |
| --- | --- |
| API base | `https://api.runinfra.ai/v1` |
| Workspace key | `rp_...` |
| Process variable | `RUNINFRA_GATEWAY_KEY` |
| Billing | The existing funding rules of your open-model account. |

Replace `MODEL_ID` with an available id from [Open models](https://runinfra.ai/inference-api?utm_source=rightnowai.co&utm_medium=referral&utm_content=docs-open-models); read current capabilities, limits and prices from `GET /v1/models`.

Tool calling is a per-model capability. Check the capability listing on the model's page before sending `tools`. A model that does not list tool calling refuses the request with `400` `hosted_capability_not_supported`.

## Minimal code

**Python**

```python
import json
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.runinfra.ai/v1",
    api_key=os.environ["RUNINFRA_GATEWAY_KEY"],
)

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

def get_weather(city: str) -> dict:
    return {"city": city, "temp_c": 21, "conditions": "partly cloudy"}

messages = [{"role": "user", "content": "What's the weather in Paris?"}]

for _ in range(10):
    response = client.chat.completions.create(
        model="MODEL_ID",
        messages=messages,
        tools=tools,
        max_tokens=16384,
    )
    msg = response.choices[0].message
    messages.append(msg)

    if not msg.tool_calls:
        print(msg.content)
        break

    for call in msg.tool_calls:
        args = json.loads(call.function.arguments)
        result = get_weather(**args)
        messages.append({
            "role": "tool",
            "tool_call_id": call.id,
            "content": json.dumps(result),
        })
```

**TypeScript**

```typescript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.runinfra.ai/v1",
  apiKey: process.env.RUNINFRA_GATEWAY_KEY,
});

const tools = [{
  type: "function" as const,
  function: {
    name: "get_weather",
    description: "Get the current weather for a city",
    parameters: {
      type: "object",
      properties: { city: { type: "string" } },
      required: ["city"],
    },
  },
}];

function getWeather(city: string) {
  return { city, temp_c: 21, conditions: "partly cloudy" };
}

const messages: any[] = [
  { role: "user", content: "What's the weather in Paris?" },
];

for (let turn = 0; turn < 10; turn += 1) {
  const response = await client.chat.completions.create({
    model: "MODEL_ID",
    messages,
    tools,
    max_tokens: 16384,
  });
  const msg = response.choices[0].message;
  messages.push(msg);

  if (!msg.tool_calls?.length) { console.log(msg.content); break; }

  for (const call of msg.tool_calls) {
    if (call.type !== "function") continue;
    const args = JSON.parse(call.function.arguments);
    const result = getWeather(args.city);
    messages.push({
      role: "tool",
      tool_call_id: call.id,
      content: JSON.stringify(result),
    });
  }
}
```

## What one turn actually looks like

> One tool-calling turn: you send messages plus tools, the model returns an assistant message carrying tool\_calls, you push that whole assistant message and then one tool message per call, then send the grown array again; the turn with no tool\_calls is the final answer.
>
> - One tool turn
> - repeat until tool\_calls is absent
> - Messages you hold
> - 01 Send
> - The model sees every tool description you registered.
> - user
> - 02 Get
> - finish\_reason is tool\_calls; arguments arrive as a JSON string.
> - 03 Push
> - Then one tool message per call, each with its tool\_call\_id.
> - user, assistant, tool
> - 04 Send
> - The model now answers, or asks for another tool.
> - user, assistant, tool, ...
> - Your messages, plus tools
> - An assistant message with tool\_calls
> - Push the whole assistant message back
> - The grown array, same call
> - Final answer
> - The turn with no tool\_calls is the one you show the user.
> - Dropping the assistant turn at step 03 breaks the state: the tool result then has nothing to attach to.

## What to tune

**Reference 133**

| Parameter | Effect |
| --- | --- |
| `tool_choice: "auto"` | The model chooses when to call a tool. This is the default. |
| `tool_choice: "required"` | Where supported, force a tool call every turn, with no free-form answer. Some models refuse `tool_choice: "required"` with `400`, code `hosted_parameter_not_supported` and `param: "tool_choice"`. The message says what to send instead. Force a named tool there, or send `auto`. |
| `tool_choice: {type:"function", function:{name:"..."}}` | Force one specific tool. |
| `parallel_tool_calls: false` | Ask for one call at a time. Whether it is honored depends on the model. |

On the model described in the [model-specific source reference](https://runinfra.ai/docs/api-reference/models?utm_source=rightnowai.co&utm_medium=referral&utm_content=docs-open-models), forcing a tool or naming one also requires `function.strict: true` on every tool. Without it the model may answer without a call even when you require one. See the [complete tool rules](https://www.rightnowai.co/docs/open-models/chat-completions#model-specific-tool-and-json-rules) before changing the selected model.

## Common mistakes

- **Dropping the assistant turn.** Push the whole assistant message, `tool_calls` and all, before you push any tool result.
- **Returning a non-string tool result.** The `content` on a tool-role message must be a string. Always serialize it.
- **A vague `description`.** Write it as though the model has never seen your API: what the tool does, what each argument means, what a good input looks like. This is the single biggest lever on whether a tool gets called at all.
- **No turn limit.** Bound the loop. Ten turns covers almost every pattern, and an unbounded loop can spend real money.
- **Trusting the arguments.** The model can invent a field or omit a required one. Validate with Pydantic or Zod before you execute.

## Next steps

[### Structured output When you want JSON back without a tool loop.](https://www.rightnowai.co/docs/open-models/guides/structured-output) [### Streaming Stream the assistant turn as it arrives.](https://www.rightnowai.co/docs/open-models/guides/streaming) [### Chat completions The full request contract.](https://www.rightnowai.co/docs/open-models/chat-completions)

[Source reference](https://runinfra.ai/docs/cookbook/tool-calling?utm_source=rightnowai.co&utm_medium=referral&utm_content=docs-open-models)

## Next

[Structured output](https://www.rightnowai.co/docs/open-models/guides/structured-output)