Tool calling

Open models Tool calling Updated 2026-11-11

On this page

Minimal code
What one turn actually looks like
What to tune
Common mistakes
Next steps

Use tool calling whenever the model needs to act on the world: call an API, query a database, run a calculation, fetch a document.

Connection	Value
API base	`https://api.runinfra.ai/v1`
Workspace key	`rp_...`
Process variable	`RUNINFRA_GATEWAY_KEY`
Billing	The existing funding rules of your open-model account.

Replace MODEL_ID with an available id from Open models; read current capabilities, limits and prices from GET /v1/models.

Tool calling is a per-model capability. Check the capability listing on the model's page before sending tools. A model that does not list tool calling refuses the request with 400 hosted_capability_not_supported.

Minimal code

Python

import json
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.runinfra.ai/v1",
    api_key=os.environ["RUNINFRA_GATEWAY_KEY"],
)

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

def get_weather(city: str) -> dict:
    return {"city": city, "temp_c": 21, "conditions": "partly cloudy"}

messages = [{"role": "user", "content": "What's the weather in Paris?"}]

for _ in range(10):
    response = client.chat.completions.create(
        model="MODEL_ID",
        messages=messages,
        tools=tools,
        max_tokens=16384,
    )
    msg = response.choices[0].message
    messages.append(msg)

    if not msg.tool_calls:
        print(msg.content)
        break

    for call in msg.tool_calls:
        args = json.loads(call.function.arguments)
        result = get_weather(**args)
        messages.append({
            "role": "tool",
            "tool_call_id": call.id,
            "content": json.dumps(result),
        })

What one turn actually looks like

One tool-calling turn: you send messages plus tools, the model returns an assistant message carrying tool_calls, you push that whole assistant message and then one tool message per call, then send the grown array again; the turn with no tool_calls is the final answer.

One tool turn
repeat until tool_calls is absent
Messages you hold
01 Send
The model sees every tool description you registered.
user
02 Get
finish_reason is tool_calls; arguments arrive as a JSON string.
03 Push
Then one tool message per call, each with its tool_call_id.
user, assistant, tool
04 Send
The model now answers, or asks for another tool.
user, assistant, tool, ...
Your messages, plus tools
An assistant message with tool_calls
Push the whole assistant message back
The grown array, same call
Final answer
The turn with no tool_calls is the one you show the user.
Dropping the assistant turn at step 03 breaks the state: the tool result then has nothing to attach to.

What to tune

Parameter	Effect
`tool_choice: "auto"`	The model chooses when to call a tool. This is the default.
`tool_choice: "required"`	Where supported, force a tool call every turn, with no free-form answer. Some models refuse `tool_choice: "required"` with `400`, code `hosted_parameter_not_supported` and `param: "tool_choice"`. The message says what to send instead. Force a named tool there, or send `auto`.
`tool_choice: {type:"function", function:{name:"..."}}`	Force one specific tool.
`parallel_tool_calls: false`	Ask for one call at a time. Whether it is honored depends on the model.

On the model described in the model-specific source reference, forcing a tool or naming one also requires function.strict: true on every tool. Without it the model may answer without a call even when you require one. See the complete tool rules before changing the selected model.

Common mistakes

Dropping the assistant turn. Push the whole assistant message, tool_calls and all, before you push any tool result.
Returning a non-string tool result. The content on a tool-role message must be a string. Always serialize it.
A vague description. Write it as though the model has never seen your API: what the tool does, what each argument means, what a good input looks like. This is the single biggest lever on whether a tool gets called at all.
No turn limit. Bound the loop. Ten turns covers almost every pattern, and an unbounded loop can spend real money.
Trusting the arguments. The model can invent a field or omit a required one. Validate with Pydantic or Zod before you execute.

Tool calling

Minimal code

What one turn actually looks like

What to tune

Common mistakes

Next steps

Structured output

Streaming

Chat completions