AI tool calling in production.

What a production tool contract contains

A tool is an API whose caller is a probabilistic model, so its contract has to carry more than a name and a JSON Schema. The model will sometimes pick the wrong tool, invent an argument, or call the same thing twice. The contract decides what happens next.

Name

A narrow verb and object, such as create_draft_invoice, never a generic call_api(method, path, body).

Purpose

One sentence on what it does and one on when not to use it.

Arguments

Typed, with enums for closed sets. Nothing the code already knows (tenant, user, current record) is a model argument.

Side-effect class

Read, reversible write, or irreversible or external effect. This drives retries and approvals.

Idempotency

How the key is derived, and where the outcome is stored.

Timeout

A per-tool limit, inside an overall run budget.

Result

IDs plus the few fields the next step needs, with a size cap and a pagination cursor.

Errors

A machine-readable code, a plain message, a hint for what to do next, and whether a retry can help.

OpenAI’s function calling guide puts one of these rules plainly: don’t make the model fill arguments you already know. If the session already has an order ID, submit_refund() takes no ID and your code supplies it. Every argument you remove is one the model cannot get wrong.

Turn on strict schemas and keep the tool list short

Strict mode makes tool arguments conform to the schema during generation instead of on a best-effort basis. As documented on 19 October 2026:

• OpenAI recommends always enabling strict: true. If you omit it, the Responses API tries to normalise your schema to strict and silently falls back to best effort when it cannot; the response then shows strict: false, so assert on it in tests. Strict mode inherits the structured output schema rules: every field required, additionalProperties: false.
• Anthropic supports strict: true on custom tools, with a combined budget per request of 20 strict tools, 24 optional parameters and 16 union-typed parameters.

Tool count is a quality setting, not just a cost. OpenAI suggests fewer than 20 functions available at the start of a turn, with tool search to defer the rest, and notes that definitions are injected into the context and billed as input tokens. Keep definitions byte-stable between requests so prompt caching keeps working, and version them like any API.

Classify side effects before you design retries and approvals

The side-effect class of a tool decides how it may be retried and who must approve it. Get this table agreed with the business owner before any code is written; it is the core of the permission model.

In the systems we run, the agent can draft almost anything, and a person or a deterministic rule releases anything that leaves the building. That split keeps the agent useful while the expensive mistakes stay behind a human decision.

Which failures are safe to retry

A timeout on a write is ambiguous: the call may have succeeded after your client gave up. Retrying it without a key is how customers get two emails and suppliers get paid twice.

Transport retries belong in your code, not the model’s reasoning. Model-level retries are for errors the model can fix, such as a malformed date. Anthropic’s documentation notes that Claude retries an invalid tool request two to three times with corrections, so cap attempts in your loop rather than relying on it to stop. Broader patterns for partial failure are in our guide to agent retry and failure recovery.

An idempotent tool wrapper

The wrapper below derives a key from the run, the tool and the canonical arguments, records a pending state before executing, and replays the stored outcome on repeats. Pass the key downstream when the target API accepts an idempotency header; otherwise store it in a reference field and look it up before writing.

import hashlib, json, time
from dataclasses import dataclass, asdict

MAX_RESULT_CHARS = 8000

@dataclass
class ToolError:
    code: str          # "not_found", "rate_limited", "in_doubt", "needs_approval"
    message: str       # what went wrong, in plain words
    hint: str          # what the model should do next
    retryable: bool

def idem_key(run_id: str, tool: str, args: dict) -> str:
    canonical = json.dumps(args, sort_keys=True, separators=(",", ":"))
    return hashlib.sha256(f"{run_id}|{tool}|{canonical}".encode()).hexdigest()

def run_tool(store, run_id, tool, args, side_effect, fn, timeout_s=20):
    key = idem_key(run_id, tool, args)
    prior = store.get(key)
    if prior and prior["state"] == "done":
        return prior["result"]                         # repeat call: replay, never re-execute
    if prior and prior["state"] == "pending" and side_effect != "read":
        return asdict(ToolError("in_doubt", "An earlier attempt may have completed.",
                                "Call get_action_status with this key first.", False))
    store.put(key, {"state": "pending", "started": time.time()})
    try:
        result = fn(**args, idempotency_key=key, timeout=timeout_s)
    except TimeoutError:
        return asdict(ToolError("timeout", f"{tool} did not respond in {timeout_s}s.",
                                "Check status before retrying.", side_effect == "read"))
    text = json.dumps(result, default=str)
    if len(text) > MAX_RESULT_CHARS:
        text = text[:MAX_RESULT_CHARS] + ' [truncated: narrow the query or use next_cursor]'
    store.put(key, {"state": "done", "result": text})
    return text

The error shape matters as much as the success path. Anthropic’s tool documentation recommends instructive errors (“Rate limit exceeded. Retry after 60 seconds.”) returned with is_error: true, because a bare “failed” gives the model nothing to recover with.

Parallel calls, timeouts and result size

Allow parallel calls for reads and serialise writes. Both major APIs let you turn parallelism off: parallel_tool_calls: false on OpenAI ensures zero or one call per turn, and disable_parallel_tool_use does the same on Claude. Two writes issued in the same turn have no defined order, and a later one may depend on an earlier one’s result.

Set a timeout per tool and a budget per run: maximum steps, maximum wall-clock time and maximum spend. A run that hits its budget stops and reports where it got to; it does not keep calling.

Keep results small. Return identifiers, the fields the next step needs and a cursor, not whole records or raw HTML. Large results crowd the context, raise cost and carry more untrusted text. Every tool result is untrusted input to the next model call, which is why tool outputs are the main entry point for indirect prompt injection.

Failure modes seen in production and how to detect them

Most tool-calling failures are visible in traces if you log the right fields: tool name, canonical arguments, idempotency key, side-effect class, outcome and latency per call.

The last row is OWASP’s ASI02 Tool Misuse and Exploitation from the Top 10 for Agentic Applications 2026: an agent acting within its granted privileges but applying a legitimate tool unsafely. Its recommended controls (per-tool least-privilege profiles, action-level approval with a dry-run diff, budgets, and immutable logs of every invocation) map directly onto the contract above.

When the same tools need to serve several AI clients, publish them through an MCP server for your business systems with the same contracts. For non-technical staff, our lesson on understanding AI tool permissions explains the side-effect classes in plain terms. This is the core of how we approach AI agent development and AI integration work.

• Every tool has a narrow verb name, a purpose line and a “do not use for” line
• Arguments the code already knows are removed from the schema
• Strict schemas are on, and tests assert the API did not fall back to non-strict
• Each tool is classified as read, reversible write or external effect
• Writes carry an idempotency key derived from run, tool and canonical arguments
• Timeouts on writes trigger a status check, never a blind retry
• Parallel calls are allowed for reads only
• Results are capped in size and return IDs plus a cursor
• Errors return a code, message, hint and retryable flag
• Record IDs are accepted only if an earlier result in the run returned them
• Runs have step, time and spend budgets
• Traces record tool, arguments, key, class, outcome and latency for every call

Written by the Levy Street team in Wellington. Last updated 19 October 2026.