What function calling actually is
Function calling is the protocol that lets a large language model ask your application to run code on its behalf. Instead of returning text directly, the model emits a structured JSON object that names a tool and supplies the arguments to call it with. Your application executes the tool, returns the result, and the model continues the conversation with that information in hand.
The mechanism turns a chatbot from a thing that answers into a thing that does. Without function calling, a support bot can describe how to check an order status. With function calling, the bot calls getOrderStatus({ orderId: "A-4471" }), receives "Shipped, arriving Friday", and tells the customer the actual answer.
OpenAI shipped the first version of this pattern in June 2023 under the name "functions". By late 2023 the API was renamed to "tools" to reflect that the same channel could host built-in capabilities (web search, code execution) alongside developer-defined ones. Anthropic launched tool use for Claude in 2024, and Google added it to Gemini around the same window. The three providers use slightly different request shapes, but the loop is identical.
The function-calling loop (declare, invoke, respond)
Every function-calling exchange follows four steps:
1. Declare the tools. The application sends the user message together with a list of tool schemas. Each schema includes a name, a natural-language description, and a JSON Schema definition of the arguments the tool accepts. The descriptions matter as much as the names because the model uses them to decide when each tool applies.
2. The model picks one or more. Instead of normal text output, the model emits a tool-call block containing the chosen tool name and a JSON object of arguments. The model is allowed to emit several tool calls in the same turn (covered below) or to skip tool calls entirely and answer in plain text.
3. The application executes. Your code parses the JSON, calls the matching function, and captures the result. This step is entirely outside the model. The function can hit an internal database, a third-party API, a local cache, or anything else you control.
4. The result goes back. You append the tool result to the conversation as a tool-result message and send the whole history back to the model. The model then continues, either replying in natural language or requesting another tool call. The loop can run several rounds before the assistant produces its final answer.
Parallel tool calling, where the model emits multiple tool-call blocks in a single assistant turn, became the default in 2024. OpenAI exposes a parallel_tool_calls flag and Anthropic exposes disable_parallel_tool_use; both let operators opt out when they want strict one-at-a-time behavior. Claude 4 models in particular are tuned to fan out tool calls aggressively when the user request implies independent sub-tasks.
Why function calling matters for AI chatbots
For a website chatbot, function calling is the bridge between conversation and the business systems behind it. The most common chatbot tools fall into a short list:
- Order lookups for ecommerce: status, tracking number, return window.
- Calendar booking for service businesses: check availability, hold a slot, confirm.
- Ticket creation for support: open a Zendesk or Linear issue with the user's description.
- Inventory queries for retail: is the product in stock, in which size, at which location.
- CRM updates for sales bots: log the lead, attach a transcript, set the stage.
Without function calling, every one of these requires a human handoff. With it, the same chat widget that answered "what is your return policy" can also answer "where is my order" because it actually checks. ChatRaj exposes function calling as an Actions feature, letting operators connect their CRM, calendar, and inventory APIs to the bot from the dashboard without writing glue code. The model still has to be told, through the system prompt and tool descriptions, when each action is appropriate.
Good prompt engineering around tools is less about clever wording and more about clear descriptions. A tool called refundOrder with the description "Issue a refund for an order. Only call after the customer explicitly confirms they want a refund" is far more reliable than the same tool described as "Refunds an order".
Function calling vs structured output vs agentic AI
Three concepts often get tangled. They are related but not the same.
Structured output forces the model to produce text matching a JSON schema. Modern APIs implement it with constrained decoding, a token-level sampling restriction that prevents the model from generating any token that would break the schema. The output is shape-guaranteed JSON. There is no execution, no callback, no second turn. Use it when you want clean data, not when you want an action.
Function calling negotiates an action. The model picks a tool, the application runs it, the result feeds back. Constrained decoding is often used under the hood to guarantee the arguments JSON is valid, which is why OpenAI's "strict mode" for tools is recommended. The protocol is conversational by design.
Agentic AI is the loop running many rounds, with planning, retries, and self-correction. An agent uses function calling as its action layer. Every agent framework, from LangGraph to Anthropic's tool runner to OpenAI's Responses API, is built on the same four-step loop above, repeated until a goal is met. Function calling is the protocol; agency is what you do with it.
The distinction matters when picking tools. If you need a chatbot to fill a form, structured output is enough. If it needs to look something up and respond, plain function calling works. If it needs to plan, act, observe, and revise across many steps, you are in agent territory and will need state management, guardrails, and a budget on tool calls.