Function Calling

The LLM is prompted to output a strictly formatted request (typically JSON) instead of natural language. This request specifies the function name and its required arguments. The format is predefined in the system prompt, often using a schema like JSON Schema or OpenAPI.

• Example: Instead of saying "Get the weather for San Francisco," the model outputs {"function": "get_weather", "location": "San Francisco", "unit": "celsius"}.
• This structured output is machine-parsable, enabling deterministic execution by a downstream orchestrator.

Function calling requires a predefined inventory of tools (functions/APIs) that the model is allowed to use. Each tool is described with:

• Name: A unique identifier.
• Description: A natural language explanation of the tool's purpose, which the model uses for planning.
• Parameter Schema: The exact data structure of required and optional arguments, including their types and descriptions.

This schema is injected into the model's context, acting as a constrained action space. The model must select from this list and populate the arguments correctly.

Function calling is often deployed within an agentic loop where the model alternates between reasoning and acting. A single interaction follows this pattern:

1. User Query: The user presents a goal (e.g., "Book a 7 pm dinner for two in Manhattan").
2. Model Reasoning: The LLM analyzes the query against available tools (find_restaurants, book_reservation, get_user_profile).
3. Structured Call: The model outputs the function call for the logical first step (e.g., {"function": "find_restaurants", "criteria": {...}}).
4. Execution & Observation: The backend executes the call, returning results (e.g., a list of restaurants).
5. Next Step: These results are fed back to the model as observations, prompting the next reasoning step and function call, until the task is complete.

This characteristic addresses the hallucination problem by tethering model outputs to verifiable external data. The model itself does not generate the final answer (e.g., the weather temperature); it generates a call to a deterministic function that retrieves or calculates the answer.

• The final response to the user is assembled from the results of executed functions, not from the model's parametric knowledge alone.
• This provides citable data provenance and significantly improves factual accuracy for dynamic or proprietary information.

The LLM is only one component in a function calling system. A separate orchestrator (or agent runtime) is required to:

• Receive and parse the model's structured output.
• Safely execute the corresponding function with the provided arguments.
• Handle errors, timeouts, and invalid requests.
• Return the function's result (or error) back to the LLM as context for the next step.
• Manage conversation state and the sequence of calls.

This decouples the model's planning capability from the secure execution of code.

ReAct (Reasoning + Acting) is a specific prompting framework that formalizes the function calling loop. It explicitly instructs the model to interleave Thought, Action, and Observation steps in its output.

• Thought: The model's internal reasoning about what to do next.
• Action: The structured function call (identical to standard function calling output).
• Observation: The result from the executed function.

This verbose output format makes the model's plan traceable and often improves reliability on complex tasks by forcing explicit step-by-step reasoning before each action.

Tool calling is the broader paradigm of enabling a language model to interact with external software, APIs, or data sources. It encompasses the entire lifecycle: defining tools, describing them to the model, parsing the model's request, executing the tool, and returning the result. Function calling is a specific, structured implementation of tool calling, often using JSON schema.

• Key Distinction: All function calling is tool calling, but not all tool calling uses formal function schemas (e.g., simpler text-based instructions).
• Implementation: Modern LLM APIs (OpenAI, Anthropic, Google) have native support for structured tool/function calling.

ReAct (Reasoning + Acting) is a prompting paradigm that synergistically combines chain-of-thought reasoning with actionable tool calls. The model is prompted to interleave verbal reasoning traces (Thought:) with specific actions (Action:), which are then executed to gather information (Observation:).

• Structure: Thought > Action > Observation loops.
• Purpose: Improves task performance by allowing the model to plan, use tools for grounding, and adapt based on real-world feedback.
• Relation to Function Calling: ReAct is a high-level prompting strategy that often uses function/tool calling as its execution mechanism for the Action steps.

An agentic cognitive architecture is a system design that gives an LLM (or other model) the capacity for autonomous, multi-step goal completion. It integrates core components like planning, memory, tool use, and reflection. Function calling is the primary mechanism by which such an agent acts upon its environment.

• Core Loop: Perceive > Plan > Act (via tools) > Reflect.
• Components: Includes planning modules (e.g., Tree of Thoughts), tool-calling interfaces, memory stores, and evaluators.
• Enterprise Relevance: Demonstrates a system's ability to decompose complex business goals into executable sequences of API calls.

Structured output prompting is the technique of instructing an LLM to generate responses in a precise, machine-parsable format like JSON, XML, or YAML. This is the foundational skill required for reliable function calling, as the model must output a strict schema matching the expected function arguments.

• Techniques: Using delimiters (```json), providing explicit schemas, or using few-shot examples of the desired format.
• Direct Application: A function call request (e.g., {"name": "get_weather", "arguments": {"location": "Boston"}}) is a specific instance of structured output.
• Importance: Ensures deterministic parsing by downstream systems, enabling automation.

API execution refers to the downstream system action that occurs after a function call is parsed: making the actual HTTP request to an external service, database query, or internal procedure. It is the concrete implementation step following the LLM's structured request.

• Workflow: LLM generates call > Orchestrator validates & parses it > Executor invokes the API with parameters > Results are formatted and returned to the LLM.
• Security Critical: This layer must enforce authentication, rate limiting, input sanitization, and error handling.
• Observability: Latency, success rates, and costs of API execution are key metrics for agentic systems.