Function Calling

The core mechanism of function calling is the model's ability to generate a valid JSON object conforming to a predefined schema, rather than natural language. This eliminates the fragility of parsing arguments from free-text responses.

• Schema Enforcement: The model adheres strictly to the provided JSON Schema definition, ensuring parameter names, types, and required fields are correct.
• Deterministic Integration: Downstream code can directly unmarshal the output into native data structures without regex or heuristic parsing.
• Example: A model asked to query a legal database outputs {"query": "res ipsa loquitur", "jurisdiction": "CA", "date_range": {"start": "2020-01-01"}} instead of a conversational suggestion.

Developers provide the model with a tool definition—a JSON Schema document describing the function's purpose, parameters, and their types. This acts as the model's API contract.

• Semantic Naming: The function name and description are critical prompts; they guide the model on when to select this tool over others.
• Parameter Typing: Explicit types like string, number, boolean, and enum constrain the model's generation, preventing type errors.
• Legal Context: A tool defined as get_case_law(query: string, court: enum) teaches the model the valid court abbreviations it can use.

Modern function calling supports dispatching multiple independent function calls simultaneously in a single response, dramatically reducing latency for complex multi-step queries.

• Non-Blocking Execution: The model identifies calls that do not depend on each other and bundles them.
• Result Aggregation: The calling application executes all functions concurrently and returns the combined results to the model for a final synthesis.
• Example: For a due diligence request, the model simultaneously calls search_litigation(company="Acme") and check_regulatory_filings(company="Acme") rather than sequencing them.

In production systems, you can force the model to always invoke a specific function, bypassing its discretion. This is essential for building deterministic agentic loops.

• Tool Use Mode: Setting the tool_choice parameter to a specific function name guarantees the model will output a call to that tool.
• Routing Guarantee: This ensures a user query like 'What's the deadline?' is always routed to a calculate_deadline function, never answered from the model's parametric memory.
• Use Case: Enforcing that every legal query first passes through a check_privilege_status function before any other processing.

Function calling enables a ReAct (Reasoning + Acting) loop where the model alternates between calling a tool and reasoning about the result until it can formulate a final answer.

• Observation Integration: The model receives the API's return value as a new message in the context window and uses it to decide the next step.
• Error Handling: If a tool returns an error, the model can reason about the failure and retry with corrected parameters.
• Legal Application: The model calls search_precedents, receives an empty result, and autonomously decides to broaden the search by calling the function again with a more general query.

Function calling is fully compatible with server-sent event (SSE) streaming, allowing the structured JSON to be transmitted incrementally as it is generated.

• Low Time-to-First-Byte: The client can begin parsing partial JSON fragments before the entire function call is complete.
• Progress Indication: The UI can show an intermediate state like 'Searching database...' as soon as the function name is streamed.
• Technical Note: The streaming protocol typically sends the function name first, followed by chunks of the arguments JSON, requiring a client-side accumulator to reconstruct the full call.