Large Language Models are impressive—but they’re not all-knowing. What makes them truly powerful is their ability to use tools, access real-time data, and generate actionable responses.
Tool Calling: Empowering LLMs to Interact with the World
Large Language Models (LLMs) are powerful in generating human-like text based on the data they were trained on. However, their knowledge is limited by this training data, and they lack the ability to perform actions in the real world or access up-to-the-minute information. This is where tool calling, also known as function calling, comes in. It’s a mechanism that allows LLMs to connect with external tools, APIs, or systems, significantly expanding their capabilities and enabling them to become more dynamic and useful.
Why LLMs Need Tool Calling :-
LLMs need tool calling to overcome their limitations:
- Access real-time or dynamic information beyond their training data (like weather information, stock prices, search on web).
- Perform actions in the real world (like sending emails or booking appointments).
- Achieve deterministic and accurate results for tasks like calculations or data lookups.
Enable complex problem-solving by using specialized tools.
Tool calling help to create specific Agent in AI:-
By using appropriate way of using tool calling we can also create an agentic ai that based on LLMs.
Agent creation involves defining external tools (functions) with descriptions and parameters. You then provide these tools to the LLM. The LLM decides when a tool is needed for a user request and outputs a structured call for that tool, including arguments. Your application executes the actual function using the provided arguments. Finally, the result from the tool execution is given back to the LLM so it can formulate the final, informed response to the user.
How Tool Calling Works Under the Hood: The Technical Mechanism
Once the LLM determines that a user’s query requires external interaction, the tool calling mechanism kicks in. Here’s a breakdown of the technical steps:
- Function Definitions (Schemas) Provided to the Model: Before any tool calling can happen, the application or developer needs to define the available tools and provide these definitions to the LLM. These definitions aren’t the actual code of the tool, but rather a structured description that the LLM can understand. This typically includes:
- The name of the function (e.g., getCurrentWeather, getStockPrice).
- A clear description of what the function does. This text is crucial for the LLM to decide when to use the tool.
- A specification of the parameters the function accepts, often in a structured format like a JSON schema. This schema defines the expected input fields, their data types (e.g., string, integer), and whether they are required (e.g., a location parameter for a weather tool).
- When you make a request to the LLM, you include these tool definitions as part of the input. The LLM is specifically trained to understand these definitions and recognize when an incoming query aligns with the purpose of a defined tool.
- Model Generating Structured Function Calls: When the LLM processes a query and identifies that a defined tool is relevant, it doesn’t execute the tool directly. Instead, it generates a structured output that represents the call to that tool. This output is typically in a machine-readable format, most commonly JSON. This structured call will contain:
- The name of the tool the LLM has decided to use.
- The arguments (parameter values) that the LLM has extracted from the user’s natural language query, formatted according to the tool’s defined schema. For example, if the user asked “What is the weather in London?”, the LLM would identify the getCurrentWeather tool and output a JSON object like {“tool_name”: “getCurrentWeather”, “parameters”: {“location”: “London”}}.
- This is the core of the tool calling concept: the LLM translates the user’s natural language intent into a structured, executable instruction for an external system.
- Routing Those Calls to the Right Tools: The application or framework interacting with the LLM receives this structured tool call output. It then acts as an intermediary. It parses the structured output, identifies the requested tool name and parameters, and then routes this information to the actual code or service that implements the defined tool. This routing layer maps the tool name received from the LLM to the corresponding internal function or external API call.
- Mentioning Frameworks: Implementing this routing and execution logic from scratch can be complex. Several frameworks and platforms provide built-in support for tool calling, simplifying the development process. Prominent examples include:
- OpenAI Function Calling API: OpenAI’s models are specifically fine-tuned for function calling, making it straightforward to define functions and receive structured outputs from the model.
- LangChain: This popular framework provides abstractions and tools for building agentic applications, including robust support for defining tools, connecting them to various LLMs, and managing the tool execution flow.
- Other Libraries and Custom Integrations: Many other libraries and platforms for working with LLMs offer similar tool calling capabilities. Developers can also build custom integration layers to connect LLMs with their existing internal systems or third-party APIs based on the structured output provided by the model.
Here is example for tool calling:-
Here first we need to define tool definition like this
get_stock_price_tool = {
“type”: “function”,
“function”: {
“name”: “get_stock_price”, # Name of the function/tool
“description”: “Get the current stock price for a given ticker symbol.”, # Description for the LLM
“parameters”: {
“type”: “object”,
“properties”: {
“ticker_symbol”: {
“type”: “string”,
“description”: “The stock ticker symbol (e.g., AAPL for Apple, GOOGL for Google). Required.”
},
“currency”: {
“type”: “string”,
“description”: “The desired currency for the stock price (e.g., USD, EUR). Optional.”
}
},
“required”: [“ticker_symbol”] # ‘ticker_symbol’ is a mandatory parameter
}
}
This is how Ollama is used for tool calling:-
response = ollama.chat(
model=”llama3.2:latest”,
# model=”mistral:latest”,
messages=[
system_prompt,
{“role”: “user”, “content”: prompt}
],
tools=[
get_stock_price_tool,
],
options={“temperature”: 0.1}
)
tool_calls = response.get(‘message’, {}).get(‘tool_calls’)
Here get the details of the function called by LLM’’s.
Supported Large language Models for tool calling
Not all LLM models directly supports tool calling like deepseek.
Here are the list that supports directly tool calling
- OpenAI (GPT-4o, GPT-4, GPT-3.5 Turbo)
- Google (Gemini 1.5 models)
- Anthropic (Claude 3 series, Claude 3.5 Sonnet)
- Mistral AI (Large, Medium, Small, Instruct models)
- Cohere (Command R+)
- Meta LLaMA (Instruct)
- Some other open source model also supports.
LLM Generates Final Response Using Tool Output
After the function call LLM gives structured output as per our function definition.
Now parse this function as needed.
And then give data to the LLM model again with a user query. It will give you proper output.
Like here as per example get stock details like
{
“ticker_symbol” : “AAPL”,
“Currency”: “USD”}
Now use this and find the current stock price using get stock price api.
And pass that output with a user query to return to the LLM model to generate humans based text output.
This way it helps to create custom chatbot for performing specific action to reduce human work,
By just entering a prompt, the user can perform an action.
This is widely used In Building Agentic Ai where one agent is specifically built for doing a specific task.
Limitation of tool calling In LLM’s
Tool calling’s effectiveness relies heavily on the LLM accurately interpreting user intent and tool definitions, which can sometimes lead to incorrect tool selections or arguments. Implementing it adds system complexity, introduces latency due to external calls, and requires robust error handling. Managing a large number of tools can be challenging for the LLM, and there are security considerations when granting LLMs access to external systems via tools.
