Function calling

Connect the xAI models to external tools and systems to build AI assistants and various integrations.

Introduction

Function calling enables language models to use external tools, which can intimately connect models to digital and physical worlds.

This is a powerful capability that can be used to enable a wide range of use cases.

Calling public APIs for actions ranging from looking up football game results to getting real-time satellite positioning data
Analyzing internal databases
Browsing web pages
Executing code
Interacting with the physical world (e.g. booking a flight ticket, opening your tesla car door, controlling robot arms)

Walkthrough

The request/response flow for function calling can be demonstrated in the following illustration.

Function call request/response flow example

You can think of it as the LLM initiating RPCs (Remote Procedure Calls) to user system. From the LLM's perspective, the "2. Response" is an RPC request from LLM to user system, and the "3. Request" is an RPC response with information that LLM needs.

The whole process looks like this in pseudocode:

We will demonstrate the function calling in the following Python script. First, let's create an API client:

Preparation - Define tool functions and function mapping

Define tool functions as callback functions to be called when model requests them in response.

Normally, these functions would either retrieve data from a database, or call another API endpoint, or perform some actions. For demonstration purposes, we hardcode to return 59° Fahrenheit/15° Celsius as the temperature, and 15,000 feet as the cloud ceiling.

The parameters definition will be sent in the initial request to Grok, so Grok knows what tools and parameters are available to be called.

To reduce human error, you can define the tools partially using Pydantic.

Function definition using Pydantic:

Function definition using raw dictionary:

Create a string -> function mapping, so we can call the function when model sends it's name. e.g.

1. Send initial message

With all the functions defined, it's time to send our API request to Grok!

Now before we send it over, let's look at how the generic request body for a new task looks like.

Note how the tool call is referenced three times:

By id and name in "previous assistant response to get tool call"
By tool_call_id in "tool call return"
In the tools field of the request body

Now we compose the request messages in the request body and send it over to Grok. Grok should return a response that asks us for a tool call.

2. Run tool functions if Grok askes tool call and append function returns to message

We retrieve the tool function names and arguments that Grok wants to call, run the functions, and add the result to messages.

At this point, you can choose to only respond to tool call with results or add a new user message request.

The tool message would contain the following: { "role": "tool", "content": <json string of tool function's returned object>, "tool_call_id": <tool_call.id included in the tool call response by Grok>}

The request body that we try to assemble and send back to Grok. Note it looks slightly different from the new task request body:

The corresponding code to append messages:

3. Send the tool function returns back to the model to get the response

4. (Optional) Continue the conversation

You can continue the conversation following Step 2. Otherwise you can terminate.