Tool support · Ollama Blog

July 25, 2024

Ollama now supports tool calling with popular models such as Llama 3.1. This enables a model to answer a given prompt using tool(s) it knows about, making it possible for models to perform more complex tasks or interact with the outside world.

Example tools include:

• Functions and APIs
• Web browsing
• Code interpreter
• much more!

Tool calling

To enable tool calling, provide a list of available tools via the tools field in Ollama’s API.

import ollama

response = ollama.chat(
    model='llama3.1',
    messages=[{'role': 'user', 'content':
        'What is the weather in Toronto?'}],

		# provide a weather checking tool to the model
    tools=[{
      'type': 'function',
      'function': {
        'name': 'get_current_weather',
        'description': 'Get the current weather for a city',
        'parameters': {
          'type': 'object',
          'properties': {
            'city': {
              'type': 'string',
              'description': 'The name of the city',
            },
          },
          'required': ['city'],
        },
      },
    },
  ],
)

print(response['message']['tool_calls'])

Supported models will now answer with a tool_calls response. Tool responses can be provided via messages with the tool role. See API documentation for more information.

Supported models

A list of supported models can be found under the Tools category on the models page:

• Llama 3.1
• Mistral Nemo
• Firefunction v2
• Command-R +

Note: please check if you have the latest model by running ollama pull <model>

OpenAI compatibility

Ollama’s OpenAI compatible endpoint also now supports tools, making it possible to switch to using Llama 3.1 and other models.

import openai

openai.base_url = "http://localhost:11434/v1"
openai.api_key = 'ollama'

response = openai.chat.completions.create(
	model="llama3.1",
	messages=messages,
	tools=tools,
)

Examples

• Python
• JavaScript

Future improvements

• Streaming tool calls: stream tool calls back to begin taking action faster when multiple tools are returned
• Tool choice: force a model to use a tool

Let’s build together

We are so excited to bring you tool support, and see what you build with it!

If you have any feedback, please do not hesitate to tell us either in our Discord or via hello@ollama.com.