本地运行大语言模型：实现函数调用与工具自主使用

定义 Prompts

为了让模型知道这些函数的存在及其参数结构，我们需要构建一个包含函数元数据的 Prompt。

functions_prompt = f"""
You have access to the following tools:
{function_to_json(get_weather)}
{function_to_json(get_gas_prices)}
{function_to_json(get_directions)}
{function_to_json(github)}

You must follow these instructions:
Always select one or more of the above tools based on the user query
If a tool is found, you must respond in the JSON format matching the following schema:
{{
   "tools": {{
        "tool": "<name of the selected tool>",
        "tool_input": <parameters for the selected tool, matching the tool's JSON schema
   }}
}}
If there are multiple tools required, make sure a list of tools are returned in a JSON array.
If there is no tool that match the user request, you must respond empty JSON {{}}.

User Query:
    """

这是一个复杂的提示，让我们一步一步来看：

1. 告知工具列表：告诉模型我提供了 4 个工具，让它自己去查询这 4 个工具的元数据。function_to_json 函数返回的内容如下：

{
  "name": "get_weather",
  "description": "Get the current weather given a city.",
  "parameters": {
    "type": "object",
    "properties": {
      "city": {
        "type": "str"
      }
    }
  },
  "returns": "str"
}

这一步就是让模型根据这几个函数的元数据来理解函数，尤其是 description 写的应该尽量详细，以便模型准确匹配意图。

2. 指定行为规则：提示模型作出自己的判断，根据上面提供的工具选择一个或多个进行调用，并指定了模型返回数据格式——JSON 以及这个 JSON 的 Schema。

3. 等待用户问题：最后留出位置等待用户的实际 Query。

完整代码实现

整段代码展示了如何结合 Python 脚本、Ollama API 以及自定义函数来实现这一流程。

import inspect
import json
import requests
from typing import get_type_hints

def generate_full_completion(model: str, prompt: str) -> dict[str, str]:
    # 调用 Ollama 本地 API
    params = {"model": model, "prompt": prompt, "stream": False}
    response = requests.post(
        "http://localhost:11434/api/generate",
        headers={"Content-Type": "application/json"},
        data=json.dumps(params),
        timeout=60,
    )
    return json.loads(response.text)

def get_gas_prices(city: str) -> float:
    """Get gas prices for specified cities."""
    print(f'Get gas prices for {city}.')

def github(project: str) -> str:
    '''Get the information such as author from github with the project name.'''
    print(f'Access the github for {project}.')

def get_weather(city: str) -> str:
    """Get the current weather given a city."""
    print(f'Getting weather for {city}.')

def get_directions(start: str, destination: str) -> float:
    """Get directions from Google Directions API.
    start: start address as a string including zipcode (if any)
    destination: end address as a string including zipcode (if any)"""
    print(f'Get directions for {start} {destination}.')

def get_type_name(t):
    name = str(t)
    if "list" in name or "dict" in name:
        return name
    else:
        return t.__name__

def function_to_json(func):
    # 动态获取函数签名和类型注解，生成 JSON 描述
    signature = inspect.signature(func)
    type_hints = get_type_hints(func)

    function_info = {
        "name": func.__name__,
        "description": func.__doc__,
        "parameters": {"type": "object", "properties": {}},
        "returns": type_hints.get("return", "void").__name__,
    }

    for name, _ in signature.parameters.items():
        param_type = get_type_name(type_hints.get(name, type(None)))
        function_info["parameters"]["properties"][name] = {"type": param_type}

    return json.dumps(function_info, indent=2)

def execute_fuc(tool_data):
    # 根据模型返回的工具名称，动态查找并执行对应的 Python 函数
    func_name = tool_data["tool"]
    func_input = tool_data["tool_input"]

    # 获取全局命名空间中的函数对象
    func = globals().get(func_name)

    if func is not None and callable(func):
        # 如果找到了函数并且是可调用的，调用它
        func(**func_input)
    else:
        print(f"Unknown function: {func_name}")

def main():
    functions_prompt = f"""
You have access to the following tools:
{function_to_json(get_weather)}
{function_to_json(get_gas_prices)}
{function_to_json(get_directions)}
{function_to_json(github)}

You must follow these instructions:
Always select one or more of the above tools based on the user query
If a tool is found, you must respond in the JSON format matching the following schema:
{{
   "tools": {{
        "tool": "<name of the selected tool>",
        "tool_input": <parameters for the selected tool, matching the tool's JSON schema
   }}
}}
If there are multiple tools required, make sure a list of tools are returned in a JSON array.
If there is no tool that match the user request, you must respond empty JSON {{}}.

User Query:
    """

    GPT_MODEL = "mistral:7b-instruct-v0.2-q8_0"

    prompts = [
        "What's the weather like in Beijing?",
        "What is the distance from Shanghai to Hangzhou and how much do I need to fill up the gas in advance to drive from Shanghai to Hangzhou?",
        "Who's the author of the 'snake-game' on github?",
        "What is the exchange rate between US dollar and Japanese yen?",
    ]

    for prompt in prompts:
        print(f"❓{prompt}")
        question = functions_prompt + prompt
        response = generate_full_completion(GPT_MODEL, question)
        try:
            data = json.loads(response.get("response", response))
            # print(data)
            for tool_data in data["tools"]:
                execute_fuc(tool_data)
        except Exception:
            print('No tools found.')
        print(f"Total duration: {int(response.get('total_duration')) / 1e9} seconds")

if __name__ == "__main__":
    main()

测试与分析

整段代码就是用来测试模型对问题的理解能力以及是否能正确判断调用哪个工具的。

我们来看下设定的 4 个问题：

1. What's the weather like in Beijing?
2. What is the distance from Shanghai to Hangzhou and how much do I need to fill up the gas in advance to drive from Shanghai to Hangzhou?
3. Who's the author of the 'snake-game' on github?
4. What is the exchange rate between US dollar and Japanese yen?

按照我们的设想，如果模型理解了我提供的工具的功能以及读懂了用户的问题，应该按照以下规则选择工具：

• 问题 1 应该对应的是 get_weather('Beijing')
• 问题 2 应该对应的是 get_directions('Shanghai', 'Hangzhou') 和 get_gas_prices('Shanghai')
• 问题 3 应该对应的是 github('snake-game')
• 问题 4 应该没有对应的函数，模型不选择任何工具

下一步，我们也写好了代码进行测试，我们在工具函数里进行参数打印，以查看是否达到我们的预期。

测试结果

❓What's the weather like in Beijing?
Getting weather for Beijing.
Total duration: 3.481918084 seconds

❓What is the distance from Shanghai to Hangzhou and how much do I need to fill up the gas in advance to drive from Shanghai to Hangzhou?
Get directions for Shanghai, China Hangzhou, China.
Get gas prices for Shanghai.
Total duration: 5.467253959 seconds

❓Who's the author of the 'snake-game' on github?
Access the github for snake-game.
Total duration: 2.510993791 seconds

❓What is the exchange rate between US dollar and Japanese yen?
{}
No tools found.
Total duration: 0.20056292 seconds

深入理解与注意事项

1. 函数元数据的重要性

在 function_to_json 中，我们使用了 inspect 模块来获取函数的签名和文档字符串。这对于本地模型尤为重要，因为本地模型通常不如云端大模型那样经过广泛的指令微调。详细的 description 能显著提高模型识别意图的准确率。建议为每个工具编写清晰、具体的中文或英文描述，明确输入参数的含义。

2. JSON Schema 约束

Prompt 中定义的 JSON Schema 必须严格遵循模型期望的格式。在上述示例中，我们要求模型返回嵌套的 JSON 对象。如果模型返回的格式不符合预期（例如缺少引号或括号），解析时会抛出异常。在实际生产中，建议增加健壮的错误处理机制，例如尝试多次重试或使用正则表达式提取 JSON 块。

3. 本地模型的局限性

虽然 Mistral 等模型表现良好，但不同量化版本（如 q8_0, q4_k_m）在函数调用能力上可能存在差异。较小的模型可能无法严格遵守复杂的 JSON 结构。如果遇到模型频繁返回错误格式的情况，可以尝试简化 Prompt 中的 Schema 描述，或者更换参数量更大的模型。

4. 扩展性设计

目前的 execute_fuc 函数通过 globals() 动态查找函数。在生产环境中，建议使用注册表模式（Registry Pattern）来管理工具，避免全局变量污染，同时便于权限控制和日志记录。此外，可以将 print 语句替换为真实的 HTTP 请求，例如调用真实的天气 API 或 GitHub API，从而实现完整的自动化工作流。

总结

本篇文章详细介绍了使用 Function Calling 使得 Ollama 模型具备调用外部工具的能力。通过对定义函数、构建 Prompt、编写执行逻辑以及测试结果的完整分析，展示了本地大模型实现自主工具调用的可行路径。

希望本系列文章能使大家对本地运行大语言模型有一些深入的了解。后续我们将继续探讨更多本地 LLM 的实用场景，包括知识库检索增强生成（RAG）以及多模态交互等内容。