眾所周知，LLM的函式function-calling能力很強悍，解決了大模型與實際業務系統的互動問題。其本質就是函式呼叫。
從openai官網摘圖：
簡而言之：
-
LLM起到決策的作用，告知業務系統應該呼叫什麼函式，以及入參是什麼。
-
業務系統負責實現對應的函式（比如本地實現，或者呼叫其他系統提供的服務），並且將函式的響應結果再次拋給LLM。
- LLM根據響應結果，組織自然語言，繼續與業務系統進行互動。
在這裡，有很多小夥伴會有一個誤區：誤以為函式呼叫是有LLM本身執行的。其實，LLM僅僅做決策，而實際的呼叫是由業務系統完成的。
現階段，function-calling能力的實現有兩種主流方式：-
LLM本身支援。
-
利用Prompt模板實現，典型如ReAct模板。
在實際的應用過程中，我們還要解決另一個重要問題：
function-calling觸發機制是怎樣的？也即：何時要使用function-calling能力，何時不應該使用？
這個問題的處理方式，對於整體流程的執行至關重要。
此時，我們可以使用特定Prompt來解決該問題：
You have access to the following tools:
{json.dumps(tools)}
You can select one of the above tools or just response user's content and respond with only a JSON object matching the following schema:
{{
"tool": <name of the selected tool>,
"tool_input": <parameters for the selected tool, matching the tool'
s JSON schema>,
"message": <direct response users content>}該Prompt告知了LLM：如果需要使用function-calling能力，那麼就從tools（tools是預定義的functions）中選取一個最匹配的函式；如果不需要，就用自然語言與使用者互動，此時與正常的對話流程無異。輸出的格式固定為json，方便解析。
由此，我們受到啟發：只要LLM基座夠強（能夠嚴格遵循Prompt響應訴求），即使LLM本身不支援function-calling，我們也可以自己實現function-calling，脫離對特定LLM的依賴！
拿到function-calling的結果後，若要用自然語言的形式輸出結果，還要再呼叫一次LLM，對結果進行整合。此時可以使用另一個Prompt：
Please generate a natural language description based on the following question and answer.
Question: [Content of the question]
Answer: [Content of the answer]
Generated Description: The result of [key phrase from the question] is [answer].
If necessary, you can polish the description.Only output the Description, with Chinese language.該Prompt的作用就是告訴LLM，你要根據我的問題和答案，用自然語言重新描述一遍。這裡指定了中文輸出，可根據實際需要進行調整。
以下是一個可執行的完整Python指令碼：
import requests
import json
import random
# 預置函式定義
tools = [
{
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city e.g. Beijing"
},
"unit": {
"type": "string",
"enum": [
"celsius"
]
}
},
"required": [
"location"
]
}
},
{
"name": "calculator",
"description": "計算器",
"parameters": {
"type": "int",
"properties": {
"a": {
"type": "int",
"description": "the first number"
},
"b": {
"type": "int",
"description": "the second number"
}
},
"required": [
"a",
"b"
]
}
}
]
# 獲取天氣（隨機返回，實際使用可以替換為api呼叫）
def get_current_weather(*args):
# 定義可能的天氣狀態
weather_conditions = ["sunny", "cloudy", "rainy", "snowy"]
# 定義可能的溫度範圍
temperature_min = -10 # 最低溫度，攝氏度
temperature_max = 35 # 最高溫度，攝氏度
# 隨機選擇一個天氣狀態
condition = random.choice(weather_conditions)
# 隨機生成一個溫度
temperature = random.randint(temperature_min, temperature_max)
# 返回一個描述當前天氣的字串
return f"The weather of {args[0].get('location')} is {condition}, and the temperature is {temperature}°C."
def calculator(args):
return sum(value for value in args.values() if isinstance(value, int))
# 函式對映集合
functions = {
"get_current_weather": get_current_weather,
"calculator": calculator,
}
# 驅動整體流程的入口prompt
entrance_prompt = f"""You have access to the following tools:
{json.dumps(tools)}
You can select one of the above tools or just response user's content and respond with only a JSON object matching the following schema:
{{
"tool": <name of the selected tool>,
"tool_input": <parameters for the selected tool, matching the tool's JSON schema>,
"message": <direct response users content>
}}"""
# 請以自然語言的形式對結果進行描述
conformity_prompt = f"""
Please generate a natural language description based on the following question and answer.
Question: [Content of the question]
Answer: [Content of the answer]
Generated Description: The result of [key phrase from the question] is [answer].
If necessary, you can polish the description.
Only output the Description, with Chinese language.
"""
def extract_json(s):
stack = 0
start = s.find('{')
if start == -1:
return None
for i in range(start, len(s)):
if s[i] == '{':
stack += 1
elif s[i] == '}':
stack -= 1
if stack == 0:
return s[start:i + 1]
return None
# 結果包裝器，type為func表示是函式呼叫返回的結果，default表示是自然語言結果。對於func返回的結果，會用LLM再次總結
class ResultWrapper:
def __init__(self, type, result):
self.type = type
self.result = result
# 解析LLM返回的結果，如果有json則去解析json
def parse_result(res):
json_str = extract_json(res["message"]["content"])
if json_str is not None:
obj = json.loads(json_str)
if "tool" in obj:
if obj["tool"] in functions:
fun = functions[obj["tool"]]
return ResultWrapper("func", fun(obj["tool_input"]))
else:
return ResultWrapper("default", obj["message"])
else:
return ResultWrapper("default", res["message"]["content"])
else:
return ResultWrapper("default", res["message"]["content"])
def invokeLLM(messages):
url = "${domain}/v1/chat/completions" #需替換域名
model = ""
payload = {
"model": model,
"messages": messages,
}
payload = json.dumps(payload)
headers = {
'Content-Type': 'application/json'
}
print("PAYLOAD: ", payload)
response = requests.request("POST", url, headers=headers, data=payload)
print("RESPONSE: ", response.text)
print("=======================================================================")
resp = json.loads(response.text)
return resp["choices"][0]
if __name__ == '__main__':
while True:
messages = [
{
"role": "system",
"content": entrance_prompt
}
]
user_input = input('Enter a string: ')
messages.append({
"role": "user",
"content": user_input
})
result_wrapper = parse_result(invokeLLM(messages))
if result_wrapper.type == "func":
messages = [
{
"role": "user",
"content": f"{conformity_prompt}\n\nThe question:{user_input}\nThe answer:{result_wrapper.result}"
}
]
print("FINAL RESULT WITH FUNCTION CALL: ", parse_result(invokeLLM(messages)).result)
else: print("FINAL RESULT: ", result_wrapper.result實驗效果：
Enter a string: 你好
PAYLOAD: {"model": "", "messages": [{"role": "system", "content": "You have access to the following tools:\n[{\"name\": \"get_current_weather\", \"description\": \"Get the current weather in a given location\", \"parameters\": {\"type\": \"object\", \"properties\": {\"location\": {\"type\": \"string\", \"description\": \"The city e.g. Beijing\"}, \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\"]}}, \"required\": [\"location\"]}}, {\"name\": \"calculator\", \"description\": \"\\u8ba1\\u7b97\\u5668\", \"parameters\": {\"type\": \"int\", \"properties\": {\"a\": {\"type\": \"int\", \"description\": \"the first number\"}, \"b\": {\"type\": \"int\", \"description\": \"the second number\"}}, \"required\": [\"a\", \"b\"]}}]\nYou can select one of the above tools or just response user's content and respond with only a JSON object matching the following schema:\n{\n \"tool\": <name of the selected tool>,\n \"tool_input\": <parameters for the selected tool, matching the tool's JSON schema>,\n \"message\": <direct response users content>\n}"}, {"role": "user", "content": "\u4f60\u597d"}]}
RESPONSE: {"model":"","object":"","choices":[{"index":0,"message":{"role":"assistant","content":"```json\n{\"tool\": null, \"tool_input\": null, \"message\": \"你好，有什麼可以幫您的嗎？\"}\n```","function_call":null},"finish_reason":"stop"}],"queueTime":0.0020923614501953125,"costTime":0.7685532569885254,"usage":{"prompt_token":244,"completion_token":29,"total_tokens":273}}
=======================================================================
FINAL RESULT: 你好，有什麼可以幫您的嗎？
Enter a string: 廈門天氣如何？
PAYLOAD: {"model": "", "messages": [{"role": "system", "content": "You have access to the following tools:\n[{\"name\": \"get_current_weather\", \"description\": \"Get the current weather in a given location\", \"parameters\": {\"type\": \"object\", \"properties\": {\"location\": {\"type\": \"string\", \"description\": \"The city e.g. Beijing\"}, \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\"]}}, \"required\": [\"location\"]}}, {\"name\": \"calculator\", \"description\": \"\\u8ba1\\u7b97\\u5668\", \"parameters\": {\"type\": \"int\", \"properties\": {\"a\": {\"type\": \"int\", \"description\": \"the first number\"}, \"b\": {\"type\": \"int\", \"description\": \"the second number\"}}, \"required\": [\"a\", \"b\"]}}]\nYou can select one of the above tools or just response user's content and respond with only a JSON object matching the following schema:\n{\n \"tool\": <name of the selected tool>,\n \"tool_input\": <parameters for the selected tool, matching the tool's JSON schema>,\n \"message\": <direct response users content>\n}"}, {"role": "user", "content": "\u53a6\u95e8\u5929\u6c14\u5982\u4f55\uff1f"}]}
RESPONSE: {"model":"","object":"","choices":[{"index":0,"message":{"role":"assistant","content":"```json\n{\"tool\": \"get_current_weather\", \"tool_input\": {\"location\": \"Xiamen\", \"unit\": \"celsius\"}, \"message\": \"\"}\n```","function_call":null},"finish_reason":"stop"}],"queueTime":0.0021338462829589844,"costTime":0.9370713233947754,"usage":{"prompt_token":247,"completion_token":36,"total_tokens":283}}
=======================================================================
PAYLOAD: {"model": "", "messages": [{"role": "user", "content": "\nPlease generate a natural language description based on the following question and answer.\nQuestion: [Content of the question]\nAnswer: [Content of the answer]\nGenerated Description: The result of [key phrase from the question] is [answer].\nIf necessary, you can polish the description.\nOnly output the Description, with Chinese language.\n\n\nThe question:\u53a6\u95e8\u5929\u6c14\u5982\u4f55\uff1f\nThe answer:The weather of Xiamen is cloudy, and the temperature is 35\u00b0C."}]}
RESPONSE: {"model":"","object":"","choices":[{"index":0,"message":{"role":"assistant","content":"廈門天氣情況是：多雲，氣溫35°C。","function_call":null},"finish_reason":"stop"}],"queueTime":0.008246660232543945,"costTime":0.3240656852722168,"usage":{"prompt_token":143,"completion_token":12,"total_tokens":155}}
=======================================================================
FINAL RESULT WITH FUNCTION CALL: 廈門天氣情況是：多雲，氣溫35°C。
Enter a string: 383加上135721等於多少？
PAYLOAD: {"model": "", "messages": [{"role": "system", "content": "You have access to the following tools:\n[{\"name\": \"get_current_weather\", \"description\": \"Get the current weather in a given location\", \"parameters\": {\"type\": \"object\", \"properties\": {\"location\": {\"type\": \"string\", \"description\": \"The city e.g. Beijing\"}, \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\"]}}, \"required\": [\"location\"]}}, {\"name\": \"calculator\", \"description\": \"\\u8ba1\\u7b97\\u5668\", \"parameters\": {\"type\": \"int\", \"properties\": {\"a\": {\"type\": \"int\", \"description\": \"the first number\"}, \"b\": {\"type\": \"int\", \"description\": \"the second number\"}}, \"required\": [\"a\", \"b\"]}}]\nYou can select one of the above tools or just response user's content and respond with only a JSON object matching the following schema:\n{\n \"tool\": <name of the selected tool>,\n \"tool_input\": <parameters for the selected tool, matching the tool's JSON schema>,\n \"message\": <direct response users content>\n}"}, {"role": "user", "content": "383\u52a0\u4e0a135721\u7b49\u4e8e\u591a\u5c11\uff1f"}]}
RESPONSE: {"model":"","object":"","choices":[{"index":0,"message":{"role":"assistant","content":"```json\n{\"tool\": \"calculator\", \"tool_input\": {\"a\": 383, \"b\": 135721}, \"message\": null}\n```","function_call":null},"finish_reason":"stop"}],"queueTime":0.0021514892578125,"costTime":0.9161381721496582,"usage":{"prompt_token":252,"completion_token":35,"total_tokens":287}}
=======================================================================
PAYLOAD: {"model": "", "messages": [{"role": "user", "content": "\nPlease generate a natural language description based on the following question and answer.\nQuestion: [Content of the question]\nAnswer: [Content of the answer]\nGenerated Description: The result of [key phrase from the question] is [answer].\nIf necessary, you can polish the description.\nOnly output the Description, with Chinese language.\n\n\nThe question:383\u52a0\u4e0a135721\u7b49\u4e8e\u591a\u5c11\uff1f\nThe answer:136104"}]}
RESPONSE: {"model":"","object":"","choices":[{"index":0,"message":{"role":"assistant","content":"383加上135721等於136104。","function_call":null},"finish_reason":"stop"}],"queueTime":0.0064160823822021484,"costTime":0.28981900215148926,"usage":{"prompt_token":134,"completion_token":11,"total_tokens":145}}
=======================================================================
FINAL RESULT WITH FUNCTION CALL: 383加上135721等於136104。在這個例子中，預置了兩個函式，分別為天氣查詢和計算器，實驗效果中進行了三輪，其中第一次屬於未命中函式呼叫的閒聊場景，後兩次分別命中了天氣查詢和計算器。
在實際的工作中，可能需要預置非常多函式能力，此時可能需要考慮到LLM的輸入token限制，必要時需要進行模組劃分，將一次LLM決策轉化為多次決策，更通用一點的說法就是意圖層級識別。