在 Model I/O 这一流程中,LangChain 抽象的组件主要有三个:Language Model、Prompts、Output Parser。
下面展开介绍一下。
⚠️ 注:下面涉及的所有代码示例中的 OPENAI_API_KEY 和 OPENAI_BASE_URL 需要提前配置好,OPENAI_API_KEY 指 OpenAI/OpenAI 代理服务的 API Key,OPENAI_BASE_URL 指 OpenAI 代理服务的 Base Url。
Language Model
Language Model 是真正与 LLM / ChatModel 进行交互的组件,它可以直接被当作普通的 openai client 来使用,在 LangChain 中,主要使用到的是 LLM,Chat Model 和 Embedding 三类 Language Model。
LLM: 最基础的通过'text in ➡️ text out'模式来使用的 Language Model,另一方面,LangChain 也收录了大量的相关模型。
from langchain.llms import OpenAI
llm = OpenAI(model_name="text-ada-001", openai_api_key=OPENAI_API_KEY, openai_api_base=OPENAI_BASE_URL)
llm("What day comes after Friday?")
# '\n\nSaturday.'
Chat Model: LLM 的变体,抽象了 Chat 这一场景下的使用模式,由'text in ➡️ text out'变成了'chat messages in ➡️ chat message out',chat message 是指 text + message type(System, Human, AI)。
from langchain.chat_models import ChatOpenAI
from langchain.schema import HumanMessage, SystemMessage
chat = ChatOpenAI(model_name="gpt-4-0613", temperature=1, openai_api_key=OPENAI_API_KEY, openai_api_base=OPENAI_BASE_URL)
chat(
[
SystemMessage(content="You are an expert on large language models and can answer any questions related to large language models."),
HumanMessage(content="What's the difference between Generic Language Models, Instruction Tuned Models and Dialog Tuned Models")
]
)
# AIMessage(content='Generic Language Models, Instruction-Tuned Models, and Dialog-Tuned Models are all various types of language models that have been trained according to different datasets and methodologies...')
from langchain.embeddings import OpenAIEmbeddings
embeddings = OpenAIEmbeddings(openai_api_key=OPENAI_API_KEY, openai_api_base=OPENAI_BASE_URL)
text_embedding = embeddings.embed_query("To embed text(it can have any length)")
print (f"Your embedding's length: {len(text_embedding)}")
print (f"Here's a sample: {text_embedding[:5]}...")
'''
Your embedding's length: 1536
Here's a sample: [-0.03194352, 0.009228715, 0.00807182, 0.0077545005, 0.008256923]...
'''
Prompts
Prompt 指用户的一系列指令和输入,是决定 Language Model 输出内容的唯一输入,主要用于帮助模型理解上下文并生成相关和连贯的输出,如回答问题、拓写句子和总结问题。在 LangChain 中的相关组件主要有 Prompt Template 和 Example selectors,以及后面会提到的辅助/补充 Prompt 的一些其它组件。
Prompt Template: 预定义的一系列指令和输入参数的 prompt 模版,支持更加灵活的输入,如支持 output instruction(输出格式指令), partial input(提前指定部分输入参数), examples(输入输出示例) 等;LangChain 提供了大量方法来创建 Prompt Template,有了这一层组件就可以在不同 Language Model 和不同 Chain 下大量复用 Prompt Template 了,Prompt Template 中也会有下面将提到的 Example selectors, Output Parser 的参与。
Example selectors: 在很多场景下,单纯的 instruction + input 的 prompt 不足以让 LLM 完成高质量的推理回答,这时候我们就还需要为 prompt 补充一些针对具体问题的示例,LangChain 将这一功能抽象为了 Example selectors 这一组件,我们可以基于关键字,相似度 (通常使用 MMR/cosine similarity/ngram 来计算相似度,在后面的向量数据库章节中会提到)。为了让最终的 prompt 不超过 Language Model 的 token 上限(各个模型的 token 上限见下表),LangChain 还提供了 LengthBasedExampleSelector,根据长度来限制 example 数量,对于较长的输入,它会选择包含较少示例的提示,而对于较短的输入,它会选择包含更多示例。
Output Parser
通常我们希望 Language Model 的输出是固定的格式,以支持我们解析其输出为结构化数据,LangChain 将这一诉求所需的功能抽象成了 Output Parser 这一组件,并提供了一系列的预定义 Output Parser,如最常用的 Structured output parser, List parser,以及在 LLM 输出无法解析时发挥作用的 Auto-fixing parser 和 Retry parser。
from langchain.chat_models import ChatOpenAI
from langchain.prompts import PromptTemplate
from langchain.output_parsers import StructuredOutputParser, ResponseSchema
from langchain.chains import LLMChain
llm = ChatOpenAI(temperature=0.5, model_name="gpt-3.5-turbo-16k-0613", openai_api_key=OPENAI_API_KEY, openai_api_base=OPENAI_BASE_URL)
template = """
## Input
{text}
## Instruction
Please summarize the piece of text in the input part above.
Respond in a manner that a 5 year old would understand.
{format_instructions}
YOUR RESPONSE:
"""# 创建一个 Output Parser,包含两个输出字段,并指定类型和说明
output_parser = StructuredOutputParser.from_response_schemas(
[
ResponseSchema(name="keywords", type="list", description="keywords of the text"),
ResponseSchema(name="summary", type="string", description="summary of the text"),
]
)
# 创建 Prompt Template,并将 format_instructions 通过 partial_variables 直接指定为 Output Parser 的 format
prompt = PromptTemplate(
input_variables=["text"],
template=template,
partial_variables={"format_instructions": output_parser.get_format_instructions()},
)
# 创建 Chain 并绑定 Prompt Template 和 Output Parser(它将自动使用 Output Parser 解析 llm 输出)
summarize_chain = LLMChain(llm=llm, verbose=True, prompt=prompt, output_parser=output_parser)
to_summarize_text = 'Abstract. Text-to-SQL aims at generating SQL queries for the given natural language questions and thus helping users to query databases. Prompt learning with large language models (LLMs) has emerged as a recent approach, which designs prompts to lead LLMs to understand the input question and generate the corresponding SQL. However, it faces challenges with strict SQL syntax requirements. Existing work prompts the LLMs with a list of demonstration examples (i.e. question-SQL pairs) to generate SQL, but the fixed prompts can hardly handle the scenario where the semantic gap between the retrieved demonstration and the input question is large.'
output = summarize_chain.predict(text=to_summarize_text)
import json
print (json.dumps(output, indent=4))
输出如下:
{"keywords":["Text-to-SQL","SQL queries","natural language questions","databases","prompt learning","large language models","LLMs","SQL syntax requirements","demonstration examples","semantic gap"],"summary":"Text-to-SQL is a method that helps users generate SQL queries for their questions about databases. One approach is to use large language models to understand the question and generate the SQL. However, this approach faces challenges with strict SQL syntax rules. Existing methods use examples to teach the language models, but they struggle when the examples are very different from the question."}
Data connection
正如我在文章开头的 LangChain 是什么一节中提到的,集成外部数据到 Language Model 中是 LangChain 提供的核心能力之一,也是市面上很多优秀的大语言模型应用成功的核心之一(例如 YouTube 视频总结助手…),在 LangChain 中,Data connection 这一层主要包含以下四个抽象组件:
from langchain.chat_models import ChatOpenAI
from langchain.vectorstores import FAISS
from langchain.chains import RetrievalQA
from langchain.document_loaders import TextLoader
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.text_splitter import RecursiveCharacterTextSplitter
# 初始化 LLM
llm = ChatOpenAI(temperature=0.5, model_name="gpt-3.5-turbo-16k-0613", openai_api_key=OPENAI_API_KEY, openai_api_base=OPENAI_BASE_URL)
# 加载文档
loader = TextLoader('path/to/related/document')
doc = loader.load()
print (f"You have {len(doc)} document")
print (f"You have {len(doc[0].page_content)} characters in that document")
# 分割字符串
text_splitter = RecursiveCharacterTextSplitter(chunk_size=3000, chunk_overlap=400)
docs = text_splitter.split_documents(doc)
num_total_characters = sum([len(x.page_content) for x in docs])
print (f"Now you have {len(docs)} documents that have an average of {num_total_characters / len(docs):,.0f} characters (smaller pieces)")
# 初始化向量化模型
embeddings = OpenAIEmbeddings(openai_api_key=OPENAI_API_KEY, openai_api_base=OPENAI_BASE_URL)
# 向量化 Document 并存入向量数据库(绑定向量和对应 Document 元数据),这里我们选择本地最常用的 FAISS 数据库# 注意:这会向 OpenAI 产生请求并产生费用
doc_search = FAISS.from_documents(docs, embeddings)
qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=doc_search.as_retriever(), verbose=True)
query = "Specific questions to be asked"
qa.run(query)
执行后 verbose 输出日志如下:
You have 1 document
You have 74663 characters in that document
Now you have 29 documents that have an average of 2,930 characters (smaller pieces)
Entering new chain… Prompt after formatting:
System: Use the following pieces of context to answer the users question. If you don't know the answer, just say that you don't know, don't try to make up an answer.
Human: What does the author describe as good work?
The author describes working on things that aren't prestigious as a sign of good work. They believe that working on unprestigious types of work can lead to the discovery of something real and that it indicates having the right kind of motives. The author also mentions that working on things that last, such as paintings, is considered good work.
Chains
接下来就是 LangChain 中的主角——Chains 了,Chains 是 LangChain 中为连接多次与 Language Model 的交互过程而抽象的重要组件,它可以将多个组件组合在一起以创建一个单一的、连贯的任务,也可以嵌套多个 Chain 组合在一起,或者将 Chain 与其他组件组合来构建更复杂的 Chain。
from langchain.chains.router import MultiPromptChain
from langchain.llms import OpenAI
from langchain.chains import ConversationChain
from langchain.chains.llm import LLMChain
from langchain.prompts import PromptTemplate
from langchain.chains.router.llm_router import LLMRouterChain, RouterOutputParser
from langchain.chains.router.multi_prompt_prompt import MULTI_PROMPT_ROUTER_TEMPLATE
# 定义要路由的 prompts
physics_template = """You are a very smart physics professor. \You are great at answering questions about physics in a concise and easy to understand manner. \When you don't know the answer to a question you admit that you don't know.
Here is a question:
{input}"""
math_template = """You are a very good mathematician. You are great at answering math questions. \You are so good because you are able to break down hard problems into their component parts, \answer the component parts, and then put them together to answer the broader question.
Here is a question:
{input}"""# 整理 prompt 和相关信息
prompt_infos = [
{
"name": "physics",
"description": "Good for answering questions about physics",
"prompt_template": physics_template,
},
{
"name": "math",
"description": "Good for answering math questions",
"prompt_template": math_template,
},
]
llm = OpenAI(temperature=0.5, openai_api_key=OPENAI_API_KEY, openai_api_base=OPENAI_BASE_URL+"/v1")
destination_chains = {}
for p_info in prompt_infos:
# 以每个 prompt 为基础创建一个 destination_chain(开启 verbose)
name = p_info["name"]
prompt_template = p_info["prompt_template"]
prompt = PromptTemplate(template=prompt_template, input_variables=["input"])
chain = LLMChain(llm=llm, prompt=prompt)
destination_chains[name] = chain
# 创建一个缺省 chain,如果没有其他 chain 满足路由条件,则使用该 chain
default_chain = ConversationChain(llm=llm, output_key="text")
destinations = [f"{p['name']}: {p['description']}"for p in prompt_infos]
destinations_str = "\n".join(destinations)
# 根据 prompt_infos 中的映射关系创建 router_prompt
router_template = MULTI_PROMPT_ROUTER_TEMPLATE.format(destinations=destinations_str)
router_prompt = PromptTemplate(
template=router_template,
input_variables=["input"],
output_parser=RouterOutputParser(),
)
# 创建 router_chain(开启 verbose)
router_chain = LLMRouterChain(llm_chain=LLMChain(llm=llm, prompt=router_prompt, verbose=True), verbose=True)
# 将 router_chain 和 destination_chains 以及 default_chain 组合成 MultiPromptChain(开启 verbose)
chain = MultiPromptChain(
router_chain=router_chain,
destination_chains=destination_chains,
default_chain=default_chain,
verbose=True,
)
# run
chain.run("What is black body radiation?")
执行后 verbose 输出日志如下:
Entering new chain… Prompt after formatting: Given a raw text input to a language model select the model prompt best suited for the input. You will be given the names of the available prompts and a description of what the prompt is best suited for. You may also revise the original input if you think that revising it will ultimately lead to a better response from the language model.
<< FORMATTING >> Return a markdown code snippet with a JSON object formatted to look like:
{"destination": string \ name of the prompt to use or "DEFAULT""next_inputs": string \ a potentially modified version of the original input
}
REMEMBER: 'destination' MUST be one of the candidate prompt names specified below OR it can be 'DEFAULT' if the input is not well suited for any of the candidate prompts. REMEMBER: 'next_inputs' can just be the original input if you don't think any modifications are needed.
<< CANDIDATE PROMPTS >> physics: Good for answering questions about math math: Good for answering math questions
<< INPUT >>
What is black body radiation?
<< OUTPUT >>
Finished chain. physics: {'input': 'What is black body radiation?' Entering new chain… Prompt after formatting:
You are a very smart physics professor. You are great at answering questions about physics in a concise and easy to understand manner. When you don't know the answer to a question you admit that you don't know.
Here is a question:
What is black body radiation?
Finished chain.
Memory
Memory 可以帮助 Language Model 补充历史信息的上下文,LangChain 中的 Memory 是一个有点模糊的术语,它可以像记住你过去聊天过的信息一样简单,也可以结合向量数据库做更加复杂的历史信息检索,甚至维护相关实体及其关系的具体信息,这取决于具体的应用。
from langchain.chains import ConversationChain
from langchain.memory import ConversationSummaryBufferMemory
from langchain.llms import OpenAI
from langchain.schema import SystemMessage, AIMessage, HumanMessage
from langchain.memory.prompt import SUMMARY_PROMPT
from langchain.prompts import PromptTemplate
llm = OpenAI(temperature=0.7, openai_api_key=OPENAI_API_KEY, openai_api_base=OPENAI_BASE_URL+"/v1")
# ConversationSummaryBufferMemory 默认使用 langchain.memory.prompt.SUMMARY_PROMPT 作为 summary 的 PromptTemplate# 如果对它 summary 的格式/内容有特殊要求,可以自定义 PromptTemplate(实测默认的 summary 有些流水账)
prompt_template_str = """
## Instruction
Progressively summarize the lines of conversation provided, adding onto the previous summary returning a new concise and detailed summary.
Don't repeat the conversation directly in the summary, extract key information instead.
## EXAMPLE
Current summary:
The human asks what the AI thinks of artificial intelligence. The AI thinks artificial intelligence is a force for good.
New lines of conversation:
Human: Why do you think artificial intelligence is a force for good?
AI: Because artificial intelligence will help humans reach their full potential.
New summary:
The human inquires about the AI's opinion on artificial intelligence. The AI believes that it is a force for good as it can help humans reach their full potential.
## Current summary
{summary}
## New lines of conversation
{new_lines}
## New summary
"""
prompt = PromptTemplate(
input_variables=SUMMARY_PROMPT.input_variables, # input_variables 为 SUMMARY_PROMPT 中的 input_variables 不变
template=prompt_template_str, # template 替换为上面重新编写的 prompt_template_str
)
memory = ConversationSummaryBufferMemory(llm=llm, prompt=prompt, max_token_limit=60)
# 添加历史 memory,其中第一条 SystemMessage 为历史对话中 Summary 的内容,第二条 HumanMessage 和第三条 AIMessage 为历史对话中最后的对话内容
memory.chat_memory.add_message(SystemMessage(content="The human asks what the AI thinks of artificial intelligence. The AI thinks artificial intelligence is a force for good because it will help humans reach their full potential. The human then asks the difference between python and golang in short. The AI responds that python is a high-level interpreted language with an emphasis on readability and code readability, while golang is a statically typed compiled language with a focus on concurrency and performance. Python is typically used for general-purpose programming, while golang is often used for building distributed systems."))
memory.chat_memory.add_user_message("Then if I want to build a distributed system, which language should I choose?")
memory.chat_memory.add_ai_message("If you want to build a distributed system, I would recommend golang as it is a statically typed compiled language that is designed to facilitate concurrency and performance.")
# 调用 memory.prune() 确保 chat_memory 中的对话内容不超过 max_token_limit
memory.prune()
conversation_with_summary = ConversationChain(
llm=llm,
# We set a very low max_token_limit for the purposes of testing.
memory=memory,
verbose=True,
)
# memory.prune() 会在每次调用 predict() 后自动执行
conversation_with_summary.predict(input="Is there any well-known distributed system built with golang?")
conversation_with_summary.predict(input="Is there a substitutes for Kubernetes in python?")
执行后 verbose 输出日志如下:
> Entering new chain…
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Current conversation: System: The human asks the AI about its opinion on artificial intelligence and is told that it is a force for good that can help humans reach their full potential. The human then inquires about the differences between python and golang, with the AI explaining that python is a high-level interpreted language for general-purpose programming, while golang is a statically typed compiled language often used for building distributed systems. Human: Then if I want to build a distributed system, which language should I choose? AI: If you want to build a distributed system, I would recommend golang as it is a statically typed compiled language that is designed to facilitate concurrency and performance. Human: Is there any well-known distributed system built with golang?
AI:
> Finished chain.
> Entering new chain…
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Current conversation: System: The human asks the AI about its opinion on artificial intelligence and is told that it is a force for good that can help humans reach their full potential. The human then inquires about the differences between python and golang, with the AI explaining that python is a high-level interpreted language for general-purpose programming, while golang is a statically typed compiled language designed to facilitate concurrency and performance, thus better suited for distributed systems. The AI recommends golang for building distributed systems. Human: Is there any well-known distributed system built with golang? AI: Yes, there are several well-known distributed systems built with golang. These include Kubernetes, Docker, and Consul. Human: Is there a substitutes for Kubernetes in python?
AI:
'Yes, there are several substitutes for Kubernetes in python. These include Dask, Apache Mesos and Marathon, and Apache Aurora.'
抽象 Agent'决定做什么'的过程为'planning what to do'和'executing the sub tasks'(这种方法来自这一篇论文),其中'planning what to do'这一步通常完全由 LLM 完成,而'executing the sub tasks'这一任务则通常由更多的 Tools 来完成。