基于 Llama3 与 LangChain 搭建本地私有知识库的 RAG 实践

4. 检索与增强

向量数据库加载完成后，可作为检索器使用。通过用户查询的嵌入向量与数据库中的向量计算相似度，获取最相关的上下文片段。

from langchain.prompts import ChatPromptTemplate

# 定义检索器
retriever = vectorstore.as_retriever()

# 定义聊天提示模板
template = """You are an assistant for question-answering tasks. 
   Use the following pieces of retrieved context to answer the question. 
   If you don't know the answer, just say that you don't know. 
   Use three sentences maximum and keep the answer concise.
   Question: {question} 
   Context: {context} 
   Answer:
   """
prompt = ChatPromptTemplate.from_template(template)

5. 生成回答

最后，将检索器、提示模板和 LLM 组合成 RAG 链。

from langchain_community.chat_models import ChatOllama
from langchain.schema.runnable import RunnablePassthrough
from langchain.schema.output_parser import StrOutputParser

# 初始化 LLM，注意 temperature 参数通常设为 0.7 左右
llm = ChatOllama(model="llama3", temperature=0.7)

# 构建 RAG 链
rag_chain = (
        {"context": retriever, "question": RunnablePassthrough()} 
        | prompt
        | llm
        | StrOutputParser()
)

# 执行查询
query = "What did the president mainly say?"
result = rag_chain.invoke(query)
print(result)

运行上述代码，LLM 将基于检索到的上下文生成回答。例如，针对国情咨文内容，模型会总结总统关于新冠疫情、投资政策等核心观点。

完整代码示例

为了方便集成，以下是整合后的完整脚本：

from langchain_community.document_loaders import TextLoader
from langchain.text_splitter import CharacterTextSplitter
from langchain_community.embeddings import OllamaEmbeddings
import weaviate
from weaviate.embedded import EmbeddedOptions
from langchain.prompts import ChatPromptTemplate
from langchain_community.chat_models import ChatOllama
from langchain.schema.runnable import RunnablePassthrough
from langchain.schema.output_parser import StrOutputParser
from langchain_community.vectorstores import Weaviate
import requests

# 1. 下载数据
url = "https://raw.githubusercontent.com/langchain-ai/langchain/master/docs/docs/modules/state_of_the_union.txt"
res = requests.get(url)
with open("state_of_the_union.txt", "w") as f:
    f.write(res.text)

# 2. 加载数据
loader = TextLoader('./state_of_the_union.txt')
documents = loader.load()

# 3. 文本分块
text_splitter = CharacterTextSplitter(chunk_size=500, chunk_overlap=50)
chunks = text_splitter.split_documents(documents)

# 4. 初始化向量数据库并嵌入目标文档
client = weaviate.Client(
    embedded_options=EmbeddedOptions()
)
vectorstore = Weaviate.from_documents(
    client=client,
    documents=chunks,
    embedding=OllamaEmbeddings(model="llama3"),
    by_text=False
)

# 5. 检索器
retriever = vectorstore.as_retriever()

# 6. LLM 提示模板
template = """You are an assistant for question-answering tasks. 
   Use the following pieces of retrieved context to answer the question. 
   If you don't know the answer, just say that you don't know. 
   Use three sentences maximum and keep the answer concise.
   Question: {question} 
   Context: {context} 
   Answer:
   """
prompt = ChatPromptTemplate.from_template(template)

# 7. 初始化 LLM
llm = ChatOllama(model="llama3", temperature=0.7)

# 8. 构建 RAG 链
rag_chain = (
        {"context": retriever, "question": RunnablePassthrough()}
        | prompt
        | llm
        | StrOutputParser()
)

# 9. 开始查询&生成
query = "What did the president mainly say?"
print(rag_chain.invoke(query))

常见问题与排查

1. Ollama 连接失败：确保 Ollama 服务已启动且端口可访问。默认端口为 11434。
2. 模型未找到：确认已在 Ollama 中拉取 llama3 模型 (ollama pull llama3)。
3. Weaviate 启动慢：嵌入式模式首次启动可能需要时间初始化数据库，请耐心等待。
4. 温度参数错误：原示例中 temperature=10 会导致输出异常，建议调整为 0.7 以获得平衡的创造性与准确性。

优化建议

• 分块策略：根据文档类型调整 chunk_size。长文档可适当增大重叠率以保证上下文连贯性。
• 混合检索：除了向量检索，可结合关键词检索（BM25）提高准确率。
• 重排序：在检索后增加重排序（Rerank）步骤，进一步筛选最相关的片段。

通过上述步骤，您可以快速构建一个基于本地大模型的私有知识库系统，有效保护数据隐私并提升问答准确性。