Skip to main content
Friendli 通过可扩展、高效的部署选项增强 AI 应用性能并优化成本节省，专为高需求 AI 工作负载量身定制。
本教程将指导您使用 LangChain 集成 ChatFriendli 以构建聊天应用程序。ChatFriendli 提供了一种灵活的方式来生成对话式 AI 响应，同时支持同步和异步调用。

​
设置

确保已安装 langchain_community 和 friendli-client。 
pip install -U langchain-community friendli-client.
登录 Friendli Suite 创建个人访问令牌，并将其设置为 FRIENDLI_TOKEN 环境变量。 
import getpass
import os

if "FRIENDLI_TOKEN" not in os.environ:
    os.environ["FRIENDLI_TOKEN"] = getpass.getpass("Friendi Personal Access Token: ")
您可以通过选择要使用的模型来初始化 Friendli 聊天模型。默认模型为 mixtral-8x7b-instruct-v0-1。您可以在 docs.friendli.ai 查看可用模型。 
from langchain_community.chat_models.friendli import ChatFriendli

chat = ChatFriendli(model="meta-llama-3.1-8b-instruct", max_tokens=100, temperature=0)

​
使用方法

FrienliChat 支持 ChatModel 的所有方法，包括异步 API。 您还可以使用 invoke、batch、generate 和 stream 等功能。 
from langchain.messages import HumanMessage, SystemMessage

system_message = SystemMessage(content="Answer questions as short as you can.")
human_message = HumanMessage(content="Tell me a joke.")
messages = [system_message, human_message]

chat.invoke(messages)

AIMessage(content="Why don't eggs tell jokes? They'd crack each other up.", additional_kwargs={}, response_metadata={}, id='run-d47c1056-54e8-4ea9-ad63-07cf74b834b7-0')

chat.batch([messages, messages])

[AIMessage(content="Why don't eggs tell jokes? They'd crack each other up.", additional_kwargs={}, response_metadata={}, id='run-36775b84-2a7a-48f0-8c68-df23ffffe4b2-0'),
 AIMessage(content="Why don't eggs tell jokes? They'd crack each other up.", additional_kwargs={}, response_metadata={}, id='run-b204be41-bc06-4d3a-9f74-e66ab1e60e4f-0')]

chat.generate([messages, messages])

LLMResult(generations=[[ChatGeneration(text="Why don't eggs tell jokes? They'd crack each other up.", message=AIMessage(content="Why don't eggs tell jokes? They'd crack each other up.", additional_kwargs={}, response_metadata={}, id='run-2e4cb949-8c51-40d5-92a0-cd0ac577db83-0'))], [ChatGeneration(text="Why don't eggs tell jokes? They'd crack each other up.", message=AIMessage(content="Why don't eggs tell jokes? They'd crack each other up.", additional_kwargs={}, response_metadata={}, id='run-afcdd1be-463c-4e50-9731-7a9f5958e396-0'))]], llm_output={}, run=[RunInfo(run_id=UUID('2e4cb949-8c51-40d5-92a0-cd0ac577db83')), RunInfo(run_id=UUID('afcdd1be-463c-4e50-9731-7a9f5958e396'))], type='LLMResult')

for chunk in chat.stream(messages):
    print(chunk.content, end="", flush=True)

Why don't eggs tell jokes? They'd crack each other up.
您也可以使用所有异步 API 功能：ainvoke、abatch、agenerate 和 astream。 
await chat.ainvoke(messages)

AIMessage(content="Why don't eggs tell jokes? They'd crack each other up.", additional_kwargs={}, response_metadata={}, id='run-ba8062fb-68af-47b8-bd7b-d1e01b914744-0')

await chat.abatch([messages, messages])

[AIMessage(content="Why don't eggs tell jokes? They'd crack each other up.", additional_kwargs={}, response_metadata={}, id='run-5d2c77ab-2637-45da-8bbe-1b1f18a22369-0'),
 AIMessage(content="Why don't eggs tell jokes? They'd crack each other up.", additional_kwargs={}, response_metadata={}, id='run-f1338470-8b52-4d6e-9428-a694a08ae484-0')]

await chat.agenerate([messages, messages])

LLMResult(generations=[[ChatGeneration(text="Why don't eggs tell jokes? They'd crack each other up.", message=AIMessage(content="Why don't eggs tell jokes? They'd crack each other up.", additional_kwargs={}, response_metadata={}, id='run-d4e44569-39cc-40cc-93fc-de53e599fd51-0'))], [ChatGeneration(text="Why don't eggs tell jokes? They'd crack each other up.", message=AIMessage(content="Why don't eggs tell jokes? They'd crack each other up.", additional_kwargs={}, response_metadata={}, id='run-54647cc2-bee3-4154-ad00-2e547993e6d7-0'))]], llm_output={}, run=[RunInfo(run_id=UUID('d4e44569-39cc-40cc-93fc-de53e599fd51')), RunInfo(run_id=UUID('54647cc2-bee3-4154-ad00-2e547993e6d7'))], type='LLMResult')

async for chunk in chat.astream(messages):
    print(chunk.content, end="", flush=True)

Why don't eggs tell jokes? They'd crack each other up.
Edit this page on GitHub or file an issue.
Connect these docs to Claude, VSCode, and more via MCP for real-time answers.