Weights & Biases 实验追踪集成

本笔记本介绍如何将你的 LangChain 实验追踪到一个集中的 Weights and Biases 仪表板。要了解更多关于提示工程和回调的内容，请参阅以下笔记本，其中解释了两者及其生成的仪表板效果：

查看 W&B 文章中的详细描述和示例。注意：WandbCallbackHandler 正在被弃用，推荐使用 WandbTracer。未来请使用 WandbTracer，因为它更灵活，支持更精细的日志记录。要了解更多关于 WandbTracer 的信息，请参考 agent_with_wandb_tracing 笔记本或使用以下 colab 笔记本。要了解更多关于 Weights & Biases Prompts 的信息，请参考以下 Prompts 文档。

pip install -qU  wandb
pip install -qU  pandas
pip install -qU  textstat
pip install -qU  spacy
!python -m spacy download en_core_web_sm

import os

os.environ["WANDB_API_KEY"] = ""
# os.environ["OPENAI_API_KEY"] = ""
# os.environ["SERPAPI_API_KEY"] = ""

from datetime import datetime

from langchain_community.callbacks import WandbCallbackHandler
from langchain_core.callbacks import StdOutCallbackHandler
from langchain_openai import OpenAI

记录到 Weights and Biases 的回调处理器。

参数：
    job_type (str): 任务类型。
    project (str): 要记录到的项目。
    entity (str): 要记录到的实体。
    tags (list): 要记录的标签。
    group (str): 要记录到的组。
    name (str): 运行的名称。
    notes (str): 要记录的备注。
    visualize (bool): 是否可视化运行。
    complexity_metrics (bool): 是否记录复杂度指标。
    stream_logs (bool): 是否将回调操作流式传输到 W&B

WandbCallbackHandler(...) 的默认值

visualize: bool = False,
complexity_metrics: bool = False,
stream_logs: bool = False,

注意：对于测试版工作流，我们基于 textstat 进行了默认分析，基于 spacy 进行了可视化

"""主函数。

此函数用于测试回调处理器。
场景：
1. OpenAI LLM
2. 多代链中的多个子链
3. 带工具的 Agent
"""
session_group = datetime.now().strftime("%m.%d.%Y_%H.%M.%S")
wandb_callback = WandbCallbackHandler(
    job_type="inference",
    project="langchain_callback_demo",
    group=f"minimal_{session_group}",
    name="llm",
    tags=["test"],
)
callbacks = [StdOutCallbackHandler(), wandb_callback]
llm = OpenAI(temperature=0, callbacks=callbacks)

wandb: Currently logged in as: harrison-chase. Use `wandb login --relogin` to force relogin

Tracking run with wandb version 0.14.0

Run data is saved locally in <code>/Users/harrisonchase/workplace/langchain/docs/ecosystem/wandb/run-20230318_150408-e47j1914</code>

Syncing run <strong><a href='https://wandb.ai/harrison-chase/langchain_callback_demo/runs/e47j1914' target="_blank">llm</a></strong> to <a href='https://wandb.ai/harrison-chase/langchain_callback_demo' target="_blank">Weights & Biases</a> (<a href='https://wandb.me/run' target="_blank">docs</a>)<br/>

View project at <a href='https://wandb.ai/harrison-chase/langchain_callback_demo' target="_blank">https://wandb.ai/harrison-chase/langchain_callback_demo</a>

View run at <a href='https://wandb.ai/harrison-chase/langchain_callback_demo/runs/e47j1914' target="_blank">https://wandb.ai/harrison-chase/langchain_callback_demo/runs/e47j1914</a>

wandb: WARNING The wandb callback is currently in beta and is subject to change based on updates to `langchain`. Please report any issues to https://github.com/wandb/wandb/issues with the tag `langchain`.

# WandbCallbackHandler.flush_tracker(...) 的默认值

reset: bool = True,
finish: bool = False,

flush_tracker 函数用于将 LangChain 会话记录到 Weights & Biases。它接受 LangChain 模块或 agent，至少记录提示词和生成内容以及 LangChain 模块的序列化形式到指定的 Weights & Biases 项目。默认情况下，我们重置会话而非直接结束会话。

使用场景

与 LLM 配合使用

# 场景 1 - LLM
llm_result = llm.generate(["Tell me a joke", "Tell me a poem"] * 3)
wandb_callback.flush_tracker(llm, name="simple_sequential")

Waiting for W&B process to finish... <strong style="color:green">(success).</strong>

View run <strong style="color:#cdcd00">llm</strong> at: <a href='https://wandb.ai/harrison-chase/langchain_callback_demo/runs/e47j1914' target="_blank">https://wandb.ai/harrison-chase/langchain_callback_demo/runs/e47j1914</a><br/>Synced 5 W&B file(s), 2 media file(s), 5 artifact file(s) and 0 other file(s)

Find logs at: <code>./wandb/run-20230318_150408-e47j1914/logs</code>

VBox(children=(Label(value='Waiting for wandb.init()...\r'), FloatProgress(value=0.016745895149999985, max=1.0…

Tracking run with wandb version 0.14.0

Run data is saved locally in <code>/Users/harrisonchase/workplace/langchain/docs/ecosystem/wandb/run-20230318_150534-jyxma7hu</code>

Syncing run <strong><a href='https://wandb.ai/harrison-chase/langchain_callback_demo/runs/jyxma7hu' target="_blank">simple_sequential</a></strong> to <a href='https://wandb.ai/harrison-chase/langchain_callback_demo' target="_blank">Weights & Biases</a> (<a href='https://wandb.me/run' target="_blank">docs</a>)<br/>

View project at <a href='https://wandb.ai/harrison-chase/langchain_callback_demo' target="_blank">https://wandb.ai/harrison-chase/langchain_callback_demo</a>

View run at <a href='https://wandb.ai/harrison-chase/langchain_callback_demo/runs/jyxma7hu' target="_blank">https://wandb.ai/harrison-chase/langchain_callback_demo/runs/jyxma7hu</a>

在链中使用

from langchain_classic.chains import LLMChain
from langchain_core.prompts import PromptTemplate

# 场景 2 - 链
template = """You are a playwright. Given the title of play, it is your job to write a synopsis for that title.
Title: {title}
Playwright: This is a synopsis for the above play:"""
prompt_template = PromptTemplate(input_variables=["title"], template=template)
synopsis_chain = LLMChain(llm=llm, prompt=prompt_template, callbacks=callbacks)

test_prompts = [
    {
        "title": "documentary about good video games that push the boundary of game design"
    },
    {"title": "cocaine bear vs heroin wolf"},
    {"title": "the best in class mlops tooling"},
]
synopsis_chain.apply(test_prompts)
wandb_callback.flush_tracker(synopsis_chain, name="agent")

Waiting for W&B process to finish... <strong style="color:green">(success).</strong>

View run <strong style="color:#cdcd00">simple_sequential</strong> at: <a href='https://wandb.ai/harrison-chase/langchain_callback_demo/runs/jyxma7hu' target="_blank">https://wandb.ai/harrison-chase/langchain_callback_demo/runs/jyxma7hu</a><br/>Synced 4 W&B file(s), 2 media file(s), 6 artifact file(s) and 0 other file(s)

Find logs at: <code>./wandb/run-20230318_150534-jyxma7hu/logs</code>

VBox(children=(Label(value='Waiting for wandb.init()...\r'), FloatProgress(value=0.016736786816666675, max=1.0…

Tracking run with wandb version 0.14.0

Run data is saved locally in <code>/Users/harrisonchase/workplace/langchain/docs/ecosystem/wandb/run-20230318_150550-wzy59zjq</code>

Syncing run <strong><a href='https://wandb.ai/harrison-chase/langchain_callback_demo/runs/wzy59zjq' target="_blank">agent</a></strong> to <a href='https://wandb.ai/harrison-chase/langchain_callback_demo' target="_blank">Weights & Biases</a> (<a href='https://wandb.me/run' target="_blank">docs</a>)<br/>

View project at <a href='https://wandb.ai/harrison-chase/langchain_callback_demo' target="_blank">https://wandb.ai/harrison-chase/langchain_callback_demo</a>

View run at <a href='https://wandb.ai/harrison-chase/langchain_callback_demo/runs/wzy59zjq' target="_blank">https://wandb.ai/harrison-chase/langchain_callback_demo/runs/wzy59zjq</a>

与 Agent 配合使用

from langchain.agents import create_agent, load_tools

# 场景 3 - 带工具的 Agent
tools = load_tools(["serpapi", "llm-math"], llm=llm)

agent = create_agent(
    model=llm,
    tools=tools,
    callbacks=callbacks,
)

agent.invoke(
    "Who is Leo DiCaprio's girlfriend? What is her current age raised to the 0.43 power?"
)

wandb_callback.flush_tracker(agent, reset=False, finish=True)

> Entering new AgentExecutor chain...
 I need to find out who Leo DiCaprio's girlfriend is and then calculate her age raised to the 0.43 power.
Action: Search
Action Input: "Leo DiCaprio girlfriend"
Observation: DiCaprio had a steady girlfriend in Camila Morrone. He had been with the model turned actress for nearly five years, as they were first said to be dating at the end of 2017. And the now 26-year-old Morrone is no stranger to Hollywood.
Thought: I need to calculate her age raised to the 0.43 power.
Action: Calculator
Action Input: 26^0.43
Observation: Answer: 4.059182145592686

Thought: I now know the final answer.
Final Answer: Leo DiCaprio's girlfriend is Camila Morrone and her current age raised to the 0.43 power is 4.059182145592686.

> Finished chain.

Waiting for W&B process to finish... <strong style="color:green">(success).</strong>

View run <strong style="color:#cdcd00">agent</strong> at: <a href='https://wandb.ai/harrison-chase/langchain_callback_demo/runs/wzy59zjq' target="_blank">https://wandb.ai/harrison-chase/langchain_callback_demo/runs/wzy59zjq</a><br/>Synced 5 W&B file(s), 2 media file(s), 7 artifact file(s) and 0 other file(s)

Find logs at: <code>./wandb/run-20230318_150550-wzy59zjq/logs</code>

在 GitHub 上编辑此页面或提交问题。

通过 MCP 将这些文档连接到 Claude、VSCode 等以获取实时答案。

Popular Providers

Integrations by component

使用场景

与 LLM 配合使用

在链中使用

与 Agent 配合使用

Popular Providers

Integrations by component

​使用场景

​与 LLM 配合使用

​在链中使用

​与 Agent 配合使用

使用场景

与 LLM 配合使用

在链中使用

与 Agent 配合使用