docs/docs/genai/tracing/integrations/listing/langchain.mdx
import { APILink } from "@site/src/components/APILink"; import Tabs from "@theme/Tabs"; import TabItem from "@theme/TabItem"; import TabsWrapper from "@site/src/components/TabsWrapper"; import ImageBox from "@site/src/components/ImageBox";
LangChain is an open-source framework for building LLM-powered applications.
MLflow Tracing provides automatic tracing capability for LangChain. You can enable tracing
for LangChain by calling the <APILink fn="mlflow.langchain.autolog" /> function, and nested traces are automatically logged to the active MLflow Experiment upon invocation of chains. In TypeScript, you can pass the MLflow LangChain callback to the callbacks option.
import mlflow
mlflow.langchain.autolog()
LangChain.js tracing is supported via the OpenTelemetry ingestion. See the Getting Started section below for the full setup.
</TabItem> </Tabs> </TabsWrapper>MLflow support tracing for LangChain in both Python and TypeScript/JavaScript. Please select the appropriate tab below to get started.
<TabsWrapper> <Tabs> <TabItem value="python" label="Python" default>Start the MLflow server following the Self-Hosting Guide, if you don't have one already.
pip install langchain langchain-openai mlflow
import mlflow
# Calling autolog for LangChain will enable trace logging.
mlflow.langchain.autolog()
# Optional: Set a tracking URI and an experiment
mlflow.set_experiment("LangChain")
mlflow.set_tracking_uri("http://localhost:5000")
import mlflow
import os
from langchain.prompts import PromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0.7, max_tokens=1000)
prompt_template = PromptTemplate.from_template(
"Answer the question as if you are {person}, fully embodying their style, wit, personality, and habits of speech. "
"Emulate their quirks and mannerisms to the best of your ability, embracing their traits—even if they aren't entirely "
"constructive or inoffensive. The question is: {question}"
)
chain = prompt_template | llm | StrOutputParser()
# Let's test another call
chain.invoke({
"person": "Linus Torvalds",
"question": "Can I just set everyone's access to sudo to make things easier?",
})
Visit http://localhost:5000 (or your custom MLflow tracking server URL) to view the trace in the MLflow UI.
Start the MLflow server following the Self-Hosting Guide, if you don't have one already.
npm i langchain @langchain/core @langchain/openai @arizeai/openinference-instrumentation-langchain
Enable OpenTelemetry instrumentation for LangChain in your application:
import { NodeTracerProvider, SimpleSpanProcessor } from "@opentelemetry/sdk-trace-node";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto";
import { LangChainInstrumentation } from "@arizeai/openinference-instrumentation-langchain";
import * as CallbackManagerModule from "@langchain/core/callbacks/manager";
// Set up the OpenTelemetry
const provider = new NodeTracerProvider(
{
spanProcessors: [new SimpleSpanProcessor(new OTLPTraceExporter({
// Set MLflow tracking server URL with `/v1/traces` path. You can also use the OTEL_EXPORTER_OTLP_TRACES_ENDPOINT environment variable instead.
url: "http://localhost:5000/v1/traces",
// Set the experiment ID in the header. You can also use the OTEL_EXPORTER_OTLP_TRACES_HEADERS environment variable instead.
headers: {
"x-mlflow-experiment-id": "123",
},
}))],
}
);
provider.register();
// Enable LangChain instrumentation
const lcInstrumentation = new LangChainInstrumentation();
lcInstrumentation.manuallyInstrument(CallbackManagerModule);
Note that the createAgent API is available in LangChain.js v1.0 and later. If you are on LangChain 0.x, see the v0 example instead.
import { createAgent, tool } from "langchain";
import * as z from "zod";
const getWeather = tool(
(input) => `It's always sunny in ${input.city}!`,
{
name: "get_weather",
description: "Get the weather for a given city",
schema: z.object({
city: z.string().describe("The city to get the weather for"),
}),
}
);
const agent = createAgent({
model: "gpt-4o-mini",
tools: [getWeather],
});
await agent.invoke({
messages: [{ role: "user", content: "What's the weather in Tokyo?" }],
});
Visit http://localhost:5000 (or your custom MLflow tracking server URL) to view the trace in the MLflow UI.
Start the MLflow server following the Self-Hosting Guide, if you don't have one already.
Install the required dependencies:
npm i langchain @langchain/core @langchain/openai @arizeai/openinference-instrumentation-langchain
Enable OpenTelemetry instrumentation for LangChain in your application:
import { NodeTracerProvider, SimpleSpanProcessor } from "@opentelemetry/sdk-trace-node";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto";
import { LangChainInstrumentation } from "@arizeai/openinference-instrumentation-langchain";
import * as CallbackManagerModule from "@langchain/core/callbacks/manager";
// Set up the OpenTelemetry
const provider = new NodeTracerProvider(
{
spanProcessors: [new SimpleSpanProcessor(new OTLPTraceExporter({
// Set MLflow tracking server URL. You can also use the OTEL_EXPORTER_OTLP_TRACES_ENDPOINT environment variable instead.
url: "http://localhost:5000/v1/traces",
// Set the experiment ID in the header. You can also use the OTEL_EXPORTER_OTLP_TRACES_HEADERS environment variable instead.
headers: {
"x-mlflow-experiment-id": "123",
},
}))],
}
);
provider.register();
// Enable LangChain instrumentation
const lcInstrumentation = new LangChainInstrumentation();
lcInstrumentation.manuallyInstrument(CallbackManagerModule);
import { OpenAI } from "@langchain/openai";
import { PromptTemplate } from "@langchain/core/prompts";
const model = new OpenAI("gpt-4o-mini");
const prompt = PromptTemplate.fromTemplate("What is a good name for a company that makes {product}?");
const chain = prompt.pipe({ llm: model });
const res = await chain.invoke({ product: "colorful socks" });
console.log({ res });
Visit http://localhost:5000 (or your custom MLflow tracking server URL) to view the trace in the MLflow UI.
:::note This example above has been confirmed working with the following requirement versions:
pip install openai==1.30.5 langchain==0.2.1 langchain-openai==0.1.8 langchain-community==0.2.1 mlflow==2.14.0 tiktoken==0.7.0
:::
:::tip Image Support for LangChain Traces MLflow captures image content parts passed through LangChain models. See Multimodal Content and Attachments in Traces for details. :::
The following APIs are supported by auto tracing for LangChain.
invokebatchstreamainvokeabatchastreamget_relevant_documents (for retrievers)__call__ (for Chains and AgentExecutors)MLflow automatically tracks token usage and cost for LangChain. The token usage for each LLM call during a chain invocation will be logged in each Trace/Span and the aggregated cost and time trend are displayed in the built-in dashboard. See the Token Usage and Cost Tracking documentation for details on accessing this information programmatically.
Sometimes you may want to customize what information is logged in the traces. You can achieve this by creating a custom callback handler that inherits from <APILink fn="mlflow.langchai.langchain_tracer.MlflowLangchainTracer">MlflowLangchainTracer</APILink>. MlflowLangchainTracer is a callback handler that is injected into the langchain model inference process to log traces automatically. It starts a new span upon a set of actions of the chain such as on_chain_start, on_llm_start, and concludes it when the action is finished. Various metadata such as span type, action name, input, output, latency, are automatically recorded to the span.
The following example demonstrates how to record an additional attribute to the span when a chat model starts running.
from mlflow.langchain.langchain_tracer import MlflowLangchainTracer
class CustomLangchainTracer(MlflowLangchainTracer):
# Override the handler functions to customize the behavior. The method signature is defined by LangChain Callbacks.
def on_chat_model_start(
self,
serialized: Dict[str, Any],
messages: List[List[BaseMessage]],
*,
run_id: UUID,
tags: Optional[List[str]] = None,
parent_run_id: Optional[UUID] = None,
metadata: Optional[Dict[str, Any]] = None,
name: Optional[str] = None,
**kwargs: Any,
):
"""Run when a chat model starts running."""
attributes = {
**kwargs,
**metadata,
# Add additional attribute to the span
"version": "1.0.0",
}
# Call the _start_span method at the end of the handler function to start a new span.
self._start_span(
span_name=name or self._assign_span_name(serialized, "chat model"),
parent_run_id=parent_run_id,
span_type=SpanType.CHAT_MODEL,
run_id=run_id,
inputs=messages,
attributes=kwargs,
)
When using LangChain.js with OpenTelemetry-based tracing, you can combine the automatically generated traces with the MLflow Tracing SDK (@mlflow/core) to add custom spans, set tags, and update trace metadata within the same trace.
import { init, withSpan } from "@mlflow/core";
import { LangChainInstrumentation } from "@arizeai/openinference-instrumentation-langchain";
import * as CallbackManagerModule from "@langchain/core/callbacks/manager";
// Initialize MLflow SDK - sets up the OTel provider to capture all spans
init({
trackingUri: "http://localhost:5000",
experimentId: "<your-experiment-id>",
});
// Enable LangChain instrumentation
const lcInstrumentation = new LangChainInstrumentation();
lcInstrumentation.manuallyInstrument(CallbackManagerModule);
// Add custom MLflow spans alongside the auto-generated LangChain traces
const result = await withSpan(
{ name: "custom_step", inputs: { query: "test" } },
async (span) => {
// your LangChain application logic here
return { result: "success" };
}
);
For detailed instructions and examples, see Combining the OpenTelemetry SDK and the MLflow Tracing SDK.
Auto tracing for LangChain can be disabled globally by calling mlflow.langchain.autolog(disable=True) or mlflow.autolog(disable=True).