Back to Litellm

from https://cloud.langfuse.com/

cookbook/logging_observability/LiteLLM_Langfuse.ipynb

1.84.0-dev.21.1 KB
Original Source

Use LiteLLM with Langfuse

https://docs.litellm.ai/docs/observability/langfuse_integration

Install Dependencies

python
!pip install litellm langfuse

Set Env Variables

python
import litellm
from litellm import completion
import os

# from https://cloud.langfuse.com/
os.environ["LANGFUSE_PUBLIC_KEY"] = ""
os.environ["LANGFUSE_SECRET_KEY"] = ""


# OpenAI and Cohere keys
# You can use any of the litellm supported providers: https://docs.litellm.ai/docs/providers
os.environ['OPENAI_API_KEY']=""
os.environ['COHERE_API_KEY']=""

Set LangFuse as a callback for sending data

OpenAI completion call

python
# set langfuse as a callback, litellm will send the data to langfuse
litellm.success_callback = ["langfuse"]

# openai call
response = completion(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "user", "content": "Hi 👋 - i'm openai"}
  ]
)

print(response)
python
# we set langfuse as a callback in the prev cell
# cohere call
response = completion(
  model="command-nightly",
  messages=[
    {"role": "user", "content": "Hi 👋 - i'm cohere"}
  ]
)

print(response)