Back to Llama Index

LlamaIndex + Klavis AI Integration

docs/examples/tools/Use_Klavis_with_LlamaIndex.ipynb

0.14.216.6 KB
Original Source

LlamaIndex + Klavis AI Integration

This tutorial demonstrates how to build AI agents using LlamaIndex's agent framework with Klavis MCP (Model Context Protocol) servers for enhanced functionality.

Prerequisites

Before we begin, you'll need:

python
# Install the required packages
%pip install -qU llama-index llama-index-tools-mcp klavis
python
import os
from klavis import Klavis
from klavis.types import McpServerName, ConnectionType
from llama_index.llms.openai import OpenAI
from llama_index.tools.mcp import BasicMCPClient

# Set environment variables
os.environ[
    "OPENAI_API_KEY"
] = "YOUR_OPENAI_API_KEY"  # Replace with your actual OpenAI API key
os.environ[
    "KLAVIS_API_KEY"
] = "YOUR_KLAVIS_API_KEY"  # Replace with your actual Klavis API key

Case 1: YouTube AI Agent

Create an AI agent to summarize YouTube videos using LlamaIndex and Klavis MCP Server.

Step 1 - using Klavis to create youtube MCP Server

python
klavis_client = Klavis(api_key=os.getenv("KLAVIS_API_KEY"))

# Create a YouTube MCP server and get the server URL
youtube_mcp_instance = klavis_client.mcp_server.create_server_instance(
    server_name=McpServerName.YOUTUBE,
    user_id="1234",
    platform_name="Klavis",
    connection_type=ConnectionType.STREAMABLE_HTTP,
)

youtube_mcp_server_url = youtube_mcp_instance.server_url
# print(f"🔗 YouTube MCP server created at: {youtube_mcp_server_url}")

Step 2 - using Llamaindex to create AI Agent with the MCP Server

python
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.tools.mcp import (
    get_tools_from_mcp_url,
    aget_tools_from_mcp_url,
)

llm = OpenAI(model="gpt-4o-mini", api_key=os.getenv("OPENAI_API_KEY"))

youtube_tools = await aget_tools_from_mcp_url(
    youtube_mcp_server_url, client=BasicMCPClient(youtube_mcp_server_url)
)

youtube_agent = FunctionAgent(
    name="youtube_agent",
    description="Agent using MCP-based tools",
    tools=youtube_tools,
    llm=llm,
    system_prompt="You are an AI assistant that uses MCP tools.",
)

Step 3 - Run your AI Agent to summarize your favorite video!

python
YOUTUBE_VIDEO_URL = "https://www.youtube.com/watch?v=MmiveeGxfX0&t=528s"  # pick a video you like!

response = await youtube_agent.run(
    f"Summarize this video: {YOUTUBE_VIDEO_URL}"
)
print(response)

✅ Nice work! You’ve successfully oursource your eyeball and summarized your favorite YouTube video!

Case 2: Multi-Agent Workflow

Build a LlamaIndex AgentWorkflow that summarizes YouTube videos and sends the summary via email.

Step 1 - using Klavis to create YouTube and Gmail MCP Servers

python
import webbrowser

klavis_client = Klavis(api_key=os.getenv("KLAVIS_API_KEY"))

# Create YouTube MCP server
youtube_mcp_instance = klavis_client.mcp_server.create_server_instance(
    server_name=McpServerName.YOUTUBE,
    user_id="1234",
    platform_name="Klavis",
    connection_type=ConnectionType.STREAMABLE_HTTP,
)

# Create Gmail MCP server with OAuth authorization
gmail_mcp_instance = klavis_client.mcp_server.create_server_instance(
    server_name=McpServerName.GMAIL,
    user_id="1234",
    platform_name="Klavis",
    connection_type=ConnectionType.STREAMABLE_HTTP,
)

print("✅ Created YouTube and Gmail MCP instances")

# Open Gmail OAuth authorization
webbrowser.open(gmail_mcp_instance.oauth_url)
print(
    f"🔐 Opening OAuth authorization for Gmail, if you are not redirected, please open the following URL in your browser: {gmail_mcp_instance.oauth_url}"
)

Step 2 - using LlamaIndex to create Multi-Agent Workflow with the MCP Servers

python
from llama_index.llms.openai import OpenAI
from llama_index.core.agent.workflow import FunctionAgent, AgentWorkflow
from llama_index.tools.mcp import (
    BasicMCPClient,
    get_tools_from_mcp_url,
    aget_tools_from_mcp_url,
)

llm = OpenAI(model="gpt-4o-mini", api_key=os.getenv("OPENAI_API_KEY"))

# Get MCP server URLs
youtube_mcp_server_url = youtube_mcp_instance.server_url
gmail_mcp_server_url = gmail_mcp_instance.server_url

# Get tools from both MCP servers
youtube_tools = await aget_tools_from_mcp_url(
    youtube_mcp_server_url, client=BasicMCPClient(youtube_mcp_server_url)
)
gmail_tools = await aget_tools_from_mcp_url(
    gmail_mcp_server_url, client=BasicMCPClient(gmail_mcp_server_url)
)

# Create specialized agents
youtube_agent = FunctionAgent(
    name="youtube_agent",
    description="Agent that can summarize YouTube videos",
    tools=youtube_tools,
    llm=llm,
    system_prompt="You are a YouTube video summarization expert. Use MCP tools to analyze and summarize videos.",
    can_handoff_to=["gmail_agent"],
)

gmail_agent = FunctionAgent(
    name="gmail_agent",
    description="Agent that can send emails via Gmail",
    tools=gmail_tools,
    llm=llm,
    system_prompt="You are an email assistant. Use MCP tools to send emails via Gmail.",
)

# Create multi-agent workflow
workflow = AgentWorkflow(
    agents=[youtube_agent, gmail_agent],
    root_agent="youtube_agent",
)

print("🤖 Multi-agent workflow created with YouTube and Gmail agents!")

Step 3 - run the workflow!

python
YOUTUBE_VIDEO_URL = "https://www.youtube.com/watch?v=MmiveeGxfX0&t=528s"  # pick a video you like!
EMAIL_RECIPIENT = "[email protected]"  # Replace with your email

resp = await workflow.run(
    user_msg=f"Summarize this video {YOUTUBE_VIDEO_URL} and send it to {EMAIL_RECIPIENT}"
)
print("\n✅ Report:\n", resp.response.content)

Summary

In this tutorial, we explored how to integrate LlamaIndex with Klavis AI to build powerful AI agents using MCP (Model Context Protocol) servers. Here's what we accomplished:

Key Takeaways:

  1. Single Agent Setup: Created a YouTube AI agent that can summarize videos using the Klavis YouTube MCP server
  2. Multi-Agent Workflow: Built a sophisticated workflow combining YouTube and Gmail agents to summarize videos and automatically send summaries via email
  3. MCP Integration: Learned how to use Klavis MCP servers with LlamaIndex's agent framework for enhanced functionality

This integration opens up endless possibilities for building AI agents that can interact with various services and platforms through Klavis MCP servers. You can now create agents that work with YouTube, Gmail, GitHub, Slack, and many other services supported by Klavis.

Happy building! 🚀