ContextQMD
Back to Llama Index

Nebius LLMs

docs/examples/llm/nebius.ipynb

0.14.212.1 KB
Original Source

<a href="https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/examples/llm/nebius.ipynb" target="_parent"></a>

Nebius LLMs

This notebook demonstrates how to use LLMs from Nebius AI Studio with LlamaIndex. Nebius AI Studio implements all state-of-the-art LLMs available for commercial use.

First, let's install LlamaIndex and dependencies of Nebius AI Studio.

python
%pip install llama-index-llms-nebius llama-index

Upload your Nebius AI Studio key from system variables below or simply insert it. You can get it by registering for free at Nebius AI Studio and issuing the key at API Keys section."

python
import os

NEBIUS_API_KEY = os.getenv("NEBIUS_API_KEY")  # NEBIUS_API_KEY = ""
python
from llama_index.llms.nebius import NebiusLLM

llm = NebiusLLM(
    api_key=NEBIUS_API_KEY, model="meta-llama/Llama-3.3-70B-Instruct-fast"
)

Call complete with a prompt

python
response = llm.complete("Amsterdam is the capital of ")
print(response)

Call chat with a list of messages

python
from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(role="system", content="You are a helpful AI assistant."),
    ChatMessage(
        role="user",
        content="Answer briefly: who is Wall-e?",
    ),
]
response = llm.chat(messages)
print(response)

Streaming

Using stream_complete endpoint

python
response = llm.stream_complete("Amsterdam is the capital of ")
for r in response:
    print(r.delta, end="")

Using stream_chat with a list of messages

python
from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(role="system", content="You are a helpful AI assistant."),
    ChatMessage(
        role="user",
        content="Answer briefly: who is Wall-e?",
    ),
]
response = llm.stream_chat(messages)
for r in response:
    print(r.delta, end="")