ContextQMD
Back to Llama Index

Cerebras

docs/examples/llm/cerebras.ipynb

0.14.213.2 KB
Original Source

<a href="https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/examples/llm/cerebras.ipynb" target="_parent"></a>

Cerebras

At Cerebras, we've developed the world's largest and fastest AI processor, the Wafer-Scale Engine-3 (WSE-3). The Cerebras CS-3 system, powered by the WSE-3, represents a new class of AI supercomputer that sets the standard for generative AI training and inference with unparalleled performance and scalability.

With Cerebras as your inference provider, you can:

  • Achieve unprecedented speed for AI inference workloads
  • Build commercially with high throughput
  • Effortlessly scale your AI workloads with our seamless clustering technology

Our CS-3 systems can be quickly and easily clustered to create the largest AI supercomputers in the world, making it simple to place and run the largest models. Leading corporations, research institutions, and governments are already using Cerebras solutions to develop proprietary models and train popular open-source models.

Want to experience the power of Cerebras? Check out our website for more resources and explore options for accessing our technology through the Cerebras Cloud or on-premise deployments!

For more information about Cerebras Cloud, visit cloud.cerebras.ai. Our API reference is available at inference-docs.cerebras.ai.

Setup

If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙.

python
% pip install llama-index-llms-cerebras
python
!pip install llama-index
python
from llama_index.llms.cerebras import Cerebras

Get an API Key from cloud.cerebras.ai and add it to your environment variables:

bash
export CEREBRAS_API_KEY=<your api key>

Alternatively, you can pass your API key to the LLM when you init it:

python
import os
import getpass

os.environ["CEREBRAS_API_KEY"] = getpass.getpass(
    "Enter your Cerebras API key: "
)

llm = Cerebras(model="llama-3.3-70b", api_key=os.environ["CEREBRAS_API_KEY"])

A list of available LLM models can be found at inference-docs.cerebras.ai.

python
response = llm.complete("What is Generative AI?")
python
print(response)

Call chat with a list of messages

python
from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="What is your name"),
]
resp = llm.chat(messages)
python
print(resp)

Streaming

Using stream_complete endpoint

python
response = llm.stream_complete("What is Generative AI?")
python
for r in response:
    print(r.delta, end="")

Using stream_chat endpoint

python
from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="What is your name"),
]
resp = llm.stream_chat(messages)
python
for r in resp:
    print(r.delta, end="")