ContextQMD
Back to Llama Index

Fireworks

docs/examples/llm/fireworks.ipynb

0.14.211.6 KB
Original Source

<a href="https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/examples/llm/openai.ipynb" target="_parent"></a>

Fireworks

If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙.

python
%pip install llama-index llama-index-llms-fireworks

Basic Usage

python
from llama_index.llms.fireworks import Fireworks

# For updated list of available models see: https://app.fireworks.ai/models
llm = Fireworks(
    model="accounts/fireworks/models/llama-v3p1-8b-instruct",
    # api_key="some key",  # uses FIREWORKS_API_KEY env var by default
)

Call complete with a prompt

python
resp = llm.complete("Paul Graham is ")
python
print(resp)

Call chat with a list of messages

python
from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="What is your name"),
]
resp = llm.chat(messages)
python
print(resp)

Streaming

Using stream_complete endpoint

python
resp = llm.stream_complete("Paul Graham is ")
python
for r in resp:
    print(r.delta, end="")

Using stream_chat endpoint

python
from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="What is your name"),
]
resp = llm.stream_chat(messages)
python
for r in resp:
    print(r.delta, end="")