ContextQMD
Back to Llama Index

Cohere

docs/examples/llm/cohere.ipynb

0.14.212.8 KB
Original Source

<a href="https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/examples/llm/cohere.ipynb" target="_parent"></a>

Cohere

Basic Usage

Call complete with a prompt

If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙.

python
%pip install llama-index-llms-openai
%pip install llama-index-llms-cohere
python
!pip install llama-index
python
from llama_index.llms.cohere import Cohere

api_key = "Your api key"
resp = Cohere(api_key=api_key).complete("Paul Graham is ")
python
print(resp)

Call chat with a list of messages

python
from llama_index.core.llms import ChatMessage
from llama_index.llms.cohere import Cohere

messages = [
    ChatMessage(role="user", content="hello there"),
    ChatMessage(
        role="assistant", content="Arrrr, matey! How can I help ye today?"
    ),
    ChatMessage(role="user", content="What is your name"),
]

resp = Cohere(api_key=api_key).chat(
    messages, preamble_override="You are a pirate with a colorful personality"
)
python
print(resp)

Streaming

Using stream_complete endpoint

python
from llama_index.llms.cohere import Cohere

llm = Cohere(api_key=api_key)
resp = llm.stream_complete("Paul Graham is ")
python
for r in resp:
    print(r.delta, end="")

Using stream_chat endpoint

python
from llama_index.llms.openai import OpenAI

llm = Cohere(api_key=api_key)
messages = [
    ChatMessage(role="user", content="hello there"),
    ChatMessage(
        role="assistant", content="Arrrr, matey! How can I help ye today?"
    ),
    ChatMessage(role="user", content="What is your name"),
]
resp = llm.stream_chat(
    messages, preamble_override="You are a pirate with a colorful personality"
)
python
for r in resp:
    print(r.delta, end="")

Configure Model

python
from llama_index.llms.cohere import Cohere

llm = Cohere(model="command", api_key=api_key)
python
resp = llm.complete("Paul Graham is ")
python
print(resp)

Async

python
from llama_index.llms.cohere import Cohere

llm = Cohere(model="command", api_key=api_key)
python
resp = await llm.acomplete("Paul Graham is ")
python
print(resp)
python
resp = await llm.astream_complete("Paul Graham is ")
python
async for delta in resp:
    print(delta.delta, end="")

Set API Key at a per-instance level

If desired, you can have separate LLM instances use separate API keys.

python
from llama_index.llms.cohere import Cohere

llm_good = Cohere(api_key=api_key)
llm_bad = Cohere(model="command", api_key="BAD_KEY")

resp = llm_good.complete("Paul Graham is ")
print(resp)

resp = llm_bad.complete("Paul Graham is ")
print(resp)