ContextQMD
Back to Litellm

Only set keys for the LLMs you want to use

cookbook/liteLLM_Getting_Started.ipynb

1.84.0-dev.21.8 KB
Original Source

🚅 liteLLM Quick Start Demo

TLDR: Call 50+ LLM APIs using chatGPT Input/Output format

https://github.com/BerriAI/litellm

liteLLM is package to simplify calling OpenAI, Azure, Llama2, Cohere, Anthropic, Huggingface API Endpoints. LiteLLM manages

Installation and setting Params

python
!pip install litellm
python
from litellm import completion
import os

Set your API keys

  • liteLLM reads your .env, env variables or key manager for Auth

Set keys for the models you want to use below

python
# Only set keys for the LLMs you want to use
os.environ['OPENAI_API_KEY'] = "" #@param
os.environ["ANTHROPIC_API_KEY"] = "" #@param
os.environ["REPLICATE_API_KEY"] = "" #@param
os.environ["COHERE_API_KEY"] = "" #@param
os.environ["AZURE_API_BASE"] = "" #@param
os.environ["AZURE_API_VERSION"] = "" #@param
os.environ["AZURE_API_KEY"] = "" #@param

Call chatGPT

python
completion(model="gpt-3.5-turbo", messages=[{ "content": "what's the weather in SF","role": "user"}])

Call Claude-2

python
completion(model="claude-2", messages=[{ "content": "what's the weather in SF","role": "user"}])

Call llama2 on replicate

python
model = "replicate/llama-2-70b-chat:2c1608e18606fad2812020dc541930f2d0495ce32eee50074220b87300bc16e1"
completion(model=model, messages=[{ "content": "what's the weather in SF","role": "user"}])

Call Command-Nightly

python
completion(model="command-nightly", messages=[{ "content": "what's the weather in SF","role": "user"}])

Call Azure OpenAI

For azure openai calls ensure to add the azure/ prefix to model. If your deployment-id is chatgpt-test set model = azure/chatgpt-test

python
completion(model="azure/chatgpt-v-2", messages=[{ "content": "what's the weather in SF","role": "user"}])