litellm/proxy/client/README.md
A Python client library for interacting with the LiteLLM proxy server. This client provides a clean, typed interface for managing models, keys, credentials, and making chat completions.
uv add litellm
from litellm.proxy.client import Client
# Initialize the client
client = Client(
base_url="http://localhost:4000", # Your LiteLLM proxy server URL
api_key="sk-api-key" # Optional: API key for authentication
)
# Make a chat completion request
response = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[
{"role": "user", "content": "Hello, how are you?"}
]
)
print(response.choices[0].message.content)
The client is organized into several resource clients for different functionality:
chat: Chat completionsmodels: Model managementmodel_groups: Model group managementkeys: API key managementcredentials: Credential managementusers: User managementMake chat completion requests to your LiteLLM proxy:
# Basic chat completion
response = client.chat.completions.create(
model="gpt-4",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What's the capital of France?"}
]
)
# Stream responses
for chunk in client.chat.completions.create(
model="gpt-4",
messages=[{"role": "user", "content": "Tell me a story"}],
stream=True
):
print(chunk.choices[0].delta.content or "", end="")
Manage available models on your proxy:
# List available models
models = client.models.list()
# Add a new model
client.models.add(
model_name="gpt-4",
litellm_params={
"api_key": "your-openai-key",
"api_base": "https://api.openai.com/v1"
}
)
# Delete a model
client.models.delete(model_name="gpt-4")
Manage virtual API keys:
# Generate a new API key
key = client.keys.generate(
models=["gpt-4", "gpt-3.5-turbo"],
aliases={"gpt4": "gpt-4"},
duration="24h",
key_alias="my-key",
team_id="team123"
)
# List all keys
keys = client.keys.list(
page=1,
size=10,
return_full_object=True
)
# Delete keys
client.keys.delete(
keys=["sk-key1", "sk-key2"],
key_aliases=["alias1", "alias2"]
)
Manage model credentials:
# Create new credentials
client.credentials.create(
credential_name="azure1",
credential_info={"api_type": "azure"},
credential_values={
"api_key": "your-azure-key",
"api_base": "https://example.azure.openai.com"
}
)
# List all credentials
credentials = client.credentials.list()
# Get a specific credential
credential = client.credentials.get(credential_name="azure1")
# Delete credentials
client.credentials.delete(credential_name="azure1")
Manage model groups for load balancing and fallbacks:
# Create a model group
client.model_groups.create(
name="gpt4-group",
models=[
{"model_name": "gpt-4", "litellm_params": {"api_key": "key1"}},
{"model_name": "gpt-4-backup", "litellm_params": {"api_key": "key2"}}
]
)
# List model groups
groups = client.model_groups.list()
# Delete a model group
client.model_groups.delete(name="gpt4-group")
Manage users on your proxy:
from litellm.proxy.client import UsersManagementClient
users = UsersManagementClient(base_url="http://localhost:4000", api_key="sk-test")
# List users
user_list = users.list_users()
# Get user info
user_info = users.get_user(user_id="u1")
# Create a new user
created = users.create_user({
"user_email": "[email protected]",
"user_role": "internal_user",
"user_alias": "Alice",
"teams": ["team1"],
"max_budget": 100.0
})
# Delete users
users.delete_user(["u1", "u2"])
The client provides access to a low-level HTTP client for making direct requests to the LiteLLM proxy server. This is useful when you need more control or when working with endpoints that don't yet have a high-level interface.
# Access the HTTP client
client = Client(
base_url="http://localhost:4000",
api_key="sk-api-key"
)
# Make a custom request
response = client.http.request(
method="POST",
uri="/health/test_connection",
json={
"litellm_params": {
"model": "gpt-4",
"api_key": "your-api-key",
"api_base": "https://api.openai.com/v1"
},
"mode": "chat"
}
)
# The response is automatically parsed from JSON
print(response)
api_key is provided)request method parametersmethod: HTTP method (GET, POST, PUT, DELETE, etc.)uri: URI path (will be appended to base_url)data: (optional) Data to send in the request bodyjson: (optional) JSON data to send in the request bodyheaders: (optional) Custom HTTP headersThe client provides clear error handling with custom exceptions:
from litellm.proxy.client.exceptions import UnauthorizedError
try:
response = client.chat.completions.create(
model="gpt-4",
messages=[{"role": "user", "content": "Hello"}]
)
except UnauthorizedError as e:
print("Authentication failed:", e)
except Exception as e:
print("Request failed:", e)
All methods support returning the raw request object for inspection or modification:
# Get the prepared request without sending it
request = client.models.list(return_request=True)
print(request.method) # GET
print(request.url) # http://localhost:8000/models
print(request.headers) # {'Content-Type': 'application/json', ...}
Methods that return lists support pagination:
# Get the first page of keys
page1 = client.keys.list(page=1, size=10)
# Get the second page
page2 = client.keys.list(page=2, size=10)
Many list methods support filtering:
# Filter keys by user and team
keys = client.keys.list(
user_id="user123",
team_id="team456",
include_team_keys=True
)
Contributions are welcome! Please check out our contributing guidelines for details.
This project is licensed under the MIT License - see the LICENSE file for details.
The LiteLLM CLI supports SSO authentication through a polling-based approach that works with any OAuth-compatible SSO provider.
sequenceDiagram
participant CLI as CLI
participant Browser as Browser
participant Proxy as LiteLLM Proxy
participant SSO as SSO Provider
CLI->>Proxy: POST /sso/cli/start
Proxy->>CLI: Return login_id, poll_secret, user_code
CLI->>Browser: Open /sso/key/generate?source=litellm-cli&key=login_id
Browser->>Proxy: GET /sso/key/generate?source=litellm-cli&key=login_id
Proxy->>Proxy: Set cli_state = litellm-session-token:login_id
Proxy->>SSO: Redirect with state=litellm-session-token:login_id
SSO->>Browser: Show login page
Browser->>SSO: User authenticates
SSO->>Proxy: Redirect to /sso/callback?state=litellm-session-token:login_id
Proxy->>Proxy: Check if state starts with "litellm-session-token:"
Proxy->>Browser: Prompt for user_code
Browser->>Proxy: POST /sso/cli/complete/login_id
CLI->>Proxy: Poll /sso/cli/poll/login_id with poll_secret header
Proxy->>CLI: Return {"status": "ready", "key": "jwt"}
CLI->>CLI: Save the secret to the OS keychain (metadata to ~/.litellm/token.json)
The CLI provides these authentication commands:
lite login - Start SSO authentication flowlite login --pkce - Sign in through the system browser with OAuth authorization code + PKCE; the key renews itself with a refresh tokenlite logout - Clear stored authentication token (and revoke a --pkce refresh token on the proxy)lite whoami - Show current authentication statuslite auth print-token - Print the cached token (used as Claude Code's apiKeyHelper); renews a --pkce key first and fails once a classic token has expired/sso/cli/start/sso/key/generate with CLI source and login ID parameterslitellm-session-token:{login_id}) as OAuth state parameter and redirects to SSO provider/sso/cli/poll/{login_id} with the polling secret header until the JWT is ready. When CLI_SSO_CLAIM_MAP is configured on the proxy, the poll response may include attribution_metadata (allowlisted scalar OIDC claims for client attribution).~/.litellm/token.jsonThe key itself, together with the refresh token that renews a --pkce credential, goes into the OS keychain (macOS Keychain, Windows Credential Manager, or the Linux Secret Service) under service litellm-cli, account credential. Only the non-secret session metadata is written to ~/.litellm/token.json, in a 0700 directory with 0600 file permissions:
{
"base_url": "https://your-proxy.com",
"user_id": "cli-user",
"user_email": "[email protected]",
"user_role": "cli",
"auth_header_name": "Authorization",
"timestamp": 1234567890
}
Keychain storage needs the keyring package, which ships with pip install 'litellm[cli]'. Headless boxes and CI runners usually have no keychain either. In all of those cases the key and the refresh token stay in the same 0600 file alongside the metadata, exactly as they did before, and lite login names which one applies: the package is missing, the machine has no keychain, or you set LITELLM_CLI_DISABLE_KEYRING=1 to force the file even where a keychain exists. A token.json written by an older lite keeps working and is moved into the keychain, and scrubbed from the file, the first time a keychain-capable lite reads it. That includes a refresh token left behind by the release that moved only the key.
lite logout clears both stores. If the keychain is locked at that moment it says so, and re-running it once the keychain is unlocked finishes the job.
The stored credential is a short-lived, per-session agent token, not a managed virtual key. It is scoped to the user and team you logged in as and inherits their models and budgets; spend is tracked against the shared team and user budgets rather than a separate per-session cap, so multiple logins or several concurrent agents all draw down the same allowance. It is short-lived by design (default 24h, configurable via LITELLM_CLI_JWT_EXPIRATION_HOURS); re-run lite login to refresh it and pick up your latest team and user settings. lite auth print-token (usable as Claude Code's apiKeyHelper) prints it while fresh and fails once it expires -- there is no silent renewal. It is accepted on a default deployment without EXPERIMENTAL_UI_LOGIN, does not appear in the Keys UI, and cannot be rotated or revoked mid-session. A credential from lite login --pkce is the exception: it carries a refresh token, so the CLI renews the key shortly before it expires and lite logout revokes the refresh token on the proxy (see Browser sign-in with PKCE). Only the holder can end a --pkce session early, with lite logout; an admin has no button for it, but every renewal re-reads the user on the proxy, so deactivating the user or removing them from the team makes the next renewal fail and the key runs out within LITELLM_CLI_JWT_EXPIRATION_HOURS. On a proxy with more than one worker or replica, configure Redis (litellm_settings.cache with Redis cache_params, or general_settings.coordination_redis) so a refresh token stays single-use and lite logout holds on every worker; without Redis each worker keeps its own record. For a long-lived, rotatable, Keys-UI-visible credential, create a dedicated virtual key in the dashboard and pass it via --api-key or LITELLM_PROXY_API_KEY.
Once authenticated, the CLI will automatically use the stored token for all requests. You no longer need to specify --api-key for subsequent commands.
# Login
lite login
# Use CLI without specifying API key
lite models list
# Check authentication status
lite whoami
# Logout
lite logout