Connecting DocsGPT to Cloud LLM Providers

DocsGPT is designed to seamlessly integrate with a variety of Cloud Large Language Model (LLM) providers, giving you access to state-of-the-art AI models for document question answering.

Configuration via .env file

The primary method for configuring your LLM provider in DocsGPT is through the .env file. For a comprehensive understanding of all available settings, please refer to the detailed DocsGPT Settings Guide.

To connect to a cloud LLM provider, you will typically need to configure the following basic settings in your .env file:

• LLM_PROVIDER: This setting is essential and identifies the specific cloud provider you wish to use (e.g., openai, google, anthropic).
• LLM_NAME: Specifies the exact model you want to utilize from your chosen provider (e.g., gpt-5.1, gemini-3.5-flash, claude-3-5-sonnet-20241022). Refer to your provider's documentation for a list of available models.
• API_KEY: Almost all cloud LLM providers require an API key for authentication. Obtain your API key from your chosen provider's platform and securely store it in your .env file.

Explicitly Supported Cloud Providers

DocsGPT offers direct, streamlined support for the following cloud LLM providers, making configuration straightforward. The table below outlines the LLM_PROVIDER and example LLM_NAME values to use for each provider in your .env file.

DocsGPT also ships a model catalog (application/core/models/*.yaml) that the in-app model picker reads, so common models from these providers — including DeepSeek — appear ready to select once the matching API key is set.

Connecting to OpenAI-Compatible Cloud APIs

DocsGPT's flexible architecture allows you to connect to any cloud provider that offers an API compatible with the OpenAI API standard. This opens up a vast ecosystem of LLM services.

To connect to an OpenAI-compatible cloud provider, you will still use LLM_PROVIDER=openai in your .env file. However, you will also need to specify the API endpoint of your chosen provider using the OPENAI_BASE_URL setting. You will also likely need to provide an API_KEY and LLM_NAME as required by that provider.

Example for DeepSeek (OpenAI-Compatible API):

To connect to DeepSeek, which offers an OpenAI-compatible API, your .env file could be configured as follows:

LLM_PROVIDER=openai
API_KEY=YOUR_API_KEY # Your DeepSeek API key
LLM_NAME=deepseek-chat # Or your desired DeepSeek model name
OPENAI_BASE_URL=https://api.deepseek.com/v1 # DeepSeek's OpenAI API URL

Remember to consult the documentation of your chosen OpenAI-compatible cloud provider for their specific API endpoint, required model names, and authentication methods.

Dedicated openai_compatible provider (bring-your-own-model)

Beyond the global OPENAI_BASE_URL, DocsGPT has a first-class openai_compatible provider. It lets a model carry its own base_url and api_key, which is how per-user "bring your own model" (BYOM) endpoints work — each model can point at a different OpenAI-compatible server without changing instance-wide settings. Outbound requests use an SSRF-pinned HTTP client for safety.

This is the mechanism behind catalog entries like DeepSeek, which declare their own base_url and API key environment variable rather than relying on OPENAI_BASE_URL.

OpenAI Responses API and reasoning

For OpenAI models that support it, DocsGPT can call the newer Responses API (/v1/responses) instead of Chat Completions. This is selected per model in the catalog via an api_flavor: responses capability and enables features like server-side reasoning. Related settings:

• reasoning_effort — per-model reasoning effort hint (for example medium) declared in the model catalog.
• OPENAI_RESPONSES_STORE (default false) — when true, lets OpenAI persist Responses API state server-side.

See App Configuration for the full settings reference.

Adding Support for Other Cloud Providers

If you wish to connect to a cloud provider that is not explicitly listed above or doesn't offer OpenAI API compatibility, you can extend DocsGPT to support it. Within the DocsGPT repository, navigate to the application/llm directory. Here, you will find Python files defining the existing LLM integrations. You can use these files as examples to create a new module for your desired cloud provider. After creating your new LLM module, you will need to register it within the llm_creator.py file. This process involves some coding, but it allows for virtually unlimited extensibility to connect to any cloud-based LLM service with an accessible API.