doc/administration/gitlab_duo_self_hosted/logging.md
{{< details >}}
{{< /details >}}
{{< history >}}
ai_custom_model. Disabled by default.ai_custom_model removed in GitLab 17.8.{{< /history >}}
Monitor your self-hosted model performance and debug issues more effectively with detailed logging.
Prerequisites:
Turn on AI logs to collect data about AI-related activities and requests. This data is stored in your GitLab installation and is not shared with GitLab.
To turn on AI logs:
The logging setup is designed to protect sensitive information while maintaining transparency about system operations, and is made up of the following components:
llm.log file.Logging in the application.json, production_json.log, and production.log files, among others, capture requests to the GitLab instance:
Filtered Requests: We log the requests in these files but ensure that sensitive data (such as input parameters) is filtered. This means that while the request metadata is captured (for example, the request type, endpoint, and response status), the actual input data (for example, the query parameters, variables, and content) is not logged to prevent the exposure of sensitive information.
Example 1: In the case of a code suggestions completion request, the logs capture the request details while filtering sensitive information:
{
"method": "POST",
"path": "/api/graphql",
"controller": "GraphqlController",
"action": "execute",
"status": 500,
"params": [
{"key": "query", "value": "[FILTERED]"},
{"key": "variables", "value": "[FILTERED]"},
{"key": "operationName", "value": "chat"}
],
"exception": {
"class": "NoMethodError",
"message": "undefined method `id` for {:skip=>true}:Hash"
},
"time": "2024-08-28T14:13:50.328Z"
}
As shown, while the error information and general structure of the request are logged, the sensitive input parameters are marked as [FILTERED].
Example 2: In the case of a code suggestions completion request, the logs also capture the request details while filtering sensitive information:
{
"method": "POST",
"path": "/api/v4/code_suggestions/completions",
"status": 200,
"params": [
{"key": "prompt_version", "value": 1},
{"key": "current_file", "value": {"file_name": "/test.rb", "language_identifier": "ruby", "content_above_cursor": "[FILTERED]", "content_below_cursor": "[FILTERED]"}},
{"key": "telemetry", "value": []}
],
"time": "2024-10-15T06:51:09.004Z"
}
As shown, while the general structure of the request is logged, the sensitive input parameters such as content_above_cursor and content_below_cursor are marked as [FILTERED].
To control a subset of logs, turn LLM logs on and off through the GitLab Duo settings page. Turning LLM logs off disables logging for specific operations.
llm.log fileWhen AI Logs are enabled, code generation and Chat events that occur through your GitLab Self-Managed instance are captured in the llm.log file. The log file does not capture anything when it is not enabled. Code completion logs are captured directly in the AI Gateway. These logs are not transmitted to GitLab, and are only visible on your GitLab Self-Managed infrastructure.
llm.log.To specify the location of logs generated by AI Gateway and the GitLab Duo Agent Platform, run:
docker run -e AIGW_GITLAB_URL=<your_gitlab_instance> \
-e AIGW_GITLAB_API_URL=https://<your_gitlab_domain>/api/v4/ \
-e DUO_WORKFLOW_SELF_SIGNED_JWT__SIGNING_KEY="your-signing-key" \
-e AIGW_LOGGING__TO_FILE="aigateway.log" \
-e DUO_WORKFLOW_LOGGING__TO_FILE="duo_agent_platform.log" \
-v <your_aigateway_file_path>:aigateway.log \
-v <your_duo_agent_platform_file_path>:duo_agent_platform.log \
<image>
By default, the logging level is set to INFO. To change the logging level to DEBUG, run:
docker run -e AIGW_GITLAB_URL=<your_gitlab_instance> \
-e AIGW_GITLAB_API_URL=https://<your_gitlab_domain>/api/v4/ \
-e DUO_WORKFLOW_SELF_SIGNED_JWT__SIGNING_KEY="your-signing-key" \
-e AIGW_LOGGING__TO_FILE="aigateway.log" \
-e DUO_WORKFLOW_LOGGING__TO_FILE="duo_agent_platform.log" \
-e AIGW_LOGGING__LEVEL="DEBUG" \
-e DUO_WORKFLOW_LOGGING__LEVEL="DEBUG" \
-v <your_aigateway_file_path>:aigateway.log \
-v <your_duo_agent_platform_file_path>:duo_agent_platform.log \
<image>
Additionally, to log all of the debug statements from litellm, add the following environment variables:
-e AIGW_LOGGING__ENABLE_LITELLM_LOGGING=true
If you do not specify a filename, logs are streamed to the output and can also be managed using Docker logs. For more information, see the Docker Logs documentation.
Additionally, the outputs of the AI Gateway execution can help with debugging issues. To access them:
When using Docker:
docker logs <container-id>
When using Kubernetes:
kubectl logs <container-name>
To ingest these logs into the logging solution, see your logging provider documentation.
When a POST request is made (for example, to the /chat/completions endpoint), the server logs the request:
The JSON payload typically includes the following fields:
messages: An array of message objects.
content: A string representing the user's input or query.role: Indicates the role of the message sender (for example, user).model: A string specifying the model to be used (for example, mistral).max_tokens: An integer specifying the maximum number of tokens to generate in the response.n: An integer indicating the number of completions to generate.stop: An array of strings denoting stop sequences for the generated text.stream: A boolean indicating whether the response should be streamed.temperature: A float controlling the randomness of the output.{
"messages": [
{
"content": "<s>[SUFFIX]None[PREFIX]# # build a hello world ruby method\n def say_goodbye\n puts \"Goodbye, World!\"\n end\n\ndef main\n say_hello\n say_goodbye\nend\n\nmain",
"role": "user"
}
],
"model": "mistral",
"max_tokens": 128,
"n": 1,
"stop": ["[INST]", "[/INST]", "[PREFIX]", "[MIDDLE]", "[SUFFIX]"],
"stream": false,
"temperature": 0.0
}
The request headers provide additional context about the client making the request. Key headers might include:
Authorization: Contains the Bearer token for API access.Content-Type: Indicates the media type of the resource (for example, JSON).User-Agent: Information about the client software making the request.X-Stainless- headers: Various headers providing additional metadata about the client environment.{
"host": "0.0.0.0:4000",
"accept-encoding": "gzip, deflate",
"connection": "keep-alive",
"accept": "application/json",
"content-type": "application/json",
"user-agent": "AsyncOpenAI/Python 1.51.0",
"authorization": "Bearer <TOKEN>",
"content-length": "364"
}
The metadata includes various fields that describe the context of the request:
requester_metadata: Additional metadata about the requester.user_api_key: The API key used for the request (anonymized).api_version: The version of the API being used.request_timeout: The timeout duration for the request.call_id: A unique identifier for the call.{
"user_api_key": "<ANONYMIZED_KEY>",
"api_version": "1.48.18",
"request_timeout": 600,
"call_id": "e1aaa316-221c-498c-96ce-5bc1e7cb63af"
}
The server responds with a structured model response. For example:
Response: ModelResponse(
id='chatcmpl-5d16ad41-c130-4e33-a71e-1c392741bcb9',
choices=[
Choices(
finish_reason='stop',
index=0,
message=Message(
content=' Here is the corrected Ruby code for your function:\n\n```ruby\ndef say_hello\n puts "Hello, World!"\nend\n\ndef say_goodbye\n puts "Goodbye, World!"\nend\n\ndef main\n say_hello\n say_goodbye\nend\n\nmain\n```\n\nIn your original code, the method names were misspelled as `say_hell` and `say_gobdye`. I corrected them to `say_hello` and `say_goodbye`. Also, there was no need for the prefix',
role='assistant',
tool_calls=None,
function_call=None
)
)
],
created=1728983827,
model='mistral',
object='chat.completion',
system_fingerprint=None,
usage=Usage(
completion_tokens=128,
prompt_tokens=69,
total_tokens=197,
completion_tokens_details=None,
prompt_tokens_details=None
)
)
GitLab does not manage logs generated by your inference service provider. See the documentation of your inference service provider on how to use their logs.
GitLab provides logging functionality for AI-related activities through the use of llm.log, which captures inputs, outputs, and other relevant information. However, the logging behavior differs depending on whether the GitLab instance and AI Gateway are self-hosted or cloud-connected.
By default, the log does not contain LLM prompt input and response output to support data retention policies of AI feature data.
In this configuration, both GitLab and the AI Gateway are hosted by the customer.
Logging Behavior: Full logging is enabled, and all prompts, inputs, and outputs are logged to llm.log on the instance.
When AI logs are enabled, extra debugging information is logged, including:
Privacy: Because both GitLab and the AI Gateway are self-hosted:
[!note] When an AI feature uses a GitLab AI third-party vendor model, no detailed logs are generated in the GitLab-hosted AI Gateway, even when AI logs are enabled. This prevents unintended leaks of sensitive information.
In this scenario, the customer hosts GitLab but relies on the GitLab-managed AI Gateway for AI processing.
The AI logs control whether additional debugging information, including prompts and inputs, is logged. This configuration is essential for monitoring and debugging AI-related activities.
llm.log on both the self-hosted instance and the AI Gateway, capturing inputs and outputs for AI models.llm.log on your GitLab Self-Managed instance.For information about how GitLab handles AI prompt and response data when using a cloud-connected AI Gateway, see GitLab Duo data usage.
The property correlation_id is assigned to every request and is carried across different components that respond to a
request. For more information, see the documentation on finding logs with a correlation ID.
The Correlation ID can be found in your AI Gateway and GitLab logs. However, it is not present in your model provider logs.