Troubleshooting webhooks

{{< details >}}

• Tier: Free, Premium, Ultimate
• Offering: GitLab.com, GitLab Self-Managed, GitLab Dedicated

{{< /details >}}

Troubleshoot and resolve common issues with GitLab webhooks.

Debug webhooks

Debug GitLab webhooks and capture payloads using these methods:

• Public webhook inspection tools
• Webhook request and response details
• GitLab Development Kit (GDK)
• Private webhook receiver

For information about webhook events and JSON payloads, see webhook events.

Use public webhook inspection tools

Use public tools to inspect and test webhook payloads. These tools provide catch-all endpoints for HTTP requests and respond with a 200 OK status code.

[!warning] Exercise caution when using public tools, as you might send sensitive data to external services. Use test tokens and rotate any secrets inadvertently sent to third parties. For enhanced privacy, create a private webhook receiver.

Public webhook inspection tools include:

• Beeceptor: Create a temporary HTTPS endpoint and inspect incoming payloads.

• Webhook.site: Review incoming payloads.
• Webhook Tester: Inspect and debug incoming payloads.

Use the GitLab Development Kit (GDK)

For a safer development environment, use the GitLab Development Kit (GDK) to work with GitLab webhooks locally. Use the GDK to send webhooks from your local GitLab instance to a webhook receiver on your machine.

To use this approach, install and configure the GDK.

Create a private webhook receiver

Create your own private webhook receiver if you cannot send webhook payloads to a public receiver.

Prerequisites:

• Ruby is installed on your system.

To create a private webhook receiver:

1. Save this script as print_http_body.rb:

   rubyCopy

   require 'webrick'
   
   server = WEBrick::HTTPServer.new(:Port => ARGV.first)
   server.mount_proc '/' do |req, res|
     puts req.body
   end
   
   trap 'INT' do
     server.shutdown
   end
   server.start
   

2. Choose an unused port (for example, 8000) and start the script:

   shellCopy

   ruby print_http_body.rb 8000
   

3. In GitLab, configure the webhook with your receiver's URL (for example, http://receiver.example.com:8000/).

4. Select Test. You should see output similar to:

   plaintextCopy

   {"before":"077a85dd266e6f3573ef7e9ef8ce3343ad659c4e","after":"95cd4a99e93bc4bbabacfa2cd10e6725b1403c60",<SNIP>}
   example.com - - [14/May/2014:07:45:26 EDT] "POST / HTTP/1.1" 200 0
   - -> /
   

[!note] To add this receiver, you might need to allow requests to the local network.

Resolve SSL certificate verification errors

When SSL verification is enabled, GitLab might fail to verify the SSL certificate of the webhook endpoint with the following error:

unable to get local issuer certificate

This error typically occurs when the root certificate is not issued by a trusted certificate authority as determined by CAcert.org.

To resolve this issue:

1. Use SSL Checker to identify specific errors.
2. Check for missing intermediate certificates, a common cause of verification failure.

Webhook not triggered

{{< history >}}

• Webhooks not triggered in Silent Mode introduced in GitLab 16.3.

{{< /history >}}

If a webhook is not triggered, verify that:

• The webhook is not disabled automatically.
• The GitLab instance is not in Silent Mode.
• The Push event activities limit and Push event hooks limit settings in the Admin area are set to a value greater than 0.

Error: Webhook rate limit exceeded

A webhook might fail because of rate limiting. GitLab.com limits the total number of webhook calls each minute for each top-level namespace. For more information, see rate limits.

To confirm if rate limiting is the issue:

1. Check your GitLab logs for the message Webhook rate limit exceeded.
2. Reduce the number of events that trigger webhooks or contact GitLab Support to discuss your rate limit requirements.