Incoming webhooks reference

This guide provides a detailed reference for developing and configuring Zulip incoming webhook integrations. For step-by-step guidance, read the incoming webhooks walkthrough, before using this reference guide.

Custom HTTP headers

Some third-party outgoing webhook APIs, such as GitHub's, don't encode all of the information about an event in the HTTP request body. Instead, they put key details like the event type that generates the particular payload in a separate HTTP header. Generally, this is clear in the third-party's API documentation that you will be referencing when creating fixtures.

Extracting event-type HTTP headers from payloads

To get the HTTP header value from the payload, in your view.py, you can use the get_event_header function in zerver/lib/webhooks/common.py, like so:

event = get_event_header(request, header, integration_name)

request is the HttpRequest object passed to your main webhook function. header is the name of the custom header you'd like to extract, such as X-Event-Key. And integration_name is the name of the third-party service in question, such as GitHub.

Because such headers are how some integrations indicate the event types of their outgoing webhook payloads, the absence of such a header usually indicates a configuration issue, where one either entered the URL for a different integration, or happens to be running an older version of the integration that doesn't set that header.

If the requisite header is missing, this function sends a direct message to the owner of the webhook bot, notifying them of the missing header.

Recording event-type HTTP headers in fixtures

In order to test Zulip's handling of this data, you will need to record which HTTP headers are used with each fixture you capture. Since this is integration-dependent, Zulip supports a simple format.

Encode the value of the HTTP header in the first part of the fixture's filename, for example:

pull_request__opened.json

pull_request is the value of the header X-Github-Event, and opened is the subtype of the event type. They are separated by a double underscore, to allow using single underscores within each segment.

Extracting event-type HTTP headers from fixtures

To get the HTTP header value from the fixture's filename in your tests.py, you can use the default_fixture_to_headers function in zerver/webhooks/common.py, like so:

fixture_to_headers = default_fixture_to_headers("HTTP_X_GITHUB_EVENT")

HTTP_X_GITHUB_EVENT is the name of the custom header you'd like to extract.

The default implementation default_fixture_to_headers uses the first part of the fixture's filename as the header value, separated by a double underscore (__). If you need to use a different method for encoding the header value(s), you can directly pass your function with the custom parsing logic to the fixture_to_headers function defined in zerver/tests/test_webhooks_common.py, instead of using default_fixture_to_headers.

Custom URL query parameters

Registering webhooks requiring custom configuration

In cases where an incoming webhook integration supports optional URL parameters, one can use the url_options feature. It's a field in the IncomingWebhookIntegration class that is used when generating an integration URL (for a bot) in the web and desktop apps, which encodes the user input for each parameter in the integration URL.

These URL options can be declared as follows:

    IncomingWebhookIntegration(
        'helloworld',
        ...
        url_options=[
          WebhookUrlOption(
            name='ignore_private_repositories',
            label='Exclude notifications from private repositories',
            input_type='checkbox',
          ),
        ],
    )

url_options is a list describing the parameters the web app UI should offer when generating the integration URL:

• name: The parameter name that is used to encode the user input in the integration's webhook URL.
• label: A short descriptive label for this URL parameter in the web app UI.
• input_type: The type of input field this option maps to in the UI. The web app UI currently supports the following input types:
  • checkbox: A checkbox input for presence-only values (true or absent), disabled by default.
  • checkbox_enabled: A checkbox input for boolean parameters, enabled by default.
  • text: A text input for string values.

To add support for other input types, you can update web/src/integration_url_modal.ts.

In rare cases, it may be necessary for an incoming webhook to require additional user configuration beyond what is specified in the POST URL. A typical use case for this would be APIs that require clients to do a callback to get details beyond an opaque object ID that one would want to include in a Zulip notification message. The config_options field in the IncomingWebhookIntegration class is reserved for this use case.

WebhookUrlOption presets

The build_preset_config method creates WebhookUrlOption objects with pre-configured fields. These preset URL options primarily serve two purposes:

• To construct common WebhookUrlOption objects that are used in various incoming webhook integrations.

• To construct WebhookUrlOption objects with special UI in the web app for generating incoming webhook URLs.

For other purposes, you can use the WebhookUrlOption class directly.

Using a preset URL option with the build_preset_config method:

# zerver/lib/integrations.py
from zerver.lib.webhooks.common import PresetUrlOption, WebhookUrlOption
  # -- snip --
    IncomingWebhookIntegration(
        "github",
        # -- snip --
        url_options=[
            WebhookUrlOption.build_preset_config(PresetUrlOption.BRANCHES),
        ],
    ),

The currently configured preset URL options are:

• BRANCHES: This preset is intended to be used for version control integrations, and adds UI for the user to configure which branches of a project's repository will trigger Zulip notification messages. When the user specifies which branches to receive notifications from, the branches parameter will be added to the generated integration URL. For example, if the user input main and dev for the branches of their repository, then &branches=main%2Cdev would be appended to the generated URL.

• IGNORE_PRIVATE_REPOSITORIES: This preset is intended to be used for version control integrations, and adds UI for the user to exclude private repositories from triggering Zulip notification messages. When the user selects this option, the ignore_private_repositories boolean parameter will be added to the generated integration URL.

• CHANNEL_MAPPING: This preset is intended to be used for chat-app integrations (like Slack), and adds a special option, Matching Zulip channel, to the web app UI for where to send Zulip notification messages. This special option maps the notification messages to Zulip channels that match the messages' original channel name in the third-party service. When selected, this requires setting a single topic for notification messages, and adds &mapping=channels to the generated integration URL.

Writing tests for custom URL query parameters

Custom arguments passed in URL query parameters work as expected in the webhook code, but require special handling in tests.

For example, here is the definition of a webhook function that gets both stream and topic from the query parameters:

@typed_endpoint
def api_querytest_webhook(request: HttpRequest, user_profile: UserProfile,
                          payload: Annotated[str, ApiParamConfig(argument_type_is_body=True)],
                          stream: str = "test",
                          topic: str= "Default Alert":

In actual use, you might configure the third-party service to call your Zulip incoming webhook integration with a URL like this:

http://myhost/api/v1/external/querytest?api_key=abcdefgh&stream=alerts&topic=queries

It provides values for stream and topic, and the integration can get those using @typed_endpoint without any special handling. How does this work in a test?

The new attribute TOPIC exists only in our class so far. In order to construct a URL with a query parameter for topic, you can pass the attribute TOPIC as a keyword argument to build_webhook_url, like so:

class QuerytestHookTests(WebhookTestCase):

    TOPIC = "Default topic"
    FIXTURE_DIR_NAME = 'querytest'

    def test_querytest_test_one(self) -> None:
        # construct the URL used for this test
        self.TOPIC = "Query test"
        self.url = self.build_webhook_url(topic=self.TOPIC)

        # define the expected message contents
        expected_topic = "Query test"
        expected_message = "This is a test of custom query parameters."

        self.check_webhook('test_one', expected_topic, expected_message,
                                          content_type="application/x-www-form-urlencoded")

You can also override get_body or get_payload if your test data needs to be constructed in an unusual way.

For more, see the definition for the base class, WebhookTestCase in zerver/lib/test_classes.py, or just grep for examples.

Negative tests

A negative test is one that should result in an error, such as incorrect data from the third-party's payload or headers. To correctly test these cases, you must explicitly code your test's execution (using other test helpers, as needed) rather than calling the usual check_webhook test helper function.

Here is an example from the WordPress integration:

def test_unknown_action_no_data(self) -> None:
    # Mimic check_webhook() to manually execute a negative test.
    # Otherwise its call to send_webhook_payload() would assert on the non-success
    # we are testing. The value of result is the error message the webhook should
    # return if no params are sent. The fixture for this test is an empty file.

    # subscribe to the target channel
    self.subscribe(self.test_user, self.channel_name)

    # post to the webhook url
    post_params = {'stream_name': self.channel_name,
                   'content_type': 'application/x-www-form-urlencoded'}
    result = self.client_post(self.url, 'unknown_action', **post_params)

    # check that we got the expected error message
    self.assert_json_error(result, "Unknown WordPress webhook action: WordPress action")

In a normal test, check_webhook would handle all the setup and then check that the incoming webhook's response matches the expected success result. If the webhook returns an error, the test fails. Instead, you can explicitly do the test setup it would have done, and check the error result yourself.

Here, subscribe is a test helper that uses test_user and channel_name (attributes from the base class) to register the user to receive messages in the given channel. If the channel doesn't exist, it creates it.

client_post, another helper function, performs the HTTP POST that calls the incoming webhook. As long as self.url is correct, you don't need to construct the webhook URL yourself. (In most cases, it is.)

assert_json_error then checks if the result matches the expected error. If you had used check_webhook, it would have called send_webhook_payload, which checks the result with assert_json_success.