docs/self-hosting/scaling-for-production/option-2-rabbitmq-based-queue-architecture.md
Reacher includes an optional, opinionated queue-based architecture designed to scale efficiently and handle high email verification volumes. This architecture comprises 4 main components and is highly configurable to meet specific business needs.
<table><thead><tr><th width="183">Component</th><th width="295">Description</th><th>Docker image</th></tr></thead><tbody><tr><td><strong>HTTP server</strong></td><td>Accepts incoming email verification requests, and post them into the queue.</td><td><code>reacherhq/commercial-license-trial</code></td></tr><tr><td><a href="https://rabbitmq.com">RabbitMQ</a></td><td>Reacher uses a reliable, mature and open-source queue implementation.</td><td><code>rabbitmq:4.0-management</code></td></tr><tr><td><strong>Workers</strong></td><td>One or more consumers of the queue, which perform the actual email verification task.</td><td><code>reacherhq/commercial-license-trial</code></td></tr><tr><td><strong>Storage</strong></td><td>A place to store all results, currently only PostgresDB is supported.</td><td><code>postgres:14</code></td></tr></tbody></table>Note that Reacher provides the same Docker image reacherhq/commercial-license-trial which can act as both a Worker and a HTTP server.
With this architecture, it's possible to horizontally scale the number of workers.
To enable the worker-based architecture, configure the following parameters in your deployment. The parameters are given in their backend_config.toml format (e.g. worker.enable) as well as in the environment variable format (e.g. RCH__WORKER__ENABLE):
worker.enable or RCH__WORKER__ENABLE: Set to true to activate the worker role.worker.rabbitmq.url or RCH__WORKER__RABBITMQ__URL: URL of the RabbitMQ instance for task queuing.postgres.storage.postgres.db_url or RCH__STORAGE__POSTGRES__DB_URL: URL of a PostgreSQL database to store verification results.Since spawning workers on cloud providers doesn't guarantee a reputable IP assigned to the worker, we configure all workers to use a proxy.
proxy.{host,port} or RCH__PROXY__{HOST,PORT}: Set a proxy to route all SMTP requests through. You can optionally pass in username and password if required.{% hint style="info" %} The Dockerfile provided in the Commercial License Trial already has these parameters set up. {% endhint %}
You may also configure the following parameters:
worker.rabbitmq.concurrency: Set to 5. Each worker processes up to 5 emails concurrently.throttle.max_requests_per_minute: Set to 60. Limits request spikes to prevent SMTP server flags.throttle.max_requests_per_day: Set to 10,000. {% hint style="info" %} The Dockerfile provided in the Commercial License Trial already has these parameters set up. {% endhint %}
We do not recommend using Docker Compose for a high-volume production setup. However, for understanding or learning the architecture, this docker_compose.yaml file can be useful.
Contact [email protected]if you have more questions about this architecture, such as: