Solid Queue as the Default Job Backend — Single-Container, Postgres-Only Deployment

Status: Done (starter + CLI + docs implemented — spree-starter#1368, spree#14323) Target: Spree 5.6 (spree-starter + tooling change only — no core gem dependency; can ship in a 5.6.x starter update if RC feedback warrants) Depends on: Single-node Docker deployment (spree-starter#1368 + spree#14323, shipped); 6.0-stock-reservations.md (consumes the recurring-jobs mechanism this plan standardizes) Author: Damian Legawiec (design discussion with Claude, 2026-07-18) Last updated: 2026-07-18 (implementation landed same day; verified on the built image)

Summary

Replace Sidekiq + Redis with the Rails 8 Solid stack in spree-starter, making the default Spree deployment one process, one container, one datastore: Puma runs the Solid Queue supervisor in-process (plugin :solid_queue), jobs live in Postgres (which Spree already requires), Solid Cache replaces the Redis cache store, and Solid Cable replaces the Redis Action Cable adapter. Redis disappears from the default stack entirely — dev compose, production compose, and the Render Blueprint each lose a service. This is the "ONCE"/Rails 8 deployment model, and it beats the simplest stories in the category: Medusa's workerMode: "shared" still wants Redis in production; Vendure's DB-backed JobQueue is the same bet without the in-process default; Saleor has no single-process story at all. Scale-out is a config flag, not a code change.

Key Decisions (do not deviate without discussion)

• Solid Queue is the default adapter in spree-starter; Sidekiq remains fully supported. Core is adapter-agnostic (audited 2026-07-18 — see Design Details) and stays that way. Existing projects keep whatever their repo configures; nothing in the gems forces a migration.
• In-Puma supervisor is the default topology. puma.rb gets plugin :solid_queue if ENV.fetch("SOLID_QUEUE_IN_PUMA", "true") == "true". The escape hatch for scale is SOLID_QUEUE_IN_PUMA=false on web plus a worker service running bin/jobs from the same image — one env var and one service definition, no rebuild.
• Single database by default; the queue-DB split is a documented scaling step, not the default. Single-DB gives commerce the property that matters: perform_later inside a transaction commits atomically with the business data ("order created → confirmation email" cannot race). The Rails 8 two-DB convention (spree_production_queue, same Postgres server) is documented for high-churn installs alongside the worker split.
• Full no-Redis: Solid Cache for Rails.cache, Solid Cable for Action Cable (the legacy admin's import live-feed broadcasts still need a cable adapter until the engine is removed in 6.0). Meilisearch stays — it is a search engine, not incidental infra.
• Solid Cache lives in the primary database — no separate cache DB. Headless reframing: the Russian-doll fragment caching that justified Redis-sized caches died with the ERB storefront, and the legacy admin's partial caching dies in 6.0. What remains is ~20 low-level Rails.cache.fetch sites (taxon descendant ids, states helper, the preferences store) plus rate-limit counters — a workload where splitting databases is over-engineering. Redis/Valkey stays a documented one-line swap (config.cache_store = :redis_cache_store) for high-traffic installs; wherever Redis remains in docs, say "Redis or Valkey" (post-relicensing, platforms steer to Valkey). It's a package deal — no half-migration that keeps Sidekiq while adopting Solid Cache; the payoff is deleting the Redis requirement entirely.
• Mission Control – Jobs replaces the /sidekiq UI (mission_control-jobs, mounted at /jobs). Guarded by Mission Control's own HTTP Basic auth (MISSION_CONTROL_USER / MISSION_CONTROL_PASSWORD, seed-admin defaults in dev/test, Render Blueprint generates the password) — deliberately not the admin Devise session, which 6.0-platform-auth.md removes from the Rails app.
• Recurring jobs standardize on Solid Queue's config/recurring.yml — this is the mechanism 6.0-stock-reservations.md should name for ExpireJob (it currently hedges "sidekiq-cron / solid_queue").

Design Details

Core audit (2026-07-18) — why the gems need zero changes

• No Sidekiq-specific APIs anywhere in spree/core, spree/api, spree/admin, spree/emails: no Sidekiq:: constants, no batches, no unique-jobs middleware, no server middleware. The only mentions are comments.
• Scheduling is plain ActiveJob: five set(wait: 2.seconds..30.seconds) sites (product taxon auto-match, import row fan-out, theme duplication, payment webhook retry). Solid Queue's dispatcher handles scheduled jobs natively at this granularity.
• Spree::BaseJob's retry_on ActiveRecord::RecordNotFound exists to absorb the enqueue-before-commit race — which single-DB Solid Queue eliminates (the enqueue commits with the transaction; Solid Queue ≥1.2 exposes enqueue_after_transaction_commit to control this). The retry stays for adapter-agnosticism, but the default deployment no longer exercises it.
• No Redis usage in core: the events system is explicitly adapter-abstracted; no advisory locks, no Redlock, no direct Redis.new.
• Action Cable surface: Spree::Import + the legacy admin's import subscribers broadcast Turbo Streams (live import feed). The React Dashboard polls instead (by design — see 5.6-admin-spa-csv-import.md), so this coupling dies with the legacy admin in 6.0; until then Solid Cable carries it.
• No sidekiq-cron/schedule file exists today — there is no recurring-job migration, only a green field for recurring.yml.
• Extensions are clean too (confirmed by Damian, 2026-07-18): all extensions — including spree_stripe, spree_adyen, spree_paypal_checkout — use plain ActiveJob with no Sidekiq-specific APIs.
• Cache audit: 20 Rails.cache call sites across core+api, all small-value fetch memoization — no fragment caching in the API surface. The highest-frequency cache operation is rate limiting: the v3 API uses Rails 8's built-in rate_limit with store: Rails.cache on the base controller (per-key, every request) plus tighter limits on login/refresh/register/password-reset/newsletter/webhooks. Verified (2026-07-18): Solid Cache implements atomic increment, and Rails' rate_limit was deliberately built over the generic ActiveSupport::Cache contract (rails/rails#50781) — the combination is supported and correct across processes. Cost model: one Postgres UPDATE per API request; fine for the median install, and the swap knob exists for hot paths.

spree-starter changes

Deployment shapes

Default (create-spree-app, docker run, Render free tier):
┌─────────────────────────────┐      ┌──────────────┐
│  app container              │      │  Postgres    │
│  Puma                       │─────▶│  (one DB)    │
│   └─ plugin :solid_queue    │      └──────────────┘
│      ├─ dispatcher          │
│      ├─ worker(s)           │
│      └─ scheduler           │
└─────────────────────────────┘

Scale-out (flag flip, same image):
web:    SOLID_QUEUE_IN_PUMA=false → Puma only
worker: bundle exec bin/jobs      → Solid Queue supervisor only
db:     optionally spree_production_queue (second DB, same server)
        via config.solid_queue.connects_to + db/queue_migrate

CLI / create-spree-app changes

• spree dev runs docker compose up web worker today — with no worker service it must detect the compose shape (worker service present → include it; absent → web only). Same for primeBundleVolume's assumptions and spree logs worker (error message pointing at Mission Control / SOLID_QUEUE_IN_PUMA instead).
• Setup summary/README/docs drop the Sidekiq UI link; add /jobs (Mission Control).
• SIDEKIQ_DB_POOL env handling removed; pool guidance moves to the database.yml comment.
• Generated project docs (README/CLAUDE.md templates) describe the single-process model and the two scale-out steps.

Competitor context (for docs/marketing framing)

• Medusa: workerMode: "server" | "worker" | "shared" — shared in one process for dev/simple deploys, but production guidance splits and the event bus/queue want Redis. Spree's story: same simplicity, Postgres-only.
• Vendure: explicit bootstrap() vs bootstrapWorker() split; default JobQueue is DB-backed (same bet), BullMQ/Redis as the production-grade plugin.
• Saleor: Django + Celery + Redis broker, mandatory separate workers — no single-process story.

Migration Path

1. spree-starter PR: the table above, in one change. CI: boot + a job-exercising smoke (e.g. seed triggers AutoMatchTaxonsJob; assert it executes in-Puma).
2. CLI + create-spree-app PR: compose-shape detection in spree dev/logs, template/doc updates. Backward-compatible — existing Sidekiq-shaped projects keep working (worker service detected and included).
3. Docs: deployment guide gains the two scale-out steps (worker split, queue-DB split); environment-variables page updated.
4. Existing projects: no forced migration. A docs recipe covers switching (add gems, flip adapter, drain Sidekiq queues before cutover — Sidekiq jobs don't transfer).
5. 6.0: 6.0-stock-reservations.md names recurring.yml for ExpireJob; legacy-admin removal deletes the last Action Cable broadcasts (Solid Cable stays for storefront/custom use).

Constraints on Current Work

• Keep core adapter-agnostic: no Sidekiq-specific APIs (batches, unique-jobs, middleware) and no direct Redis usage in any Spree gem. New jobs inherit Spree::BaseJob and use plain ActiveJob scheduling (set(wait:)).
• New recurring-work designs should name config/recurring.yml as the mechanism, not sidekiq-cron/whenever.
• Don't add Redis-backed features to the starter (e.g. Rack::Attack with a Redis store) without routing them through Rails.cache.
• CLI work touching spree dev/compose should not hard-code the worker service name — detect it.