Sidekiq Enterprise Changelog

Sidekiq Changes | Sidekiq Pro Changes | Sidekiq Enterprise Changes

Please see sidekiq.org for more details and how to buy.

HEAD

• Vendor code to remove gserver gem dependency
• Make the einhorn dependency optional since it is only required for rolling restarts

8.1.2

• Remove ActiveSupport dependency in 8.1.1 [#6965]

8.1.1

• Support for kiq

8.1.0

• Better periodic UI styling
• Integrate herb, connection_pool 3.0

8.0.3

• Fix Herb lint warnings in ERB templates
• Historical metrics subsystem now immediately shuts down upon TERM

8.0.2

• Rewrite swarm memory tracking to use the get_process_mem gem which should allow operation on Alpine Linux, Windows, etc. [#6752]
• Handle fractional values for SIDEKIQ_COUNT, e.g. "0.25", when running on low-end VMs allocated only a partial core. [#6244]

8.0.1

• The Web UI can now pause, unpause and manually enqueue periodic jobs. [#6675]

8.0.0

• Please note the upgrade docs if you use Sidekiq Batches. You need to run Sidekiq Pro 7.3.x long enough to ensure all persistent batch data in Redis is compatible with Batch 8.0.
• Move from SHA1 to SHA256 internally. Unique locks are now SHA256 but only take 10% more space due to a more efficient encoding (base64 vs hex).
• Sidekiq::Web internal refactoring [#6532]
• Limiter factory methods now raise ArgumentError if you pass them a block [#6512]

7-x

• Backport leaky limiter fix for #6641

7.3.4

• Tune inspect to avoid huge output [#6553]
• Use backported APIs for Web extension

7.3.3

• Call Process.warmup before fork in sidekiqswarm, use RUBY_DISABLE_WARMUP=1 to disable [#6279]
• Disable health checks if running sidekiqswarm, we can't use one port to monitor many children

7.3.2

• Activate unique server middleware in client-mode so uniqueness works for perform_inline [#6460]
• Add Sidekiq::Enterprise.gem_version API

7.3.1

• Fix healthcheck when defined in config YML [#6352]

7.3.0

• Allow rate limiting to use Redis Cluster [#6288]
• Remove base64 gem dependency

7.2.4

• Revert #6288 as this requires Sidekiq 7.3 [#6324]

7.2.3

• Adjust the unlimited rate limiter to work as a points limiter too [#6301]
• Limiters can now use a clustered Redis client [#6288]
• Limiters now have attr_readers for all static config elements [#6259]
• Handle fractional values for SIDEKIQ_COUNT when containers have fractional CPU allocations, e.g. SIDEKIQ_COUNT=2.5 will create 2 Sidekiq processes. [#6244]

7.2.2

• Concurrent rate limiter raising ReadTimeoutError? Work around redis/redis#11732 [#6188]

7.2.1

• Add within_limit(used: 1) option to window and bucket rate limiters. You can adjust the number of points used by a call performing batch operations [#6146]
• Use HWIA when scheduling periodic ActiveJobs, for compatibility [#6099]

7.2.0

• Kubernetes health check can be enabled through Sidekiq's config YML. The full binding address can also be configured, not just port. Examples:

---
health_check: 127.0.0.1:8111 # static
health_check: <%= ENV["SIDEKIQ_HEALTH_BINDING"] %> # dynamic!

config.health_check("127.0.0.1:8111")

7.1.2

• Add support for a Kubernetes liveness / health check port. Start it with config.health_check(port = 7433) [#5923]
• Add missing points view [#6016]
• Add Polish translations

7.1.1

• Please note that license credentials are required when running in production. We recommend configuring credentials with Bundler.
• Fix hash mutation race condition in rate limiter autoloading [#5908]

7.1.0

• NEW Points-based rate limiter popular with GraphQL endpoints at Shopify, GitHub, et al. Thanks to Thad Sauter of NexHealth for contributing the initial skeleton. [#5757]
• NEW Test helper to verify periodic job registration block [#5832]

require "sidekiq-ent/periodic/testing"
CRON_BLOCK = ->(mgr) { mgr.register("0 * * * * *", "SomeJob") }
ct = Sidekiq::Periodic::ConfigTester.new
ct.verify(&CRON_BLOCK) # => raises ArgumentError, invalid crontab syntax

• Periodic jobs may now be ActiveJobs [#5902]
• Refactor rate limiter codebase to use autoload
• Refactor concurrent and bucket rate limiter data model to be cluster-friendly [#5800]

7.0.8

• Fix mutable job arguments when rescheduling an OverLimit [#5859]

7.0.7

• Tweak concurrent rate limiter to minimize spurious ReadTimeoutErrors [#5838]

7.0.6

• Fix redis-client API usage which could result in stuck Redis connections [#5823]

7.0.5

• Revert unique impl which required Redis 7.0 [#5793]
• Fix spurious "Uh oh" messages with sidekiqswarm [#5801]

7.0.4

• Remove usage of replicate_commands Redis directive, default in 5.0, gone in 7.0
• Fix issue with rate limiter connection pool [#5752]
• Unique middleware now prints the JID holding the lock if there is a duplicate [#5736] NOTE: Unique locks set with older versions (<7.0.4) will not work with newer versions.

7.0.3

• Allow user to define the context used to calculate unique locks, see the Unique Jobs wiki page [#5544]
• Smarter connection pool sizing for rate limiters [#5685]

7.0.2

• Fix crash in graceful restarts [#5667]

7.0.1

• Fix spurious ReadTimeoutError in concurrent rate limiter [#5611]
• Fix eager connection to Redis [#5606]

7.0.0

• Componentize and capsulize Enterprise functionality for Sidekiq 7
• Remove bucket history graph from Web UI
• Rename "Cron" tab to "Periodic" [#5590]
• Add DE locale

2.5.3

• Adjust rate limiters to lazy initialize, avoiding connection issues when forking preloaded app code [#5535]

2.5.2

• Remove Redis 4.8.0 deprecation warnings

2.5.1

• Fix crash with empty periodic data [#5374]

2.5.0

• Per the 2.0 upgrade notes, Sidekiq Enterprise will stop if you do not have a licensed username configured on startup. It will extract it from the Bundler environment if possible or look for SIDEKIQ_ENT_USERNAME. SIDEKIQ_ENT_USERNAME=abcd1234 bundle exec sidekiq
• Internal refactoring for Sidekiq 6.5.
• Requires Sidekiq 6.5, Pro 5.5.

2.3.1

• Fix multi/pipe deprecation in redis-rb 4.6
• Leader now elects more often, to minimize missed cron jobs
• Fix periodic jobs missing the "fallback" hour during DST changeover [#5049]

2.3.0

• Remove jQuery usage in UI tabs
• Pass exception to rate limiter backoff proc [#5024]

2.2.3

• Fixes for leaky and unlimited limiters [#4809, #4869]
• Invalid leaders now immediately step down [#4950]
• Web UI now displays "next run time" in the specified timezone [#4833]
• Fix swarm memory monitoring on BSDs

2.2.2

• Periodic job timezone fix [#4796]

2.2.1

• Support configurable timezones for periodic jobs [#4749]
• Handle edge case leading to negative expiry in uniqueness [#4763]

2.2.0

• Add new leaky bucket rate limiter [#4414] This allows clients to burst up to X calls before throttling back to X calls per Y seconds. To limit the user to 60 calls per minute:

leaker = Sidekiq::Limiter.leaky("shopify", 60, :minute)
leaker.within_limit do
  ...
end

See the Rate Limiting wiki page for more detail.

• Rate limiters may now customize their reschedule count [#4725] To disable rate limit reschedules, use reschedule: 0.

Sidekiq::Limiter.concurrent("somename", 5, reschedule: 0)

• Allow filtering by name in the Rate Limiter UI [#4695]
• Add IT locale

2.1.2

• The Sidekiq Pro and Enterprise gem servers now bundle install much faster with Bundler 2.2+ [#4158]
• Now that ActiveJobs support sidekiq_options, add support for uniqueness in AJs [#4667]

2.1.1

• Add optional app preload in swarm, saves even more memory [#4646]
• Fix incorrect queue tags in historical metrics [#4377]

2.1.0

• Move historical metrics to use tags rather than interpolating name [#4377]

sidekiq.enqueued.#{name} -> sidekiq.queue.size with tag queue:#{name}
sidekiq.latency.#{name} -> sidekiq.queue.latency with tag queue:#{name}

• Remove concurrent-ruby gem dependency [#4586]
• Add systemd Type=notify support for swarm [#4511]
• Length swarm's boot timeout to 60 sec [#4544]
• Add NL locale

2.0.1

• Periodic job registration API adjusted to avoid loading classes in initializer [#4271]
• Remove support for deprecated ENV variables (COUNT, MAXMEM_MB, INDEX) in swarm code

2.0.0

• Except for the newly required credentials, Sidekiq Enterprise 2.0 does not have any significant migration steps.
• Sidekiq Enterprise must now be started with valid license credentials. [#4232]
• Call GC.compact if possible in sidekiqswarm before forking [#4181]
• Changes for forward-compatibility with Sidekiq 6.0.
• Add death handler to remove any lingering unique locks [#4162]
• Backoff can now be customized per rate limiter [#4219]
• Code formatting changes for StandardRB

1.8.1

• Fix excessive lock reclaims with concurrent limiter [#4105]
• Add ES translations, see issues #3949 and #3951 to add your own language.

1.8.0

• Require Sidekiq Pro 4.0 and Sidekiq 5.2.
• Refactor historical metrics API to use revamped Statsd support in Sidekiq Pro
• Add a gauge to historical metrics for default queue latency [#4079]

1.7.2

• Add PT and JA translations
• Fix elapsed time calculations to use monotonic clock [#4000, sj26]
• Fix edge case where flapping leadership would cause old periodic jobs to be fired once [#3974]
• Add support for sidekiqswarm memory monitoring on FreeBSD [#3884]

1.7.1

• Fix Lua error in concurrent rate limiter under heavy contention
• Remove superfluous freeze calls on Strings [#3759]

1.7.0

• NEW FEATURE Rolling restarts - great for long running jobs!
• Adjust middleware so unique jobs that don't push aren't registered in a Batch [#3662]
• Add new unlimited rate limiter, useful for testing [#3743]

limiter = Sidekiq::Limiter.unlimited(...any args...)

1.6.1

• Fix crash in rate limiter middleware when used with custom exceptions [#3604]

1.6.0

• Show process "leader" tag on Busy page, requires Sidekiq 5.0.2 [#2867]
• Capture custom metrics with the save_history API. [#2815]
• Implement new unique_until: 'start' policy option. [#3471]

1.5.4

• Fix broken Cron page in Web UI [#3458]

1.5.3

• Remove dependency on the algorithms gem [#3446]
• Allow user to specify max memory in megabytes with SIDEKIQ_MAXMEM_MB [#3451]
• Implement logic to detect app startup failure, sidekiqswarm will exit rather than try to restart the app forever [#3450]
• Another fix for doubly-encrypted arguments [#3368]

1.5.2

• Fix encrypted arguments double-encrypted by retry or rate limiting [#3368]
• Fix leak in concurrent rate limiter, run this in Rails console to clean up existing data [#3323]

expiry = 1.month.to_i; Sidekiq::Limiter.redis { |c| c.scan_each(match: "lmtr-cfree-*") { |key| c.expire(key, expiry) } }

1.5.1

• Fix issue with census startup when not using Bundler configuration for source credentials.

1.5.0

• Add new web authorization API [#3251]
• Update all sidekiqswarm env vars to use SIDEKIQ_ prefix [#3218]
• Add census reporting, the leader will ping contribsys nightly with aggregate usage metrics

1.4.0

• No functional changes, require latest Sidekiq and Sidekiq Pro versions

1.3.2

• Upgrade encryption to use OpenSSL's more secure GCM mode. [#3060]

1.3.1

• Fix multi-process memory monitoring on CentOS 6.x [#3063]
• Polish the new encryption feature a bit.

1.3.0

• BETA New encryption feature which automatically encrypts the last argument of a Worker, aka the secret bag.

1.2.4

• Fix issue causing some minutely jobs to execute every other minute.
• Log a warning if slow periodic processing causes us to miss a clock tick.

1.2.3

• Periodic jobs could stop executing until process restart if Redis goes down [#3047]

1.2.2

• Add API to check if a unique lock is present. See [#2932] for details.
• Tune concurrent limiters to minimize thread thrashing under heavy contention. [#2944]
• Add option for tuning which Bundler groups get preloaded with sidekiqswarm [#3025]

SIDEKIQ_PRELOAD=default,production bin/sidekiqswarm ...
# Use an empty value for maximum application compatibility
SIDEKIQ_PRELOAD= bin/sidekiqswarm ...

1.2.1

• Multi-Process mode can now monitor the RSS memory of children and restart any that grow too large. To limit children to 1GB each:

MAXMEM_KB=1048576 COUNT=2 bundle exec sidekiqswarm ...

1.2.0

• NEW FEATURE Multi-process mode! Sidekiq Enterprise can now fork multiple worker processes, enabling significant memory savings. See the wiki documentation for details.

0.7.10

• More precise gemspec dependency versioning

1.1.0

• NEW FEATURE Historical queue metrics, documented in the wiki [#2719]

0.7.9, 1.0.2

• Window limiters can now accept arbitrary window sizes [#2686]
• Fix race condition in window limiters leading to non-stop OverLimit [#2704]
• Fix invalid overage counts when nesting concurrent limiters

1.0.1

• Fix crash in periodic subsystem when a follower shuts down, thanks to @justinko for reporting.

1.0.0

• Enterprise 1.x targets Sidekiq 4.x.
• Rewrite several features to remove Celluloid dependency. No functional changes.

0.7.8

• Fix unique_for: false [#2658]

0.7.7

• Enterprise 0.x targets Sidekiq 3.x.
• Fix racy shutdown event which could lead to disappearing periodic jobs, requires Sidekiq >= 3.5.3.
• Add new :leader event which is fired when a process gains leadership.

0.7.6

• Redesign how overrated jobs are rescheduled to avoid creating new jobs. [#2619]

0.7.5

• Fix dynamic creation of concurrent limiters [#2617]

0.7.4

• Add additional check to prevent duplicate periodic job creation
• Allow user-specified TTLs for rate limiters [#2607]
• Paginate rate limiter index page [#2606]

0.7.3

• Rework Sidekiq::Limiter redis handling to match global redis handling.
• Allow user to customize rate limit backoff logic and handle custom rate limit errors.
• Fix scalability issue with Limiter index page.

0.7.2

• Fix typo which prevented limiters with '0' in their names.

0.7.1

• Fix issue where unique scheduled jobs can't be enqueued upon schedule due to the existing unique lock. [#2499]

0.7.0

Initial release.