sidekiq-worker-killer
Sidekiq is probably the best background processing framework today. At the same time, memory leaks are very hard to tackle in Ruby and we often find ourselves with growing memory consumption. Instead of spending herculean effort fixing leaks, why not kill your processes when they got to be too large?
Highly inspired by Gitlab Sidekiq MemoryKiller and Noxa Sidekiq killer.
quick-refs: install | usage | available options | development
Install
Use Bundler
gem "sidekiq-worker-killer"
Usage
Add this to your Sidekiq configuration.
require 'sidekiq/worker_killer'
Sidekiq.configure_server do |config|
config.server_middleware do |chain|
chain.add Sidekiq::WorkerKiller, max_rss: 480
end
end
Available options
The following options can be overrided.
Option | Defaults | Description |
---|---|---|
max_rss | 0 MB (disabled) | max RSS in megabytes. Above this, shutdown will be triggered. |
grace_time | 900 seconds | when shutdown is triggered, the Sidekiq process will not accept new job and wait at most 15 minutes for running jobs to finish. If Float::INFINITY specified, will wait forever. |
shutdown_wait | 30 seconds | when the grace time expires, still running jobs get 30 seconds to terminate. After that, kill signal is triggered. |
kill_signal | SIGKILL | Signal to use kill Sidekiq process if it doesn't terminate. |
gc | true | Try to run garbage collection before Sidekiq process terminate in case of max_rss exceeded. |
skip_shutdown_if | Proc.new false | Executes a block of code after max_rss exceeds but before requesting shutdown. |
skip_shutdown_if is expected to return anything other than false
or nil
to skip shutdown.
require 'sidekiq/worker_killer'
Sidekiq.configure_server do |config|
config.server_middleware do |chain|
chain.add Sidekiq::WorkerKiller, max_rss: 480, skip_shutdown_if: ->(worker, job, queue) do
worker.to_s == 'LongWorker'
end
end
end
Development
Pull Requests are very welcome!
There are tasks that may help you along the way in a makefile:
make console # Loads the whole stack in an IRB session.
make test # Run tests.
make lint # Run rubocop linter.
Please make sure that you have tested your code carefully before opening a PR, and make sure as well that you have no style issues.
Authors
See the list of contributors who participated in this project.
License
Please see LICENSE