Module: Sidekiq::Throttled

Defined in:
lib/sidekiq/throttled.rb,
lib/sidekiq/throttled/web.rb,
lib/sidekiq/throttled/fetch.rb,
lib/sidekiq/throttled/errors.rb,
lib/sidekiq/throttled/worker.rb,
lib/sidekiq/throttled/version.rb,
lib/sidekiq/throttled/registry.rb,
lib/sidekiq/throttled/strategy.rb,
lib/sidekiq/throttled/web/stats.rb,
lib/sidekiq/throttled/middleware.rb,
lib/sidekiq/throttled/queue_name.rb,
lib/sidekiq/throttled/communicator.rb,
lib/sidekiq/throttled/patches/queue.rb,
lib/sidekiq/throttled/queues_pauser.rb,
lib/sidekiq/throttled/strategy/base.rb,
lib/sidekiq/throttled/expirable_list.rb,
lib/sidekiq/throttled/web/summary_fix.rb,
lib/sidekiq/throttled/fetch/unit_of_work.rb,
lib/sidekiq/throttled/strategy/threshold.rb,
lib/sidekiq/throttled/strategy/concurrency.rb,
lib/sidekiq/throttled/communicator/listener.rb,
lib/sidekiq/throttled/communicator/callbacks.rb

Overview

Concurrency and threshold throttling for Sidekiq.

Just add somewhere in your bootstrap:

require "sidekiq/throttled"
Sidekiq::Throttled.setup!

Once you've done that you can include Worker to your job classes and configure throttling:

class MyWorker
  include Sidekiq::Worker
  include Sidekiq::Throttled::Worker

  sidekiq_options :queue => :my_queue

  sidekiq_throttle({
    # Allow maximum 10 concurrent jobs of this class at a time.
    :concurrency => { :limit => 10 },
    # Allow maximum 1K jobs being processed within one hour window.
    :threshold => { :limit => 1_000, :period => 1.hour }
  })

  def perform
    # ...
  end
end

Defined Under Namespace

Modules: Patches, QueueName, Registry, Web, Worker Classes: Communicator, Error, ExpirableList, Fetch, Middleware, QueuesPauser, Strategy

Constant Summary collapse

VERSION =

Gem version

"0.8.1"

Class Method Summary collapse

Class Method Details

.setup!void

This method returns an undefined value.

Hooks throttler into sidekiq.



53
54
55
56
57
58
59
60
61
62
63
64
65
66
# File 'lib/sidekiq/throttled.rb', line 53

def setup!
  Communicator.instance.setup!
  QueuesPauser.instance.setup!

  Sidekiq.configure_server do |config|
    require "sidekiq/throttled/fetch"
    Sidekiq.options[:fetch] = Sidekiq::Throttled::Fetch

    require "sidekiq/throttled/middleware"
    config.server_middleware do |chain|
      chain.add Sidekiq::Throttled::Middleware
    end
  end
end

.throttled?(message) ⇒ Boolean

Tells whenever job is throttled or not.

Parameters:

  • message (String)

    Job's JSON payload

Returns:

  • (Boolean)


72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
# File 'lib/sidekiq/throttled.rb', line 72

def throttled?(message)
  message = JSON.parse message
  job = message.fetch("class") { return false }
  jid = message.fetch("jid") { return false }

  preload_constant! job

  Registry.get job do |strategy|
    return strategy.throttled?(jid, *message["args"])
  end

  false
rescue
  false
end