Module: Sidekiq

Defined in:
lib/sidekiq/api.rb,
lib/sidekiq.rb,
lib/sidekiq/cli.rb,
lib/sidekiq/job.rb,
lib/sidekiq/web.rb,
lib/sidekiq/fetch.rb,
lib/sidekiq/rails.rb,
lib/sidekiq/client.rb,
lib/sidekiq/config.rb,
lib/sidekiq/deploy.rb,
lib/sidekiq/loader.rb,
lib/sidekiq/logger.rb,
lib/sidekiq/capsule.rb,
lib/sidekiq/manager.rb,
lib/sidekiq/systemd.rb,
lib/sidekiq/testing.rb,
lib/sidekiq/version.rb,
lib/sidekiq/embedded.rb,
lib/sidekiq/job_util.rb,
lib/sidekiq/launcher.rb,
lib/sidekiq/profiler.rb,
lib/sidekiq/component.rb,
lib/sidekiq/job_retry.rb,
lib/sidekiq/paginator.rb,
lib/sidekiq/processor.rb,
lib/sidekiq/scheduled.rb,
lib/sidekiq/sd_notify.rb,
lib/sidekiq/job_logger.rb,
lib/sidekiq/web/action.rb,
lib/sidekiq/web/config.rb,
lib/sidekiq/web/router.rb,
lib/sidekiq/ring_buffer.rb,
lib/sidekiq/web/helpers.rb,
lib/sidekiq/iterable_job.rb,
lib/sidekiq/job/iterable.rb,
lib/sidekiq/metrics/query.rb,
lib/sidekiq/metrics/shared.rb,
lib/sidekiq/web/application.rb,
lib/sidekiq/metrics/tracking.rb,
lib/sidekiq/middleware/chain.rb,
lib/sidekiq/redis_connection.rb,
lib/sidekiq/middleware/modules.rb,
lib/sidekiq/web/csrf_protection.rb,
lib/sidekiq/redis_client_adapter.rb,
lib/sidekiq/job/interrupt_handler.rb,
lib/generators/sidekiq/job_generator.rb,
lib/sidekiq/job/iterable/enumerators.rb,
lib/sidekiq/transaction_aware_client.rb,
lib/sidekiq/transaction_aware_client.rb,
lib/sidekiq/worker_compatibility_alias.rb,
lib/sidekiq/job/iterable/csv_enumerator.rb,
lib/sidekiq/middleware/current_attributes.rb,
lib/sidekiq/job/iterable/active_record_enumerator.rb

Overview

Use ‘Sidekiq.transactional_push!` in your sidekiq.rb initializer

Defined Under Namespace

Modules: Component, Context, CurrentAttributes, Generators, IterableJob, Job, JobUtil, Metrics, Middleware, Paginator, Queues, RedisConnection, Scheduled, SdNotify, ServerMiddleware, TestingClient, TestingExtensions, WebHelpers Classes: BasicFetch, CLI, Capsule, Client, Config, DeadSet, Deploy, Embedded, EmptyQueueError, JobLogger, JobRecord, JobRetry, JobSet, Launcher, Loader, Logger, Manager, Monitor, Process, ProcessSet, Processor, ProfileRecord, ProfileSet, Profiler, Queue, RedisClientAdapter, RetrySet, RingBuffer, ScheduledSet, Shutdown, SortedEntry, SortedSet, Stats, Testing, TransactionAwareClient, Web, Work, WorkSet

Constant Summary collapse

NAME = 
"Sidekiq"
LICENSE = 
"See LICENSE and the LGPL-3.0 for licensing details."
Workers =

Since “worker” is a nebulous term, we’ve deprecated the use of this class name. Is “worker” a process, a type of job, a thread? Undefined! WorkSet better describes the data.

WorkSet
VERSION = 
"8.0.8"
MAJOR = 
8
DEFAULT_THREAD_PRIORITY =

Ruby’s default thread priority is 0, which uses 100ms time slices. This can lead to some surprising thread starvation; if using a lot of CPU-heavy concurrency, it may take several seconds before a Thread gets on the CPU.

Negative priorities lower the timeslice by half, so -1 = 50ms, -2 = 25ms, etc. With more frequent timeslices, we reduce the risk of unintentional timeouts and starvation.

Customize like so:

Sidekiq.configure_server do |cfg|
  cfg.thread_priority = 0
end

-1
ClientMiddleware =

no difference for now

ServerMiddleware
Worker =

Sidekiq::Job is a new alias for Sidekiq::Worker as of Sidekiq 6.3.0. Use ‘include Sidekiq::Job` rather than `include Sidekiq::Worker`.

The term “worker” is too generic and overly confusing, used in several different contexts meaning different things. Many people call a Sidekiq process a “worker”. Some people call the thread that executes jobs a “worker”. This change brings Sidekiq closer to ActiveJob where your job classes extend ApplicationJob.

Job

Class Method Summary collapse

Class Method Details

.configure_client {|default_configuration| ... } ⇒ Object

Yields:



141
142
143
 
# File 'lib/sidekiq.rb', line 141

def self.configure_client
  yield default_configuration unless server?
end

.configure_embed {|cfg| ... } ⇒ Object

Creates a Sidekiq::Config instance that is more tuned for embedding within an arbitrary Ruby process. Notably it reduces concurrency by default so there is less contention for CPU time with other threads.

instance = Sidekiq.configure_embed do |config|
  config.queues = %w[critical default low]
end
instance.run
sleep 10
instance.stop

NB: it is really easy to overload a Ruby process with threads due to the GIL. I do not recommend setting concurrency higher than 2-3.

NB: Sidekiq only supports one instance in memory. You will get undefined behavior if you try to embed Sidekiq twice in the same process.

Yields:

  • (cfg) 


129
130
131
132
133
134
135
136
137
138
139
 
# File 'lib/sidekiq.rb', line 129

def self.configure_embed(&block)
  raise "Sidekiq global configuration is frozen, you must create all embedded instances BEFORE calling `run`" if @frozen

  require "sidekiq/embedded"
  cfg = default_configuration
  cfg.concurrency = 2
  @config_blocks&.each { |block| block.call(cfg) }
  yield cfg

  Sidekiq::Embedded.new(cfg)
end

.configure_server {|default_configuration| ... } ⇒ Object

Yields:



102
103
104
105
 
# File 'lib/sidekiq.rb', line 102

def self.configure_server(&block)
  (@config_blocks ||= []) << block
  yield default_configuration if server?
end

.default_configuration ⇒ Object 



90
91
92
 
# File 'lib/sidekiq.rb', line 90

def self.default_configuration
  @config ||= Sidekiq::Config.new
end

.default_job_options ⇒ Object 



86
87
88
 
# File 'lib/sidekiq.rb', line 86

def self.default_job_options
  @default_job_options ||= {"retry" => true, "queue" => "default"}
end

.default_job_options=(hash) ⇒ Object 



82
83
84
 
# File 'lib/sidekiq.rb', line 82

def self.default_job_options=(hash)
  @default_job_options = default_job_options.merge(hash.transform_keys(&:to_s))
end

.dump_json(object) ⇒ Object 



58
59
60
 
# File 'lib/sidekiq.rb', line 58

def self.dump_json(object)
  JSON.generate(object)
end

.ent? ⇒ Boolean

Returns:

  • (Boolean) 


66
67
68
 
# File 'lib/sidekiq.rb', line 66

def self.ent?
  defined?(Sidekiq::Enterprise)
end

.freeze! ⇒ Object 



107
108
109
110
111
 
# File 'lib/sidekiq.rb', line 107

def self.freeze!
  @frozen = true
  @config_blocks = nil
  default_configuration.freeze!
end

.gem_version ⇒ Object 



7
8
9
 
# File 'lib/sidekiq/version.rb', line 7

def self.gem_version
  Gem::Version.new(VERSION)
end

.load_json(string) ⇒ Object 



54
55
56
 
# File 'lib/sidekiq.rb', line 54

def self.load_json(string)
  JSON.parse(string)
end

.loader ⇒ Object 



98
99
100
 
# File 'lib/sidekiq.rb', line 98

def self.loader
  @loader ||= Loader.new
end

.logger ⇒ Object 



94
95
96
 
# File 'lib/sidekiq.rb', line 94

def self.logger
  default_configuration.logger
end

.pro? ⇒ Boolean

Returns:

  • (Boolean) 


62
63
64
 
# File 'lib/sidekiq.rb', line 62

def self.pro?
  defined?(Sidekiq::Pro)
end

.redis(&block) ⇒ Object 



74
75
76
 
# File 'lib/sidekiq.rb', line 74

def self.redis(&block)
  (Thread.current[:sidekiq_capsule] || default_configuration).redis(&block)
end

.redis_pool ⇒ Object 



70
71
72
 
# File 'lib/sidekiq.rb', line 70

def self.redis_pool
  (Thread.current[:sidekiq_capsule] || default_configuration).redis_pool
end

.server? ⇒ Boolean

Returns:

  • (Boolean) 


50
51
52
 
# File 'lib/sidekiq.rb', line 50

def self.server?
  defined?(Sidekiq::CLI)
end

.start_watchdog ⇒ Object 



10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
 
# File 'lib/sidekiq/systemd.rb', line 10

def self.start_watchdog
  usec = Integer(ENV["WATCHDOG_USEC"])
  return Sidekiq.logger.error("systemd Watchdog too fast: " + usec) if usec < 1_000_000

  sec_f = usec / 1_000_000.0
  # "It is recommended that a daemon sends a keep-alive notification message
  # to the service manager every half of the time returned here."
  ping_f = sec_f / 2
  Sidekiq.logger.info "Pinging systemd watchdog every #{ping_f.round(1)} sec"
  Thread.new do
    loop do
      sleep ping_f
      Sidekiq::SdNotify.watchdog
    end
  end
end

.strict_args!(mode = :raise) ⇒ Object 



78
79
80
 
# File 'lib/sidekiq.rb', line 78

def self.strict_args!(mode = :raise)
  Sidekiq::Config::DEFAULTS[:on_complex_arguments] = mode
end

.transactional_push! ⇒ Object 



46
47
48
49
50
51
52
53
54
55
56
57
58
 
# File 'lib/sidekiq/transaction_aware_client.rb', line 46

def self.transactional_push!
  if ActiveRecord.version < Gem::Version.new("7.2")
    begin
      require "after_commit_everywhere"
    rescue LoadError
      raise %q(You need ActiveRecord >= 7.2 or to add `gem "after_commit_everywhere"` to your Gemfile to use Sidekiq's transactional client)
    end
  end

  Sidekiq.default_job_options["client_class"] = Sidekiq::TransactionAwareClient
  Sidekiq::JobUtil::TRANSIENT_ATTRIBUTES << "client_class"
  true
end

.❨╯°□°❩╯︵┻━┻ ⇒ Object 



46
47
48
 
# File 'lib/sidekiq.rb', line 46

def self.❨╯°□°❩╯︵┻━┻
  puts "Take a deep breath and count to ten..."
end