Yabeda::Sidekiq

Built-in metrics for Sidekiq monitoring out of the box! Part of the yabeda suite.

Sample Grafana dashboard ID: 11667

Installation

gem 'yabeda-sidekiq'
# Then add monitoring system adapter, e.g.:
# gem 'yabeda-prometheus'

And then execute:

$ bundle

If you're not on Rails then configure Yabeda after your application was initialized:

Yabeda.configure!

If you're using Ruby on Rails then Yabeda will configure itself automatically!

And that is it! Sidekiq metrics are being collected!

Additionally, depending on your adapter, you may want to setup metrics export. E.g. for yabeda-prometheus:

# config/initializers/sidekiq or elsewhere
Sidekiq.configure_server do |_config|
  Yabeda::Prometheus::Exporter.start_metrics_server!
end

Metrics

• Total number of executed jobs: sidekiq_jobs_executed_total - (segmented by queue and class name)
• Number of jobs have been finished successfully: sidekiq_jobs_success_total (segmented by queue and class name)
• Number of jobs have been failed: sidekiq_jobs_failed_total (segmented by queue and class name)
• Time of job run: sidekiq_job_runtime (seconds per job execution, segmented by queue and class name)
• Time of the queue latency sidekiq_queue_latency (the difference in seconds since the oldest job in the queue was enqueued)
• Time of the job latency sidekiq_job_latency (the difference in seconds since the enqueuing until running job)
• Number of jobs in queues: sidekiq_jobs_waiting_count (segmented by queue)
• Number of scheduled jobs:sidekiq_jobs_scheduled_count
• Number of jobs in retry set: sidekiq_jobs_retry_count
• Number of jobs in dead set (“morgue”): sidekiq_jobs_dead_count
• Active workers count: sidekiq_active_processes
• Active processes count: sidekiq_active_workers_count

Custom tags

You can add additional tags to these metrics by declaring yabeda_tags method in your worker.

# This block is optional but some adapters (like Prometheus) requires that all tags should be declared in advance
Yabeda.configure do
  default_tag :importance, nil
end

class MyWorker
  include Sidekiq::Worker

  def yabeda_tags(*params) # This method will be called first, before +perform+
    { importance: extract_importance(params) }
  end

  def perform(*params)
    # Your logic here
  end
end

Roadmap (TODO or Help wanted)

• Implement optional segmentation of retry/schedule/dead sets

It should be disabled by default as it requires to iterate over all jobs in sets and may be very slow on large sets.

• Maybe add some hooks for ease of plugging in metrics for myriads of Sidekiq plugins?

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/yabeda-rb/yabeda-sidekiq.

License

The gem is available as open source under the terms of the MIT License.