Disc

Disc fills the gap between your Ruby service objects and antirez's wonderful Disque backend.

Usage

1. Install the gem

  $ gem install disc

1. Write your jobs

  require 'disc'

  class CreateGameGrid
    include Disc::Job
    disc queue: 'urgent'

    def perform(type)
      # perform rather lengthy operations here.
    end
  end

1. Enqueue them to perform them asynchronously

  CreateGameGrid.enqueue('light_cycle')

1. Or enqueue them to be performed at some time in the future.

  CreateGameGrid.enqueue_at(DateTime.new(2015, 12, 31), 'disc_arena')

1. Create a file that requires anything needed for your jobs to run

# disc_init.rb
  require 'ohm'
  Dir['./jobs/**/*.rb'].each { |job| require job }

1. Run as many Disc Worker processes as you wish.

  $ QUEUES=urgent,default disc -r ./disc_init.rb

Settings

Disc takes its configuration from environment variables.

ENV Variable	Default Value	Description
QUEUES	'default'	The list of queues that Disc::Worker will listen to, it can be a single queue name or a list of comma-separated queues
DISC_CONCURRENCY	'25'	Amount of threads to spawn when Celluloid is available.
DISQUE_NODES	'localhost:7711'	This is the list of Disque servers to connect to, it can be a single node or a list of comma-separated nodes
DISQUE_AUTH	''	Authorization credentials for Disque.
DISQUE_TIMEOUT	'100'	Time in milliseconds that the client will wait for the Disque server to acknowledge and replicate a job
DISQUE_CYCLE	'1000'	The client keeps track of which nodes are providing more jobs, after the amount of operations specified in cycle it tries to connect to the preferred node.

Error handling

When a job raises an exception, Disc.on_error is invoked with the error and the job data. By default, this method prints the error to standard error, but you can override it to report the error to your favorite error aggregator.

# On disc_init.rb
def Disc.on_error(exception, job)
  # ... report the error
end

Dir["./jobs/**/*.rb"].each { |job| require job }

Job Definition

The error handler function gets the data of the current job as a Hash, that has the following schema.

'class'	(String) The Job class.
'arguments'	(Array) The arguments passed to perform.
'queue'	(String) The queue from which this job was picked up.
'id'	(String) Disque's job ID.

PowerUps

Disc workers can run just fine on their own, but if you're using Celluloid you might want Disc to take advantage of it and spawn multiple worker threads per process, doing this is trivial! Just require Celluloid before your init file:

$ QUEUES=urgent,default disc -r celluloid/current -r ./disc_init.rb

Whenever Disc detects that Celluloid is available it will use it to spawn a number of threads equal to the DISC_CONCURRENCY environment variable, or 25 by default.

Rails and ActiveJob integration

Disc has an ActiveJob adapter, which you can use very easily:

# Gemfile
gem 'disc'

# config/application.rb
module YourApp
  class Application < Rails::Application
    require 'active_job/queue_adapters/disc_adapter'
    config.active_job.queue_adapter = :disc
  end
end

# app/jobs/clu_job.rb

class CluJob < ActiveJob::Base
  queue_as :urgent

  def perform(*args)
    # Try to take over The Grid here...
  end
end

# Wherever you want
CluJob.perform_later(a_bunch_of_arguments)

As always, make sure your disc_init.rb file requires the necessary jobs and you'll be good to go!

License

The code is released under an MIT license. See the LICENSE file for more information.

Acknowledgements

• To @godfoca for helping me ship a quality thing and putting up with my constant whining.
• To @antirez for Redis, Disque, and his refreshing way of programming wonderful tools.
• To @soveran for pushing me to work on this and publishing gems that keep me enjoying ruby.
• To all contributors