SweatShop

SweatShop provides an api to background resource intensive tasks. Much of the api design was copied from Workling, with a few tweaks. Currently, it runs rabbitmq and kestrel, but it can support any number of queues.

Installing

gem install sweat_shop
freeze in your rails directory
cd vendor/gems/sweat_shop
rake setup

Writing workers

Put email_worker.rb into app/workers and sublcass SweatShop::Worker:

class EmailWorker
  def send_mail(to)
    user = User.find_by_id(to)
    Mailer.deliver_welcome(to)
  end
end

Then, anywhere in your app you can execute:

EmailWorker.async_send_mail(1)

The async signifies that this task will be placed on a queue to be serviced by the EmailWorker possibly on another machine. You can also call:

EmailWorker.send_mail(1)

That will do the work immediately, without placing the task on the queue. You can also define a queue_group at the top of the file which will allow you to split workers out into logical groups. This is important if you have various machines serving different queues.

Running the queue

SweatShop has been tested with Rabbit and Kestrel, but it will also work with Starling. Please use the following resources to install the server:

Kestrel: http://github.com/robey/kestrel/tree/master

Rabbit: http://github.com/ezmobius/nanite/tree/master

config/sweatshop.yml specifies the machine address of the queue (default localhost:5672). You can also specify the queue type with the queue param.

Running the workers

Assuming you ran rake setup in Rails, you can type:

script/sweatshop

By default, the script will run all workers defined in the app/workers dir. Every task will be processed on each queue using a round-robin algorithm. You can also add the -d flag which will put the worker in daemon mode. The daemon also takes other params. Add a -h for more details.

script/sweatshop -d
script/sweatshop -d stop

If you would like to run SweatShop as a daemon on a linux machine, use the initd.sh script provided in the sweat_shop/script dir.

Serializers

Serializers can be swapped out on a per worker basis and set globally to a default via the config file.

The default serializer is Marshal to maintain backward compatibility with existing apps. Three serializers are available out of the box, Marshal, JSON & YAML.

To change the default serializer, add a key [serializer] to the config file with a value of %wjson yaml or your own custom serializer.

You may also write your own custom serializer classes by extending SweatShop::Serializer.

You can give your serializers an arbitrary class name, but the default pattern is to name it Serializer. Notice you can specify the Marshal, JSON or YAML serializer with the name :marshal, :json, or :yaml. This default short name is generated by underscoring the class name and removing the "Serializer" from the end of the class name, i.e. SweatShop::Serializers::JsonSerializer becomes :json and My::Custom::EncryptedSerializer would become :encrypted. You can override this default generated name by using the declaration in the top of your class: serializer_name :foo

Where :foo is what you want to register the serializer as.

This is a complete example of how to write a custom serializer:

class MyCustomSerializer < SweatShop::Serializer serializer_name :foo # overrides the generated default name of :my_custom based on the above rules

  def self.serialize(data)
    ... # custom serialize the data
  end

  def self.deserialize(data)
    ... # custom deserialize the data
  end

end

REQUIREMENTS

i_can_daemonize
memcache (for kestrel)
carrot (for rabbit)