Preface

This is a project that is currently used in production environments but isn't “production ready” unless you're adventurous. Consider this as a “sneak peak” rather than a “first release”.

Another thing to note… this has gone through many iterations (MySQL backed, memcached backed, rabbitMQ, etc). At some point, I plan on adding back in support for all of these and allowing users to choose which queue backend they want, but right now it's tied directly to rabbitmq.

Summary

ActionEvent lets you “do something later”. Many things can be taken out of the front-end flow and moved to an asynchronous task, speeding up the overall user experience. LivingSocial has over a hundred event pollers spread out on five machines processing millions of events a day.

ActionEvent allows you to:

* Queue an event from within a rails application to be processed asynchronously.
* Access your full rails environment from event processor.
* Process events across any number of physical machines (there is no master process).
* Start/stop processors on the fly without interrupting anything (think: cloud computing).
* Events support inheritance and a simple before_filter/skip_before_filter chain as well.

Queueing an Event

You can queue up as many events you want from within a rails application using the “queue_action_event” method available on all objects. This takes an event name and an optional hash of parameters. The name and parameters are serialized and stored as an active record object. You can also prioritize events (default priorities are :high, :medium and :low). Pollers will always look for events from the highest priority first, so all :high messages will be processed before a single :medium message is processed.

Queueing examples:

queue_action_event(:some_event)
queue_action_event(:some_event, :people_ids => [1,2,3])
queue_action_event(:send_email, :queue => :high)

Pollers: Processing an Event

You need to have one or more “pollers” which will process events as they come in the queue. Pollers will continuously query the database trying to get another event to process. If it finds one, it will try to take “ownership” of the event so that no other pollers will process the event. If this fails (meaning another poller already took the message), it moves on. If it succeeds and takes ownership of the message, it will process the event through the corresponding app/events/*_event.rb class.

If you have multiple pollers (likely), you can start and stop them as a daemon. Each poller needs a unique ID so it can keep track of their PID file and stop and start gracefully. After processing every message, a daemonized poller will check their PID file to see if it matches their current process id. If it doesn't match, the poller will stop gracefully. This allows you to start up a new poller on updated code without waiting for or interrupting an existing poller if it happens to be in the middle of processing a message.

Poller examples:

./script/poller
./script/poller -d start
./script/poller --daemon --id=2 start
./script/poller --daemon --id=2 --queues="high medium" start

Example

$ ./script/generate event HelloWorld

exists  db/migrate
create  db/migrate/20090207172934_create_action_event_tables.rb
create  script/poller
create  app/events
create  app/events/application_event.rb
exists  app/events/
create  app/events/hello_world_event.rb

# NOTE: running this migration is only necessary the very first time you create an action_event $ rake db:migrate

CreateActionEventTables: migrating ========================================

– create_table(:action_event_messages)

-> 0.0844s

– create_table(:action_event_statuses)

-> 0.0359s

CreateActionEventTables: migrated (0.1212s) ===============================

$ cat > app/events/hello_world_event.rb

class HelloWorldEvent < ApplicationEvent
  def process
    puts "hello #{params[:name]}"
  end
end

$ ./script/runner “queue_action_event('hello_world', :name => 'world')”

$ ./script/poller

Sat Feb 07 12:41:43 -0500 2009

Processing queues: high,medium,low

Sat Feb 07 12:44:31 -0500 2009

Processing medium:1 (:event=>“hello_world”)

hello world

Sat Feb 07 12:44:31 -0500 2009

Finished processing medium:1 (:event=>“hello_world”)

Copyright © 2009 Warren Konkel, released under the MIT license