Redis::Objects - Map Redis types directly to Ruby objects
This is not an ORM. People that are wrapping ORM’s around Redis are missing the point.
The killer feature of Redis that it allows you to perform atomic operations on individual data structures, like counters, lists, and sets. You can then use these with your existing ActiveRecord/DataMapper/etc models, or in classes that have nothing to do with an ORM or even a database. This gem maps Redis types to Ruby objects, via a thin layer over Ezra’s redis
gem.
This gem originally arose out of a need for high-concurrency atomic operations; for a fun rant on the topic, see ATOMICITY, or scroll down to “Atomicity” in this README.
There are two ways to use Redis::Objects, either as an include
in a model class, or by using new
with the type of data structure you want to create.
Installation
gem install gemcutter
gem tumble
gem install redis-objects
Example 1: Model Class Usage
Using Redis::Objects this way makes it trivial to integrate Redis types with an existing ActiveRecord model, DataMapper resource, or other class. Redis::Objects will work with any class that provides an id
method that returns a unique value. Redis::Objects will automatically create keys that are unique to each object.
Initialization
Redis::Objects needs a handle created by Redis.new. If you’re on Rails, config/initializers/redis.rb is a good place for this:
require 'redis'
require 'redis/objects'
Redis::Objects.redis = Redis.new(:host => 127.0.0.1, :port => 6379)
You can use Redis::Objects with any framework. There are no dependencies on Rails. I use it from Sinatra and rake tasks all the time.
Model Class
Include Redis::Objects in any type of class:
class Team < ActiveRecord::Base
include Redis::Objects
counter :hits
counter :runs
counter :outs
counter :inning, :start => 1
list :on_base
set :outfielders
value :at_bat
end
Familiar Ruby array operations Just Work (TM):
@team = Team.find_by_name('New York Yankees')
@team.on_base << 'player1'
@team.on_base << 'player2'
@team.on_base << 'player3'
@team.on_base # ['player1', 'player2', 'player3']
@team.on_base.pop
@team.on_base.shift
@team.on_base.length # 1
@team.on_base.delete('player2')
Sets work too:
@team.outfielders << 'outfielder1'
@team.outfielders << 'outfielder2'
@team.outfielders << 'outfielder1' # dup ignored
@team.outfielders # ['outfielder1', 'outfielder2']
@team.outfielders.each do |player|
puts player
end
player = @team.outfielders.detect{|of| of == 'outfielder2'}
And you can do intersections between ORM objects (kinda cool):
@team1.outfielders | @team2.outfielders # outfielders on both teams
@team1.outfielders & @team2.outfielders # in baseball, should be empty :-)
Counters can be atomically incremented/decremented (but not assigned):
@team.hits.increment # or incr
@team.hits.decrement # or decr
@team.hits.incr(3) # add 3
@team.runs = 4 # exception
Finally, for free, you get a redis
method that points directly to a Redis connection:
Team.redis.get('somekey')
@team = Team.new
@team.redis.get('somekey')
@team.redis.smembers('someset')
You can use the redis
handle to directly call any Redis command
Example 2: Standalone Usage
There is a Ruby object that maps to each Redis type.
Initialization
Again, Redis::Objects needs a handle to the redis
server. For standalone use, you can either set the $redis global variable:
$redis = Redis.new(:host => 'localhost', :port => 6379)
@value = Redis::Value.new('myvalue')
Or you can pass the Redis handle into the new method for each type:
redis = Redis.new(:host => 'localhost', :port => 6379)
@value = Redis::Value.new('myvalue', redis)
Your choice.
Counters
Create a new counter. The counter_name
is the key stored in Redis.
require 'redis/counter'
@counter = Redis::Counter.new('counter_name')
@counter.increment
@counter.decrement
puts @counter.value
This gem provides a clean way to do atomic blocks as well:
@counter.increment do |val|
raise "Full" if val > MAX_VAL # rewind counter
end
See the section on “Atomicity” for cool uses of atomic counter blocks.
Lists
Lists work just like Ruby arrays:
require 'redis/list'
@list = Redis::List.new('list_name')
@list << 'a'
@list << 'b'
@list.include? 'c' # false
@list.values # ['a','b']
@list << 'c'
@list.delete('c')
@list[0]
@list[0,1]
@list[0..1]
@list.shift
@list.pop
@list.clear
# etc
Complex data types are no problem:
@list << {:name => "Nate", :city => "San Diego"}
@list << {:name => "Peter", :city => "Oceanside"}
@list.each do |el|
puts "#{el[:name]} lives in #{el[:city]}"
end
Sets
Sets work like the Ruby Set class:
require 'redis/set'
@set = Redis::Set.new('set_name')
@set << 'a'
@set << 'b'
@set << 'a' # dup ignored
@set.member? 'c' # false
@set.members # ['a','b']
@set.each do |member|
puts member
end
@set.clear
# etc
You can perform Redis intersections/unions/diffs easily:
@set1 = Redis::Set.new('set1')
@set2 = Redis::Set.new('set2')
@set3 = Redis::Set.new('set3')
members = @set1 & @set2 # intersection
members = @set1 | @set2 # union
members = @set1 + @set2 # union
members = @set1 ^ @set2 # difference
members = @set1 - @set2 # difference
members = @set1.intersection(@set2, @set3) # multiple
members = @set1.union(@set2, @set3) # multiple
members = @set1.difference(@set2, @set3) # multiple
Or store them in Redis:
@set1.interstore('intername', @set2, @set3)
members = @set1.redis.get('intername')
@set1.unionstore('unionname', @set2, @set3)
members = @set1.redis.get('unionname')
@set1.diffstore('diffname', @set2, @set3)
members = @set1.redis.get('diffname')
And use complex data types too:
@set1 << {:name => "Nate", :city => "San Diego"}
@set1 << {:name => "Peter", :city => "Oceanside"}
@set2 << {:name => "Nate", :city => "San Diego"}
@set2 << {:name => "Jeff", :city => "Del Mar"}
@set1 & @set2 # Nate
@set1 - @set2 # Peter
@set1 | @set2 # all 3 people
Values
Simple values are easy as well:
require 'redis/value'
@value = Redis::Value.new('value_name')
@value.value = 'a'
@value.delete
Of course complex data is no problem:
@account = Account.create!(params[:account])
@newest = Redis::Value.new('newest_account')
@newest.value = @account
Atomic Counters and Locks
You are probably not handling atomicity correctly in your app. For a fun rant on the topic, see ATOMICITY.
Atomic counters are a good way to handle concurrency:
@team = Team.find(1)
if @team.drafted_players.increment <= @team.max_players
# do stuff
@team.team_players.create!(:player_id => 221)
@team.active_players.increment
else
# reset counter state
@team.drafted_players.decrement
end
Atomic block - a cleaner way to do the above. Exceptions or return nil rewind counter back to previous state:
@team.drafted_players.increment do |val|
raise Team::TeamFullError if val > @team.max_players
@team.team_players.create!(:player_id => 221)
@team.active_players.increment
end
Similar approach, using an if block (failure rewinds counter):
@team.drafted_players.increment do |val|
if val <= @team.max_players
@team.team_players.create!(:player_id => 221)
@team.active_players.increment
end
end
Class methods work too - notice we override ActiveRecord counters:
Team.increment_counter :drafted_players, team_id
Team.decrement_counter :drafted_players, team_id, 2
Team.increment_counter :total_online_players # no ID on global counter
Class-level atomic block (may save a DB fetch depending on your app):
Team.increment_counter(:drafted_players, team_id) do |val|
TeamPitcher.create!(:team_id => team_id, :pitcher_id => 181)
Team.increment_counter(:active_players, team_id)
end
Locks with Redis. On completion or exception the lock is released:
class Team < ActiveRecord::Base
lock :reorder # declare a lock
end
@team.reorder_lock.lock do
@team.reorder_all_players
end
Class-level lock (same concept)
Team.obtain_lock(:reorder, team_id) do
Team.reorder_all_players(team_id)
end
Lock expiration. Sometimes you want to make sure your locks are cleaned up should the unthinkable happen (server failure). You can set lock expirations to handle this. Expired locks are released by the next process to attempt lock. Just make sure you expiration value is sufficiently large compared to your expected lock time.
class Team < ActiveRecord::Base
lock :reorder, :expiration => 15.minutes
end
Author
Copyright © 2009-2010 Nate Wiger. All Rights Reserved. Released under the Artistic License.